Facility cucumber growth visualization method and apparatus based on multi-modal time-series data

Abstract

Embodiments of the present disclosure disclose a facility cucumber growth visualization method and device based on multi-modal time series data. A specific embodiment of the method comprises: preprocessing a collected phenotype image sequence and a corresponding environmental parameter sequence in a facility cucumber generation process to generate a preprocessed phenotype image sequence and a preprocessed environmental parameter sequence; performing feature extraction on the preprocessed phenotype image sequence and the preprocessed environmental parameter sequence to generate a fused feature vector sequence; constructing a growth time series data model; performing smoothing processing and change point detection on key growth parameters of the facility cucumber to generate a plurality of growth stage identifiers and a plurality of corresponding growth state parameter sets; dynamically generating a visualization interface of the facility cucumber according to the plurality of growth stage identifiers and the plurality of growth state parameter sets; and in response to receiving an interactive instruction of a target user, updating the visualization interface according to the interactive instruction. The embodiment can realize interactive dynamic display of the cucumber growth process.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to a method and apparatus for visualizing the growth of facility cucumbers based on multimodal time-series data. Background Technology

[0002] Currently, in the process of greenhouse cucumber cultivation, existing technologies mostly use independent charts to display growth data or simulate the growth process through simple animations.

[0003] However, when using the above methods to display the growth process of cucumbers in facilities, the following technical problems often exist: existing technologies are difficult to realize the mining of growth patterns and dynamic interactive visualization based on time-series data, resulting in a lack of realism in the display of the cucumber growth process and a lack of human-computer interaction capabilities, thus wasting data resources.

[0004] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure provide a method, apparatus, electronic device, and computer-readable medium for visualizing facility cucumber growth based on multimodal time-series data to address one or more of the technical problems mentioned in the background section above.

[0007] In a first aspect, some embodiments of this disclosure provide a method for visualizing the growth of greenhouse cucumbers based on multimodal time-series data, comprising: preprocessing a sequence of phenotypic images and corresponding environmental parameter sequences collected during the growth process of greenhouse cucumbers to generate a preprocessed sequence of phenotypic images and a preprocessed sequence of environmental parameters; extracting features from the preprocessed sequence of phenotypic images and the preprocessed sequence of environmental parameters to generate a sequence of fused feature vectors arranged in chronological order; constructing a growth time-series data model characterizing the dynamic growth pattern of the greenhouse cucumbers based on the fused feature vector sequence; smoothing and detecting change points in key growth parameters of the greenhouse cucumbers based on the growth time-series data model to generate multiple growth stage identifiers and corresponding sets of multiple growth state parameters; dynamically generating a visualization interface for the greenhouse cucumbers based on the multiple growth stage identifiers and the multiple sets of growth state parameters, the visualization interface including: growth animation, parameter curves, and stage annotation information; and updating the visualization interface according to the interaction instructions received from a target user to complete the dynamic response of the visualization interface.

[0008] Secondly, some embodiments of this disclosure provide a visualization device for the growth of greenhouse cucumbers based on multimodal time-series data, comprising: a preprocessing unit configured to preprocess a sequence of phenotypic images and corresponding environmental parameter sequences collected during the growth process of greenhouse cucumbers to generate a preprocessed phenotypic image sequence and a preprocessed environmental parameter sequence; a feature extraction unit configured to extract features from the preprocessed phenotypic image sequence and the preprocessed environmental parameter sequence to generate a fusion feature vector sequence arranged in chronological order; a construction unit configured to construct a growth time-series data model characterizing the dynamic growth law of the greenhouse cucumbers based on the fusion feature vector sequence; a processing and detection unit configured to perform smoothing processing and change point detection on key growth parameters of the greenhouse cucumbers based on the growth time-series data model to generate multiple growth stage identifiers and corresponding multiple growth state parameter sets; a generation unit configured to dynamically generate a visualization interface for the greenhouse cucumbers based on the multiple growth stage identifiers and the multiple growth state parameter sets, the visualization interface including: growth animation, parameter curves, and stage annotation information; and an update unit configured to update the visualization interface according to the interaction instruction received from a target user.

[0009] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, such that when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0010] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0011] The above-described embodiments of this disclosure have the following beneficial effects: The facility cucumber growth visualization method based on multimodal time-series data, as described in some embodiments of this disclosure, enables an interactive and dynamic display of the cucumber growth process. Specifically, the reason why traditional methods struggle to intuitively understand the relationship between the facility cucumber growth process and environmental response is the lack of a dynamic fusion and interactive visualization mechanism for continuous time-series data. Therefore, the facility cucumber growth visualization method based on multimodal time-series data, as described in some embodiments of this disclosure, firstly preprocesses the collected phenotypic image sequences and corresponding environmental parameter sequences during the facility cucumber growth process to generate preprocessed phenotypic image sequences and preprocessed environmental parameter sequences. Preprocessing the phenotypic image sequences and environmental parameter sequences eliminates acquisition noise and heterogeneity, providing a clean and aligned multi-source time-series data foundation for subsequent feature extraction. Then, feature extraction is performed on the preprocessed phenotypic image sequences and preprocessed environmental parameter sequences to generate a fused feature vector sequence arranged in chronological order. Visual and geometric phenotypic features are extracted from phenotypic image sequences, and temporal features are extracted from environmental parameters. These features are then fused through timestamp alignment and dimensional unification to form a joint vector sequence representing the growth state and environmental influences, providing rich and correlated data representation for modeling. Next, based on the fused feature vector sequence, a growth time-series data model is constructed to represent the dynamic growth patterns of greenhouse cucumbers. This model accurately represents the dynamic growth patterns of greenhouse cucumbers, uncovers the temporal correlation between phenotypic and environmental factors during growth, and achieves a quantitative description and pattern extraction of cucumber growth states, providing a model basis for subsequent growth stage division and parameter analysis. Furthermore, based on the growth time-series data model, key growth parameters of the greenhouse cucumbers are smoothed and change points are detected to generate multiple growth stage identifiers and corresponding sets of growth state parameters. Smoothing eliminates temporal fluctuations in key growth parameters, and change point detection accurately divides the growth cycle, generating growth stage identifiers and corresponding state parameter sets. This clearly defines the feature boundaries of each growth stage of the cucumber, quantifies the growth state of each stage, and provides staged and standardized growth data support for visualization generation. Furthermore, based on the aforementioned multiple growth stage identifiers and growth state parameter sets, a dynamic visualization interface for the facility-grown cucumbers is generated. This interface includes growth animations, parameter curves, and stage annotation information. Based on these multiple growth stage identifiers and parameter sets, the system drives 3D model deformation animations, draws parameter curves, annotates stage boundaries and descriptive text, and simultaneously displays environmental data, achieving a continuous, intuitive, and interactive visualization of the growth process. Finally, in response to received interaction commands from the target user, the visualization interface is updated accordingly to achieve dynamic responsiveness.By responding to user commands to drive the model to perform backtracking or simulation predictions and updating the visualization interface content in real time, it transforms from a passive display into an interactive and explorable "digital twin," supporting users to conduct hypothesis analysis and gain a deeper understanding of the impact of environmental and other factors on growth. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0013] Figure 1 This is a flowchart of some embodiments of the facility cucumber growth visualization method based on multimodal time series data according to the present disclosure; Figure 2 These are schematic diagrams illustrating the structure of some embodiments of the facility cucumber growth visualization device based on multimodal time-series data according to this disclosure; Figure 3 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure.

[0014] Figure 4 It can be the visualization interface of the facility cucumber growth visualization method based on multimodal time series data disclosed herein. Detailed Implementation

[0015] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0016] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0017] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0018] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0019] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0020] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0021] refer to Figure 1 The flowchart 100 illustrates some embodiments of a method for visualizing the growth of greenhouse cucumbers based on multimodal time-series data according to the present disclosure. This method for visualizing the growth of greenhouse cucumbers based on multimodal time-series data includes the following steps: Step 101: Preprocess the phenotypic image sequence and the corresponding environmental parameter sequence collected during the cucumber generation process to generate a preprocessed phenotypic image sequence and a preprocessed environmental parameter sequence.

[0022] In some embodiments, the executing entity (e.g., an electronic device) of the above-described method for visualizing facility cucumber growth based on multimodal time-series data can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is software, it can be installed in the hardware devices listed above. It can be implemented as multiple software programs or software modules to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0023] In other embodiments, the aforementioned executing entity can preprocess the phenotypic image sequence and corresponding environmental parameter sequence collected during the growth process of greenhouse cucumbers to generate a preprocessed phenotypic image sequence and a preprocessed environmental parameter sequence. Here, "greenhouse cucumber" refers to cucumbers cultivated in an artificially controlled environment (e.g., greenhouse, polytunnel). The aforementioned phenotypic images can be image data characterizing the external morphological features of greenhouse cucumber plants. For example, the aforementioned phenotypic images can be surface color RGB images of leaves, main vines, and fruits captured by a camera. The aforementioned phenotypic image sequence can be a collection of cucumber phenotypic images arranged in chronological order. The aforementioned environmental parameter sequence can be data describing environmental factors of cucumber growth recorded in chronological order, including: temperature, humidity, and light intensity. The aforementioned preprocessed phenotypic image sequence can be a phenotypic image sequence after format standardization, segmentation, and noise reduction processing. The aforementioned preprocessed environmental parameter sequence can be an environmental parameter sequence after complementation, alignment, and normalization.

[0024] In some optional implementations of certain embodiments, the aforementioned execution entity may preprocess the phenotypic image sequence and the corresponding environmental parameter sequence collected during the cucumber generation process to generate a preprocessed phenotypic image sequence and a preprocessed environmental parameter sequence, which may include the following steps: The first step involves standardizing the image format and correcting the white balance of the aforementioned phenotypic image sequence to generate a first intermediate image sequence. This first intermediate image sequence can be a phenotypic image sequence that has only undergone format standardization and white balance correction. In practice, firstly, the phenotypic image sequence is uniformly converted to a preset image format (e.g., PNG format). Then, automatic white balance correction is performed based on a white point detection algorithm to eliminate color casts under different lighting conditions. Finally, the first intermediate image sequence is output.

[0025] The second step involves background removal and target cucumber plant segmentation of the first intermediate image sequence to generate a second intermediate image sequence. The target cucumber plant can be a single cucumber crop selected as the object of observation and analysis. The second intermediate image sequence can be an image sequence generated by separating the main body of the target cucumber plant from the first intermediate image sequence and removing complex backgrounds. In practice, firstly, an image segmentation algorithm (e.g., a semantic segmentation model based on deep learning) is used to identify the cucumber plant pixel regions in the first intermediate image sequence. Then, the identified plant regions (foreground) are separated from the background regions (e.g., soil, supports, pots). Finally, a second intermediate image sequence containing only the main body of the target cucumber plant is generated.

[0026] The third step involves normalizing the image size and filtering noise in the second intermediate image sequence to generate a preprocessed phenotypic image sequence. In practice, firstly, all images in the second intermediate image sequence are scaled to a uniform pixel size (e.g., 256×256). Then, a filtering algorithm (e.g., Gaussian filtering or median filtering) is applied to eliminate random noise caused by acquisition noise or compression artifacts. Finally, the preprocessed phenotypic image sequence is generated.

[0027] The fourth step involves imputing missing values ​​in the aforementioned environmental parameter sequence to generate an intermediate environmental parameter sequence. These missing values ​​can be empty or invalid data points in the environmental parameter sequence due to sensor malfunctions, transmission interruptions, or other reasons. The intermediate environmental parameter sequence can be a complete intermediate result generated after the environmental parameter sequence has undergone missing value imputation. In practice, firstly, the environmental parameter sequence (e.g., temperature sequence) is checked for missing values ​​or outliers that significantly exceed reasonable ranges. Then, for missing data points, interpolation methods (e.g., time series linear interpolation or mean-based interpolation based on neighboring data) are used to impute them. Finally, a first intermediate environmental parameter sequence is generated.

[0028] The fifth step involves performing timestamp-based data alignment and normalization on the aforementioned first intermediate environmental parameter sequence to generate a preprocessed environmental parameter sequence. In practice, firstly, the environmental parameter data is resampled or interpolated based on the image acquisition timestamp to ensure that each image frame has a corresponding environmental parameter value, achieving data synchronization. Then, environmental parameters with different dimensions (e.g., temperature in °C, illumination in lux) are normalized to the [0, 1] interval using methods such as Min-Max scaling to eliminate the influence of dimensions. Finally, a preprocessed environmental parameter sequence that is time-aligned with the image data and numerically regularized is generated.

[0029] Step 102: Extract features from the preprocessed phenotypic image sequence and the preprocessed environmental parameter sequence to generate a fused feature vector sequence arranged in chronological order.

[0030] In some embodiments, the execution entity may extract features from the preprocessed phenotypic image sequence and the preprocessed environmental parameter sequence to generate a fused feature vector sequence arranged in chronological order. This fused feature vector sequence may be a sequence of feature vectors uniformly representing the growth state, generated by fusing comprehensive phenotypic features with environmental temporal features at the same time. For example, the fused feature vector sequence may be a fused feature vector formed by concatenating comprehensive phenotypic features with environmental temporal features (e.g., daily average temperature, average light intensity, humidity).

[0031] In some optional implementations of certain embodiments, the execution entity may perform feature extraction on the preprocessed phenotypic image sequence and the preprocessed environmental parameter sequence to generate a fused feature vector sequence arranged in chronological order, which may include the following steps: The first step involves extracting visual features from the preprocessed phenotypic image sequence to generate a phenotypic visual feature vector set. These visual features can be abstract features representing the overall visual attributes of the plant, extracted from the preprocessed phenotypic image sequence. For example, these visual features may include texture, color distribution, and shape contour. The phenotypic visual feature vector set can be a collection of visual feature vectors extracted from the phenotypic images at each time point, arranged chronologically. In practice, the preprocessed phenotypic image sequence is first input into a pre-trained convolutional neural network (e.g., ResNet). Then, the last classification layer of the network is removed, and the output of its penultimate layer (fully connected layer) is extracted as the abstract visual feature representation of the image. Finally, a high-dimensional numerical vector corresponding to each time point image is obtained, forming the phenotypic visual feature vector set.

[0032] The second step involves using a target algorithm to extract geometric parameters from the preprocessed phenotypic image sequence, generating a set of phenotypic geometric feature parameters. This target algorithm can be a specific image processing algorithm (e.g., YOLOv5) used to quantify and measure specific morphological indicators from images. In practice, firstly, the target algorithm (e.g., YOLOv5) is used to locate key organs (leaves, fruits, flowers) in the image. Then, image processing techniques (e.g., edge detection, contour analysis) are used to measure the geometric parameters of each organ. Finally, the set of phenotypic geometric feature parameters is compiled.

[0033] The third step involves extracting time-series features from the preprocessed environmental parameter sequence to generate an environmental time-series feature vector set. These time-series features can be time-varying characteristics extracted from the preprocessed environmental parameter sequence. For example, they could be 24-hour periodic features of temperature data or trend features of humidity data. This environmental time-series feature vector set can be a collection of time-series feature vectors extracted from the environmental parameter sequence within different time windows, arranged in chronological order. In practice, firstly, the preprocessed environmental parameter sequence (e.g., temperature, humidity) is divided into fixed-length sliding time windows (e.g., in days). Then, statistical characteristics (e.g., mean, variance, extreme values) and dynamic change characteristics (e.g., slope, volatility) are calculated for the data within each time window. Finally, these features are combined into vectors to form the environmental time-series feature vector set.

[0034] The fourth step involves unifying and concatenating the dimensions of the aforementioned phenotypic visual feature vector set and the aforementioned phenotypic geometric feature parameter set to generate a comprehensive phenotypic feature vector set. This comprehensive phenotypic feature vector set can be a more comprehensive set of phenotypic image features formed by concatenating or fusing the visual feature vectors and geometric parameter vectors at the feature level. In practice, firstly, due to the significant difference in dimensions between the visual feature vectors (e.g., 2048 dimensions) and the geometric parameter vectors (e.g., 3 dimensions), methods such as Principal Component Analysis (PCA) are used to reduce the dimensionality of the high-dimensional visual features, making them closer to the scale of the geometric parameter dimensions. Then, the dimensionality-reduced visual feature vectors and geometric parameter vectors are concatenated along the feature dimensions. Finally, the comprehensive phenotypic feature vector set is generated.

[0035] The fifth step involves fusing the aforementioned comprehensive phenotypic feature vector set with the aforementioned environmental temporal feature vector set according to timestamps, generating a fused feature vector sequence arranged in chronological order. In practice, firstly, it is ensured that the comprehensive phenotypic feature vector set and the environmental temporal feature vector set are perfectly aligned in terms of timestamps. Then, the comprehensive phenotypic feature vectors at the same timestamp are concatenated with the environmental temporal feature vectors to form a more comprehensive fused feature vector. Finally, the fused feature vectors from all time points are arranged in chronological order to generate the fused feature vector sequence.

[0036] Step 103: Based on the fused feature vector sequence, construct a growth time series data model that characterizes the dynamic growth pattern of cucumbers in the facility.

[0037] In some embodiments, the aforementioned executing entity can construct a growth time-series data model characterizing the growth dynamics of greenhouse cucumbers based on the aforementioned fused feature vector sequence. The aforementioned growth dynamics can be the growth variation patterns of greenhouse cucumbers over time and with environmental changes. For example, the aforementioned growth dynamics can be the growth rate of the main vine length varying with temperature and light, or the morphological evolution during the fruiting period. The aforementioned growth time-series data model can be a time-series model capable of characterizing and predicting the cucumber growth process. For example, the aforementioned growth time-series data model can be a predictive model based on the Transformer architecture, capable of taking environmental parameters as input and outputting the main vine length for the next 3 days. The aforementioned growth time-series data model may include a multi-source data input layer (the input is a fused feature vector sequence, and the output is the same sequence, which serves as the encoder input), an encoder layer (Transformer encoder, the input is a fused feature vector sequence, and the output is a hidden state sequence), a univariate prediction residual layer (parallel auxiliary module) (the input is a hidden state sequence, and the output is a prediction residual sequence of each key growth parameter), a feature enhancement and splicing layer (the input is a prediction residual sequence of each key growth parameter, and the output is an enhanced time-series feature sequence), a regression head (the input is an enhanced time-series feature sequence, and the output is growth dynamic law information), and a final model output (an executable mathematical model that can receive environmental parameters and the current state, output future growth predictions (e.g., the trajectory in the next 72 hours), and simulate the environmental response relationship).

[0038] In some optional implementations of certain embodiments, the execution entity may construct a growth time-series data model characterizing the dynamic growth pattern of cucumbers in the facility based on the aforementioned fused feature vector sequence, which may include the following steps: The first step involves inputting the aforementioned fused feature vector sequence into the target encoder to obtain a hidden state sequence representing growth context information. The target encoder can be a core neural network module used to extract temporal features. For example, it could be a Transformer encoder or the encoder portion of an LSTM. The growth context information can be implicit information representing the current growth state and its association with past states. For example, it could be information such as "the plant is in a rapid vegetative growth period and has sufficient accumulated temperature in the early stages." The hidden state sequence can be a sequence of feature vectors containing context information output by the target encoder at each time step after processing the input fused feature vector sequence. In practice, firstly, the fused feature vector sequence arranged chronologically (e.g., one 17-dimensional vector per day for 60 days) is input into the target encoder (e.g., an LSTM or Transformer encoder). Then, the target encoder comprehensively considers the features at each time point and their relationship with previous and subsequent time points, performing encoding calculations. Finally, the hidden states containing growth context information output by the target encoder at each time point are obtained, forming the hidden state sequence.

[0039] The second step involves inputting the aforementioned fused feature vector sequences into corresponding pre-trained univariate time-series prediction models to generate prediction residual sequences for each key growth parameter. These pre-trained univariate time-series prediction models can be single-parameter prediction models trained on historical data. For example, an ARIMA model can be used to predict the main vine length, or an LSTM model can be used to predict the number of leaves. The key growth parameters can be core quantitative indicators characterizing growth status, such as main vine length, number of female flowers, fruit diameter, and leaf area index. The prediction residual sequences can be sequences of differences between actual observed values ​​and univariate model predictions. For example, the prediction residual sequences can be daily residual sequences between actual measured main vine length and ARIMA predictions. In practice, firstly, the historical true value sequence of each key growth parameter (e.g., main vine length) is separated from the fused feature sequence. Then, the historical sequence of each parameter is input into the corresponding pre-trained univariate prediction model (e.g., TCN) to predict its value at the next time step. Finally, the difference between the predicted value and the actual value at each time point (i.e., the residual) is calculated, and a residual sequence is generated for each key parameter to produce the prediction residual sequence.

[0040] The third step involves concatenating the hidden state sequence with the predicted residual sequence to construct an enhanced temporal feature sequence. This enhanced temporal feature sequence can be a more information-rich feature sequence formed by concatenating the hidden state sequence (growth context information) with the residual sequence. In practice, firstly, it is ensured that the hidden state sequence obtained in the first step is perfectly aligned with each residual sequence obtained in the second step at all time points. Then, the hidden state vector at each time point is concatenated with the residual values ​​of all key growth parameters at that time point. Finally, a new enhanced temporal feature sequence is generated that contains both global growth context and specific parameter prediction bias information.

[0041] The fourth step involves inputting the enhanced temporal feature sequence into a regression head based on the growth rate equation to output the growth dynamics information of the greenhouse cucumbers. The growth rate equation can be a mathematical equation describing the relationship between cucumber growth rate and influencing factors (e.g., dL / dt = f(temperature, humidity, H_t), where L is the key growth parameter, dL / dt represents the growth rate, f() represents the growth response function (which can be a linear regression equation, a neural network, or any other parameterizable mathematical expression), and H_t represents the core phenotypic parameters of the greenhouse cucumber at time t (e.g., main vine length and stem diameter at time t) ("t" represents the current time), reflecting the influence of the cucumber's current growth baseline on the subsequent growth rate. These core phenotypic parameters can be key morphological indicators that best characterize the growth state, such as main vine length, number of leaves, number of female flowers per node, and fruit diameter). The growth dynamics information can be quantitative data on cucumber growth patterns, such as the main vine length growth rate and environmental parameter influence coefficients at each growth stage. In practice, firstly, a regression head is designed, the core of which is a neural network layer built based on a growth rate equation that correlates the growth rate with the current state (e.g., main vine length) and the environment (e.g., temperature, humidity). Then, an enhanced temporal feature sequence is input into this regression head for training or inference. Finally, the regression head outputs specific parameters that quantify the growth dynamics (growth rate under specific environmental conditions).

[0042] The fifth step involves generating a growth time-series data model based on the aforementioned growth dynamics information. This model can predict future growth states and simulate environmental responses. In practice, firstly, the preceding steps (encoder, univariate prediction residual layer, regression head) are integrated into a complete, end-to-end computational graph or model structure. Then, this integrated model is trained using historical data to optimize its internal parameters (e.g., the weight matrix in the encoder, the coefficients of the growth rate equation in the regression head). The training objective is to minimize the error (e.g., mean squared error) between the model's predicted future growth states and the actual observed values, enabling it to accurately fit the historical growth process. Finally, a trained growth time-series data model is obtained, which possesses the ability to predict the future and simulate environmental responses based on the input sequence.

[0043] Step 104: Based on the growth time series data model, smooth the key growth parameters of greenhouse cucumbers and detect the change points to generate multiple growth stage identifiers and corresponding multiple growth state parameter sets.

[0044] In some embodiments, the aforementioned execution entity can perform smoothing and change point detection on the key growth parameters of the greenhouse cucumber based on the aforementioned growth time series data model, to generate multiple growth stage identifiers and corresponding multiple growth state parameter sets. The aforementioned change points can be time nodes in the time series of greenhouse cucumber growth parameters where the trend changes significantly, for example, the date corresponding to a sudden acceleration (5cm / day) in the daily growth rate of the main vine from slow (<2cm / day). The aforementioned multiple growth stage identifiers can be labels for different physiological stages divided into the entire growth period of cucumber, such as the seedling stage, vine-growing stage, and flowering and fruiting stage. The aforementioned multiple growth state parameter sets can be sets of typical numerical ranges of key growth parameters within each growth stage, representing the characteristic state of that stage. For example, the growth state parameter set for the flowering and fruiting stage might be: {Main vine length: 150 to 200cm, number of leaves: 15 to 25, number of flowers per day: 3 to 8}.

[0045] In some optional implementations of certain embodiments, the execution entity may, based on the aforementioned growth time-series data model, perform smoothing and change point detection on the key growth parameters of the facility cucumber to generate multiple growth stage identifiers and corresponding multiple growth state parameter sets, which may include the following steps: The first step is to extract the raw time-series sequences of multiple key growth parameters from the aforementioned growth time-series data model. These raw time-series sequences can be the raw, unsmoothed data of the key growth parameters arranged in chronological order, directly output from the growth model. In practice, firstly, based on the growth time-series data model, historical data for key growth parameters (e.g., main vine length, leaf area) are located from its output or intermediate results. Then, all data points for each parameter throughout the entire growth cycle are extracted in chronological order (e.g., from the planting date). Finally, an independent raw time-series sequence is generated for each key growth parameter.

[0046] The second step involves applying a moving average smoothing process to the original time series sequences of each of the aforementioned key growth parameters to generate multiple smoothed time series sequences. These smoothed time series sequences can be data sequences that better reflect the overall trend, obtained by filtering and smoothing the original time series sequences. In practice, firstly, a smoothing algorithm (e.g., a moving average with a window size of 5) is selected to process each original time series sequence. Then, the smoothing window is slid across the sequence, and the mean (or median) of the data within the window is calculated to replace the original value at the window's center point. Finally, a smoothed time series sequence displaying the long-term trend is generated.

[0047] The third step involves detecting change points in each of the multiple smoothed time series sequences based on cumulative differences to identify potential switching points between multiple growth stages. The cumulative difference can be a time series analysis technique used to detect abrupt changes in the data mean. It calculates the cumulative deviation of the data sequence from its mean; a sharp change in deviation indicates a potential change point. These potential switching points between growth stages can be candidate points detected from the smoothed sequences by an algorithm (e.g., the PELT algorithm) that suggest a possible shift in growth stages. In practice, first, the cumulative sum sequence of the smoothed time series is calculated. Then, the difference between the cumulative sum and its linear trend is calculated to form a cumulative difference sequence. Finally, inflection points or extreme points in this difference sequence are detected and identified as potential switching points between growth stages.

[0048] The fourth step involves dividing the entire growth cycle of the greenhouse cucumber into multiple stages based on the potential switching time points and preset stage thresholds, thereby generating multiple growth stage identifiers. The preset stage thresholds can be predefined quantitative standards for determining stage switching, such as the threshold for entering the flowering and fruiting stage: ≥1 female flower. The entire growth cycle can be the entire growth and development time range of the greenhouse cucumber from sowing to final harvest. In practice, firstly, the detected potential switching time points are sorted by time. Then, the time periods between adjacent potential points are verified according to preset threshold conditions (e.g., "≥1 female flower for 3 consecutive days" marks the entry into the flowering stage). Finally, consecutive time periods meeting the threshold conditions are merged, and each time period is assigned a unique stage identifier.

[0049] The fifth step involves determining the range of key growth parameters for each of the multiple growth stage identifiers mentioned above, thereby generating multiple sets of growth state parameters. These ranges can be numerical intervals describing key parameters within a specific growth stage; for example, the main vine length range for seedlings is 10 to 50 cm. In practice, firstly, for each identified growth stage, the raw values ​​of all key growth parameters within that time period are extracted. Then, the statistics for each parameter are calculated, such as minimum, maximum, average, and standard deviation. Finally, the statistics for each parameter are summarized to form the growth state parameter set for that stage, resulting in multiple sets of growth state parameters. For example, for the "flowering and fruiting period" (D46-D90), the calculated main vine length range is [150, 320] cm, the fruit quantity range is [5, 15], and the average daily flowering count is 0.5, forming one set of growth state parameters.

[0050] Step 105: Dynamically generate a visualization interface for the cucumbers in the above-mentioned facility based on multiple growth stage identifiers and multiple growth state parameter sets.

[0051] In some embodiments, the executing entity can dynamically generate a visualization interface for the facility cucumber based on the multiple growth stage identifiers and the multiple growth state parameter sets. The visualization interface includes: growth animation, parameter curves, and stage annotation information. The visualization interface can be a graphical user interface for intuitively displaying the cucumber's growth process, parameter changes, and stage information. The growth animation can be a visual animation that dynamically displays the cucumber's growth process by continuously playing a sequence of plant morphology. The parameter curves can be trend graphs representing the changes of key growth parameters over time in a two-dimensional coordinate system. The stage annotation information can be a comprehensive identifier combining stage boundary lines and stage names on the parameter curves.

[0052] In addressing the technical challenges mentioned above, and considering the application scenario—such as agricultural science exhibition halls and school laboratories—where the complete growth process of greenhouse cucumbers needs to be visually demonstrated to the public or students, the following technical issues often arise: traditional video demonstrations use pre-recorded, fixed content that cannot be dynamically adjusted based on actual growth data. This wastes time and material costs associated with customized production for science popularization and teaching demonstrations, resulting in a disconnect between the displayed content and the actual crop growth status. Given the following requirements for this application scenario—high continuity of visualization, clear stage-by-stage processing, synchronous linkage of multiple elements, and accurate matching of actual growth status—we have decided to adopt the following solution: In some optional implementations of certain embodiments, the execution entity can dynamically generate a visual interface for the facility cucumber based on the plurality of growth stage identifiers and the plurality of growth state parameter sets, which may include the following steps: The first step involves using pre-defined 3D cucumber models corresponding to the aforementioned growth stages to determine the basic visualization object. These pre-defined 3D cucumber models can be typical 3D geometric models of cucumber plants created beforehand for different growth stages (e.g., seedling stage, vine-spreading stage, flowering and fruiting stage). For example, a seedling model with only two true leaves and a mature plant model with multiple fruits. The basic visualization object can be an initial 3D model retrieved from a model library based on the current growth stage, without parameter adjustments. In practice, first, the growth stage identifier corresponding to the current desired display time (e.g., "flowering and fruiting stage") is used. Then, a pre-defined 3D cucumber model corresponding to the growth stage is retrieved. Finally, this model is set as the basic visualization object for the current scene, ready for subsequent personalized adjustments.

[0053] The second step involves deforming and texturing the basic visualization object based on the aforementioned sets of growth state parameters to generate a corresponding 3D model of the cucumber plant. This corresponding 3D model can be a modified and rendered version of the basic model that matches the current specific growth state parameters. In practice, first, the set of growth state parameters corresponding to the current time point is obtained (e.g., main vine length, number of main vine nodes, leaf area). Then, based on these parameter values, the vertex positions of the basic visualization object are deformed (e.g., height is scaled), and the surface texture is rendered (e.g., leaf color depth is adjusted based on leaf area). Finally, a personalized 3D model of the cucumber plant that highly matches the actual plant morphology is generated.

[0054] The third step involves using an inter-frame interpolation algorithm, based on the historical sequence of the aforementioned growth state parameter set, to perform morphological transition processing on the 3D models of cucumber plants at adjacent time points, thereby constructing a highly continuous plant morphology sequence. The aforementioned historical sequence can be a set of past values ​​of key growth parameters arranged chronologically. High continuity means smooth and natural transitions between frames in an animation, without jumps or flickering. The aforementioned plant morphology sequence can be a sequence of models representing the 3D morphology of cucumber plants at different times, arranged chronologically. For example, the plant morphology sequence could be a time series composed of 3D cucumber models from day 1, day 2, up to day 60. In practice, first, the historical sequence of the growth state parameter set is obtained. Then, for each adjacent time point (e.g., day n and day n+1), the vertex position of the intermediate transition morphology is calculated using an interpolation algorithm (e.g., linear interpolation or spline interpolation). Finally, a continuously changing plant morphology sequence containing all historical time points and intermediate transition frames is generated.

[0055] The fourth step involves driving the 3D model of the cucumber plant frame-by-frame for deformation and lighting rendering based on the aforementioned plant morphology sequence to generate a growth animation. In practice, firstly, the constructed plant morphology sequence is loaded into the graphics rendering engine in chronological order. Then, the engine renders the model in each morphology sequence onto the screen sequentially at a set frame rate (e.g., 30 frames per second), while simultaneously calculating the lighting effects corresponding to each frame. Finally, through the persistence of vision effect, a continuous and smooth growth animation is formed.

[0056] The fifth step is to extract historical sequences of key growth parameters from the growth state parameter set to generate parameter curves. In practice, first, historical data sequences of key parameters to be displayed (e.g., "main vine length") are extracted from the growth state parameter set. Then, a two-dimensional coordinate system is established on the graphics canvas (horizontal axis for time, vertical axis for parameter values), and each data point is plotted at its corresponding position. Finally, these data points are connected sequentially with straight lines or smooth curves to form parameter variation curves.

[0057] Step 6: Based on the multiple growth stage identifiers mentioned above, mark the corresponding stage boundary lines and stage names on the parametric curve to generate stage labeling information. The stage boundary lines can be vertical reference lines on the parametric curve marking the start or end times of a growth stage. The stage names can be labels for the growth stage, such as "seedling stage" or "flowering and fruiting stage." In practice, first, based on the multiple growth stage identifiers, find the start and end times of each stage on the time axis of the parametric curve. Then, draw vertical dashed lines at these time points as stage boundary lines. Finally, add corresponding stage name text labels near the boundary lines or inside the stage area.

[0058] Step 7: Synchronously overlay and display the growth animation, parameter curves, and stage annotation information to generate a visual interface. In practice, firstly, allocate display areas for the growth animation, parameter curves, and stage annotation information in the interface layout (e.g., animation on the left, curves on the right). Then, integrate the rendered growth animation video stream, the drawn parameter curve graph, and the annotation information into the same interface window. Finally, ensure that the timelines of the three are synchronized and provide unified interactive controls (e.g., a timeline slider). Figure 4 This is a visualization interface for greenhouse cucumbers. The left side displays a live animation of the cucumber plant's growth during the "early flowering and fruiting" stage. The upper right side shows the model-adapted environmental range for this stage (e.g., air temperature, humidity, CO2 concentration) and stage characteristic parameters (e.g., stage duration, main vine length, number of leaves). The lower right side displays the fruit growth trend as a parametric curve (horizontal axis for growth period, vertical axis for fruit length), with the boundaries and names of four stages—I. Slow Growth Stage, II. Accelerated Growth Stage, III. Rapid Enlargement Stage, and IV. Stabilization Stage—labeled on the curve. This achieves an organic integration of growth animation, parametric curves, and stage labeling information, intuitively presenting the linkage between the cucumber growth process, parameter changes, and stage divisions.

[0059] The above-described steps, as an inventive point of this disclosure, solve the technical problem mentioned in the background: "Traditional video displays are pre-recorded, fixed content that cannot be dynamically adjusted based on actual growth data, wasting the customized production time and material costs of popular science displays and teaching demonstrations, resulting in a disconnect between the displayed content and the actual crop growth state." The reasons for this technical problem are as follows: traditional video displays are disconnected from real-time growth data, lack a data-driven dynamic generation mechanism, and cannot achieve synchronous linkage and personalized adjustment of multi-dimensional visualization elements. This invention establishes a parametric mapping relationship between growth data and a three-dimensional model, and uses inter-frame interpolation technology to ensure animation continuity, achieving synchronous rendering of multiple views. This enables dynamic, continuous, and interactive visualization based on real growth data, accurately reflecting the stage characteristics of crop growth, saving the customized production time and material costs of visualization content in agricultural popular science and teaching, as well as the labor costs of manually compiling growth data and creating display content.

[0060] Step 106: In response to receiving the interaction command from the target user, update the visualization interface according to the interaction command to complete the dynamic response of the visualization interface.

[0061] In some embodiments, the execution entity may, in response to receiving an interaction instruction from a target user, update the visualization interface according to the interaction instruction to achieve dynamic response of the visualization interface. The target user may be an operator of the visualization system, such as a researcher, teacher, or student. The interaction instruction may be an operation command issued by the user to control or query the visualization content. For example, the user may drag the timeline slider to day 25. The dynamic response may be a process of updating the visualization interface content in real time according to the target user's interaction instruction.

[0062] In addressing the technical challenges mentioned above, and considering the application scenario—interactive science exhibitions or digital agricultural classrooms—where visitors or students want to explore cucumber growth patterns through hands-on activities, the following technical problems often arise: traditional display systems can only passively play fixed content, preventing visitors from actively exploring and interacting based on their interests. This results in low information acquisition efficiency, wasted data resources, and wasted user time. To meet the specific requirements of this application scenario—support for user interaction, real-time interface updates, coordinated adjustment of multiple visualization elements, and precise retrieval of target growth stage information—we have decided to adopt the following solution: In some optional implementations of certain embodiments, the execution entity may update the visualization interface in response to receiving an interaction instruction from the target user, thereby completing the dynamic response of the visualization interface. This may include the following steps: The first step is to parse the types and parameters of the aforementioned interactive instructions to generate parsed instruction information. The types can be categories of interactive instructions, such as "time jump, environmental parameter adjustment, growth stage selection, animation control (play / pause)". The parameters can be specific values ​​or identifiers accompanying the instruction type. For example, the parameters for a time jump instruction could be "2025-10-01", and the parameters for an environmental adjustment instruction could be "temperature: 25℃". The parsed instruction information can be the result of a structured interpretation of the interactive instructions. For example, the parsed instruction information could be parsing touch coordinates "(320, 150)" into "select time point: day 30". In practice, first, the target user's input event (e.g., touch, click, drag) is determined. Then, a predefined instruction type is matched based on the input position and pattern. Finally, key parameters from the instructions are extracted to form structured information as the parsed instruction information.

[0063] The second step involves generating a corresponding time-series data subset based on the parsed instruction information and the aforementioned growth time-series data model. This subset can be a portion of the data related to the instruction extracted from the complete growth time-series data model. For example, when a user selects to view days 30-40, the subset could be the growth feature vector sequence extracted from the model for those 10 days. In practice, first, the parsed instruction information (such as the target time point) is input into the growth time-series data model. Then, the model performs calculations according to the instruction requirements, such as backtracking to a historical time point or predicting backwards from the current point. Finally, the model outputs the growth data sequence for the time period or time point corresponding to the instruction, which is the time-series data subset.

[0064] The third step involves generating an updated set of growth state parameters based on the aforementioned subset of time-series data. This updated set can be derived from newly extracted time-series data and recalculated growth state parameters. In practice, firstly, key growth parameter information for the target time point (or time period) is extracted from the time-series data subset generated by the model. Then, these parameters are integrated or calculated to obtain a complete snapshot of the plant's state at that moment. Finally, an updated set of growth state parameters containing the current values ​​of all key parameters, such as main vine length and leaf area, is generated.

[0065] The fourth step involves deforming and rendering the 3D model of the cucumber plant based on the updated growth state parameter set, dynamically updating the growth animation. In practice, firstly, the 3D model of the cucumber plant in the current scene is deformed (e.g., scaled) according to the updated growth state parameter set (e.g., main vine length, cucumber length). Then, the model texture details are adjusted according to the updated growth state parameters (e.g., leaf area), and real-time lighting rendering is performed. Finally, the updated model is presented to the user, enabling transitions or changes in the growth animation.

[0066] The fifth step involves redrawing the corresponding segments of the parameter curves in real time based on the updated growth state parameter set, thus generating redrawn parameter curves. These redrawn parameter curves can be a partial or complete update of the curves in the chart based on the updated growth state parameter set. In practice, first, based on the instructions and the new data range, the curve segments that need to be redrawn (e.g., from the start to the target time point) are determined. Then, the original graphs of the segments on the canvas are cleared, and the curves are redrawn in those segments using the new data points. Finally, the redrawn parameter curves are generated.

[0067] Step 6: Based on the stage selection information from the interactive instructions, adjust the timeline labels and stage identifiers on the visualization interface. The stage selection information can be a growth stage directly specified by the user or implicitly included in the instructions. For example, the user might select "flowering and fruiting period" from a dropdown menu. The timeline labels can be text or symbols marking key time points (such as the start of a stage). For example, the timeline labels might have "initial flowering" and "peak fruiting" tags on days 20 and 40. The stage identifiers can be graphic or text elements on the interface used to indicate the current growth stage. For example, a label displaying "Current Stage: Flowering and Fruiting Period" in a corner of the interface. In practice, first, determine the growth stage to be highlighted based on the stage information implicitly or directly specified in the interactive instructions. Then, move the highlighted marker or slider on the timeline and update the corresponding stage label text. Finally, update the identifier text for the current growth stage in a prominent location on the interface (e.g., a corner).

[0068] Step 7: Based on the updated growth status parameter set described above, synchronously update the environmental parameter display content on the visualization interface to achieve dynamic response of the visualization interface. The environmental parameter display content can be numerical values ​​or charts of environmental data such as temperature and humidity displayed on the interface. In practice, firstly, obtain the environmental parameter values ​​corresponding to the target time point from the updated growth status parameter set. Then, update the various components in the interface used to display the environmental parameters, such as the digital dashboard and line graph. Finally, ensure that the display of environmental parameters is synchronized with the growth animation and parameter curves in time.

[0069] The above-described steps, as an inventive point of this disclosure, solve the technical problem mentioned in the background: "Traditional display systems can only passively play fixed content, preventing visitors from actively exploring and interacting deeply based on their interests. This results in low information acquisition efficiency, wasted data resources, and wasted user time." The reasons for this technical problem are as follows: the visualization content generation and user interaction processing flow of traditional systems are disconnected, and the system architecture cannot support dynamically querying data models and regenerating related multi-dimensional view content based on real-time user input commands. This invention constructs a closed-loop response link from user command parsing and model data retrieval to real-time redrawing of multi-view content. This achieves dynamic interactive exploration capabilities based on user exploration intentions and deep linkage between growth data and the visualization interface. It reduces the time cost of repeated user attempts and waiting for system responses and avoids the storage and computing resources consumed by pre-generating massive amounts of static content to meet diverse interaction needs.

[0070] In addressing the technical challenges mentioned above, and considering the application scenarios—facility agriculture training bases, agricultural technology extension centers, and digital science exhibition halls—where virtual simulation is used to teach cucumber growth and environmental control, the following technical issues often arise: existing technologies cannot achieve precise virtual simulation and trend prediction of cucumber growth, nor can they predict growth effects through environmental control commands, resulting in wasted experimental materials and energy consumption, and increased time spent on teaching, training, and science demonstrations. To meet the following requirements for this application scenario: support for virtual growth simulation and trend prediction, the ability to bind parameters to drive the virtual model, the ability to respond to environmental control commands and provide feedback on effectiveness, and a visual interface adaptable to different displays, we have decided to adopt the following solution: In some optional implementations of certain embodiments, the aforementioned execution entity may further perform the following steps: The first step involves constructing a virtual mapping model based on the aforementioned growth time-series data model and multiple growth state parameter sets. This virtual mapping model is used for growth simulation and growth trend prediction. The virtual mapping model can be a computable digital virtual model corresponding to the growth process of a real cucumber plant. For example, it could be a three-dimensional digital cucumber including the skeletal structure of the main vine, leaves, flowers, and fruit, as well as parameterized growth rules. The growth simulation involves using the model to reproduce or extrapolate the plant's growth process in a virtual environment. For example, inputting a set of environmental parameters can drive the virtual model to "grow" 5 centimeters in 24 hours. The growth trend prediction can be based on the current state and environmental parameters to estimate the plant's growth status over a future period. For example, it could predict that the cucumber fruit will gain 15 grams in the next 72 hours. In practice, firstly, the dynamic patterns provided by the growth time-series data model and the current snapshot provided by the growth state parameter sets are integrated. Then, this information is used to initialize a three-dimensional virtual plant with growth logic and current morphology. Finally, a virtual mapping model is constructed that reflects the current real state and can simulate and predict based on new inputs.

[0071] The second step involves binding the core phenotypic parameters from the aforementioned multiple growth state parameter sets with the geometric attributes of the 3D model in the aforementioned virtual mapping model to generate a parameter mapping relationship for data-driven purposes. The core phenotypic parameters can be key morphological indicators that best characterize the growth state. Examples include main vine length, number of leaves, number of female flowers per node, female flower node rate, and fruit diameter. The geometric attributes can be quantifiable features in the 3D model that describe shape, size, and position. Examples include the length of the main vine skeleton, vertex coordinates of the leaf mesh, and the size of the fruit model's bounding box. Data-driven approaches can involve the morphology, animation, or behavior of the virtual mapping model being controlled in real-time by the input data. The parameter mapping relationship can be a correspondence rule that associates phenotypic parameters with model geometric attributes. For example, the parameter mapping relationship can be a linear mapping formula establishing "main vine length (parameter)" and "main vine skeleton scaling factor (attribute)." In practice, firstly, core parameters such as "main vine length" and "fruit diameter" are selected from multiple growth state parameter sets. Then, the corresponding geometric attributes controlling the scaling of the main vine and the size of the fruit (e.g., skeleton scaling factor, mesh vertex coordinates) are located in the virtual mapping model. Finally, establish a mathematical relationship (e.g., a linear mapping function) for each pair of parameters and attributes. For example, establish the rule: "Main Vine Skeleton" in the virtual model = actual "Main Vine Length (cm)" / 100.

[0072] The third step involves determining the growth information of each organ of the cucumber plant in the facility, based on a pre-defined morphogenesis model, to generate organ-level growth driving instructions. This pre-defined morphogenesis model refers to a mathematical model describing the rules of plant organ formation and development. For example, the morphogenesis model might specify that "the rate of new leaf growth is positively correlated with effective accumulated temperature, and its final size is affected by light intensity." This morphogenesis model can include: an input layer (inputs are a sequence of environmental parameters (e.g., temperature, light, humidity) and the plant's current internal state (e.g., physiological age, carbon and nitrogen levels), outputs standardized environmental driving factors and plant state signals), a computational layer (inputs are standardized environmental driving factors and plant state signals, outputs quantified growth instructions for each organ), and an output layer (inputs are quantified growth instructions for each organ, outputs specific, executable organ-level growth driving instructions). The computational layer can contain pre-defined plant physiological rules, responsible for converting environmental input signals into specific organ growth instructions. For example, a pre-defined plant physiological rule might be: daily leaf growth = 0.01. Light intensity (Optimal temperature - current temperature). The above growth information can be specific requirements for organ development output by the morphogenetic model. The above organ-level growth driving instructions can be a specific set of instructions controlling how a specific organ (e.g., leaf, fruit) grows in the virtual model. For example, the above organ-level growth driving instructions could be to generate a new leaf for the 5th segment, with an initial size X and a growth rate Y. In practice, firstly, the current environmental data is input into the preset morphogenetic model. Then, the model calculates the growth rate and target morphology of each organ (e.g., apical bud, lateral bud) according to the built-in plant physiological rules. Finally, it outputs specific organ-level growth driving instructions. For example, the morphological model outputs: "Apical growth point: elongate 0.5cm in the current direction; 5th segment leaf primordium: unfold into a leaf with an area of ​​10cm²."

[0073] The fourth step involves converting the organ-level growth drive commands into target displacement and rotation parameters for the corresponding skeletal nodes in the virtual mapping model to generate a skeletal animation drive sequence. These skeletal nodes can be key points in the virtual mapping model that constitute the skeletal system and control model deformation (e.g., joints controlling bending on the main skeleton of a cucumber model). The target displacement can be the coordinates of the target position the skeletal node needs to move to. For example, the target displacement could be the node at the top of the main vine needing to move 2 cm along the positive Z-axis after one day of growth. The rotation parameters can be the angle the skeletal node needs to rotate around an axis. In practice, first, organ-level drive commands, such as "leaf unfolds 30 degrees," are received. Then, the skeletal node controlling the leaf is found in the skeletal hierarchy of the virtual model. Next, the new displacement and rotation target values ​​that the node needs to achieve are calculated based on the command. Finally, these target values ​​are arranged in chronological order to form a drive sequence. For example, the "leaf unfolds" command is converted into an animation sequence of "leaf skeletal node.rotation.x linearly changes from 0 degrees to 30 degrees, lasting 2 seconds."

[0074] The fifth step involves updating the virtual mapping model in real time based on the aforementioned skeletal animation-driven sequence, resulting in an updated virtual mapping model. In practice, the generated skeletal animation-driven sequence is first fed into a graphics rendering engine (e.g., Unity's Animator). Then, the graphics rendering engine updates the transformation matrices (position, rotation, scaling) of all relevant bone nodes frame by frame according to the sequence. Finally, the graphics rendering engine recalculates and renders the skinned mesh based on the updated bone pose, obtaining an updated virtual mapping model with continuously changing shape.

[0075] The sixth step involves performing multi-resolution hierarchical processing on the updated virtual mapping model to generate a multi-level visualization interface. This multi-resolution hierarchical processing can divide the model into different levels based on detail (e.g., whole plant, organ, tissue). For example, the first level could display the overall plant outline, the second level could show leaf veins, and the third level could display cell structures. This hierarchical visualization interface can be an interactive interface that allows the target user to view models at different levels of detail. In practice, firstly, the updated virtual mapping model is hierarchically processed to generate model versions with different levels of detail (LOD). Then, control controls (e.g., sliders) are integrated into the visualization interface. Finally, a multi-level visualization interface is implemented that allows users to seamlessly switch between viewing the entire plant and organ details.

[0076] Step 7: Input the acquired real-time environmental control commands into the updated virtual mapping model to perform growth simulation and generate predicted growth information. The real-time environmental control commands can be commands for simulating environmental changes, such as adjusting the temperature. For example, the real-time environmental control command could be: the user adjusts the temperature setting from 25℃ to 28℃ in the interface. The predicted growth information can be future growth data extrapolated by the updated virtual mapping model based on the new environmental commands. For example, the predicted growth information could be: at 28℃, the main vine will grow 3 cm longer in 7 days than at 25℃. In practice, first, the real-time environmental control commands input by the user (e.g., temperature adjustment, increased light) are acquired. Then, the real-time environmental control commands are input into the updated virtual mapping model to start the growth simulation function. Finally, through model simulation, predicted growth information for the cucumber's future is generated.

[0077] Step 8: By comparing and analyzing the predicted growth information with the pre-acquired historical growth data, the effectiveness information of the environmental control instructions is generated. The historical growth data can be actual growth process data recorded in the past. For example, the historical growth data could be the main vine growth curves recorded at 25℃ and 28℃ in a past growing season. The effectiveness information can be evaluation data on the impact of the environmental control instructions on cucumber growth. For example, the effectiveness information could be the evaluation information that "after temperature adjustment, the growth rate of cucumber main vine length increased by 15%". In practice, first, the predicted growth information generated by the model is extracted, then the pre-acquired historical growth data for the same period is retrieved, the two are compared and analyzed, the growth difference is calculated, and finally, the control effect is evaluated based on the difference to generate the effectiveness information of the environmental control instructions. For example, comparing the predicted main vine length with the historical main vine length, the effectiveness information that "the growth rate of main vine length increased by 12% after control" is obtained.

[0078] The ninth step is to provide feedback on the aforementioned validity information through the visualization interface. In practice, the validity information can be converted into intuitive text or chart formats. Then, it can be displayed in designated areas of the multi-level visualization interface to complete the feedback of the validity information.

[0079] The above-described operational steps, as an inventive point of this disclosure, solve the technical problem mentioned in the background art: "Existing technologies cannot achieve accurate virtual simulation and trend prediction of cucumber growth, cannot extrapolate growth effects through environmental control commands, waste experimental materials and energy consumption, and increase the time for teaching, training, and popular science demonstrations." The reasons for these technical problems are as follows: Existing popular science and management systems are mainly based on recording and replaying processes that have already occurred; their underlying models lack the ability to perform forward simulation and extrapolation based on new environmental inputs, and the relationship between model data and visualization is only a one-way static mapping. This invention constructs a digital twin model (i.e., a virtual mapping model) that integrates growth dynamics and supports parameter-driven and simulation extrapolation, and establishes a real-time bidirectional data flow between it and the visualization interface. This enables digital pre-simulation of environmental control commands, intelligent prediction of growth trends, and immediate quantitative evaluation of control effects. It saves the material costs of seeds, water, fertilizer, and energy consumed in comparative experiments in real facilities and significantly shortens the natural growth cycle of crops required for teaching demonstrations and scheme verification.

[0080] The above-described embodiments of this disclosure have the following beneficial effects: The facility cucumber growth visualization method based on multimodal time-series data, as described in some embodiments of this disclosure, enables an interactive and dynamic display of the cucumber growth process. Specifically, the reason why traditional methods struggle to intuitively understand the relationship between the facility cucumber growth process and environmental response is the lack of a dynamic fusion and interactive visualization mechanism for continuous time-series data. Therefore, the facility cucumber growth visualization method based on multimodal time-series data, as described in some embodiments of this disclosure, firstly preprocesses the collected phenotypic image sequences and corresponding environmental parameter sequences during the facility cucumber growth process to generate preprocessed phenotypic image sequences and preprocessed environmental parameter sequences. Preprocessing the phenotypic image sequences and environmental parameter sequences eliminates acquisition noise and heterogeneity, providing a clean and aligned multi-source time-series data foundation for subsequent feature extraction. Then, feature extraction is performed on the preprocessed phenotypic image sequences and preprocessed environmental parameter sequences to generate a fused feature vector sequence arranged in chronological order. Visual and geometric phenotypic features are extracted from phenotypic image sequences, and temporal features are extracted from environmental parameters. These features are then fused through timestamp alignment and dimensional unification to form a joint vector sequence representing the growth state and environmental influences, providing rich and correlated data representation for modeling. Next, based on the fused feature vector sequence, a growth time-series data model is constructed to represent the dynamic growth patterns of greenhouse cucumbers. This model accurately represents the dynamic growth patterns of greenhouse cucumbers, uncovers the temporal correlation between phenotypic and environmental factors during growth, and achieves a quantitative description and pattern extraction of cucumber growth states, providing a model basis for subsequent growth stage division and parameter analysis. Furthermore, based on the growth time-series data model, key growth parameters of the greenhouse cucumbers are smoothed and change points are detected to generate multiple growth stage identifiers and corresponding sets of growth state parameters. Smoothing eliminates temporal fluctuations in key growth parameters, and change point detection accurately divides the growth cycle, generating growth stage identifiers and corresponding state parameter sets. This clearly defines the feature boundaries of each growth stage of the cucumber, quantifies the growth state of each stage, and provides staged and standardized growth data support for visualization generation. Furthermore, based on the aforementioned multiple growth stage identifiers and growth state parameter sets, a dynamic visualization interface for the facility-grown cucumbers is generated. This interface includes growth animations, parameter curves, and stage annotation information. Based on these multiple growth stage identifiers and parameter sets, the system drives 3D model deformation animations, draws parameter curves, annotates stage boundaries and descriptive text, and simultaneously displays environmental data, achieving a continuous, intuitive, and interactive visualization of the growth process. Finally, in response to received interaction commands from the target user, the visualization interface is updated accordingly to achieve dynamic responsiveness.By responding to user commands to drive the model to perform backtracking or simulation predictions and updating the visualization interface content in real time, it transforms from a passive display into an interactive and explorable "digital twin," supporting users to conduct hypothesis analysis and gain a deeper understanding of the impact of environmental and other factors on growth.

[0081] Further reference Figure 2 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a facility cucumber growth visualization device based on multimodal time-series data. These device embodiments are similar to... Figure 1 Corresponding to the method embodiments shown, this facility cucumber growth visualization device based on multimodal time-series data can be specifically applied to various electronic devices.

[0082] like Figure 2 As shown, a visualization device 200 for greenhouse cucumber growth based on multimodal time-series data includes: a preprocessing unit 201, a feature extraction unit 202, a construction unit 203, a processing and detection unit 204, a generation unit 205, and an update unit 206. The preprocessing unit 201 is configured to preprocess the acquired phenotypic image sequence and corresponding environmental parameter sequence during the greenhouse cucumber growth process to generate a preprocessed phenotypic image sequence and a preprocessed environmental parameter sequence. The feature extraction unit 202 is configured to extract features from the preprocessed phenotypic image sequence and the preprocessed environmental parameter sequence to generate a fused feature vector sequence arranged in chronological order. The construction unit 203 is configured to construct a growth time-series data model representing the dynamic growth law of the greenhouse cucumber based on the fused feature vector sequence. The processing and detection unit 204 is configured to perform smoothing processing and change point detection on the key growth parameters of the greenhouse cucumber based on the growth time-series data model to generate multiple growth stage identifiers and corresponding multiple growth state parameter sets. The generation unit 205 is configured to dynamically generate a visualization interface for the facility cucumber based on the multiple growth stage identifiers and the multiple growth state parameter sets. The visualization interface includes growth animation, parameter curves, and stage annotation information. The update unit 206 is configured to update the visualization interface in response to receiving an interaction command from a target user.

[0083] It is understandable that the units described in the facility cucumber growth visualization device 200 based on multimodal time-series data are similar to the reference units. Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the facility cucumber growth visualization device 200 based on multimodal time-series data and the units contained therein, and will not be repeated here.

[0084] The following is for reference. Figure 3It shows a schematic diagram of the structure of an electronic device (e.g., an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0085] like Figure 3 As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0086] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0087] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0088] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0089] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0090] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: preprocess the phenotypic image sequence and corresponding environmental parameter sequence collected during the growth process of the greenhouse cucumber to generate a preprocessed phenotypic image sequence and a preprocessed environmental parameter sequence; extract features from the preprocessed phenotypic image sequence and the preprocessed environmental parameter sequence to generate a fusion feature vector sequence arranged in chronological order; construct a growth time-series data model characterizing the dynamic growth law of the greenhouse cucumber based on the fusion feature vector sequence; perform smoothing and change point detection on key growth parameters of the greenhouse cucumber based on the growth time-series data model to generate multiple growth stage identifiers and corresponding multiple growth state parameter sets; dynamically generate a visualization interface for the greenhouse cucumber based on the multiple growth stage identifiers and the multiple growth state parameter sets, the visualization interface including: growth animation, parameter curves, and stage annotation information; and update the visualization interface according to the interaction command received from the target user to complete the dynamic response of the visualization interface.

[0091] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0093] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a preprocessing unit, a feature extraction unit, a construction unit, a processing and detection unit, a generation unit, and an update unit. The names of these units do not necessarily limit the specific unit itself; for example, a preprocessing unit may be described as "a unit that preprocesses the phenotypic image sequence and the corresponding environmental parameter sequence collected during the cucumber generation process to generate a preprocessed phenotypic image sequence and a preprocessed environmental parameter sequence."

[0094] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0095] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for visualizing the growth of cucumbers in greenhouses based on multimodal time-series data, comprising: The phenotypic image sequences and corresponding environmental parameter sequences collected during the growth process of facility cucumbers are preprocessed to generate preprocessed phenotypic image sequences and preprocessed environmental parameter sequences. Feature extraction is performed on the preprocessed phenotypic image sequence and the preprocessed environmental parameter sequence to generate a fused feature vector sequence arranged in chronological order; Based on the fused feature vector sequence, a growth time series data model characterizing the growth dynamics of the facility cucumber is constructed; Based on the growth time series data model, the key growth parameters of the greenhouse cucumber are smoothed and change points are detected to generate multiple growth stage identifiers and corresponding multiple growth state parameter sets. Based on the multiple growth stage identifiers and the multiple growth state parameter sets, a visualization interface for the greenhouse cucumber is dynamically generated. The visualization interface includes: growth animation, parameter curves, and stage labeling information. In response to receiving an interaction command from the target user, the visualization interface is updated according to the interaction command to complete the dynamic response of the visualization interface.

2. The method according to claim 1, wherein, The preprocessing of the phenotypic image sequence and corresponding environmental parameter sequence collected during the cucumber generation process to generate a preprocessed phenotypic image sequence and a preprocessed environmental parameter sequence includes: The phenotypic image sequence is subjected to image format unification and white balance correction to generate a first intermediate image sequence; The first intermediate image sequence is subjected to background removal and target cucumber plant segmentation to generate a second intermediate image sequence; The second intermediate image sequence is normalized in size and filtered for noise to generate a preprocessed phenotypic image sequence; Missing values ​​in the environmental parameter sequence are imputed to generate a first intermediate environmental parameter sequence; The intermediate environmental parameter sequence is subjected to timestamp-based data alignment and normalization to generate a preprocessed environmental parameter sequence.

3. The method according to claim 1, wherein, The step of extracting features from the preprocessed phenotypic image sequence and the preprocessed environmental parameter sequence to generate a fused feature vector sequence arranged in chronological order includes: Visual features are extracted from the preprocessed phenotypic image sequence to generate a set of phenotypic visual feature vectors; Using the target algorithm, geometric parameters are extracted from the preprocessed phenotypic image sequence to generate a set of phenotypic geometric feature parameters; Temporal features are extracted from the preprocessed environmental parameter sequence to generate an environmental temporal feature vector set; The phenotypic visual feature vector set and the phenotypic geometric feature parameter set are dimension-unified and concatenated to generate a comprehensive phenotypic feature vector set; The comprehensive phenotypic feature vector set and the environmental temporal feature vector set are associated and fused according to timestamps to generate a fused feature vector sequence arranged in chronological order.

4. The method according to claim 1, wherein, The step of constructing a growth time-series data model characterizing the dynamic growth pattern of the greenhouse cucumber based on the fused feature vector sequence includes: The fused feature vector sequence is input into the target encoder to obtain a hidden state sequence representing growth context information; The fused feature vector sequence is input into the corresponding pre-trained univariate time series prediction model to generate the prediction residual sequence of each key growth parameter. The hidden state sequence is associated and concatenated with the predicted residual sequence to construct an enhanced temporal feature sequence; The enhanced temporal feature sequence is input into a regression head based on the growth rate equation to output the growth dynamics information of the greenhouse cucumber. Based on the aforementioned growth dynamics information, a growth time series data model is generated that can predict future growth status and simulate environmental response relationships.

5. The method according to claim 1, wherein, Based on the growth time-series data model, the key growth parameters of the greenhouse cucumber are smoothed and change points are detected to generate multiple growth stage identifiers and corresponding sets of multiple growth state parameters, including: Extract the original time series of multiple key growth parameters from the growth time series data model; The original time series of each of the multiple key growth parameters is smoothed by moving average to generate multiple corresponding smoothed time series. For each of the plurality of smooth time series sequences, change point detection based on cumulative difference is performed to determine potential switching time points of multiple growth stages; Based on the potential switching time points of the multiple growth stages and the preset stage threshold information, the entire growth cycle of the facility cucumber is divided to generate multiple growth stage identifiers. For each of the multiple growth stage identifiers, the range of corresponding key growth parameters is determined to generate multiple sets of growth state parameters.

6. A facility cucumber growth visualization device based on multimodal time-series data, comprising: The preprocessing unit is configured to preprocess the phenotypic image sequence and the corresponding environmental parameter sequence collected during the cucumber generation process to generate a preprocessed phenotypic image sequence and a preprocessed environmental parameter sequence. The feature extraction unit is configured to extract features from the preprocessed phenotypic image sequence and the preprocessed environmental parameter sequence to generate a fused feature vector sequence arranged in chronological order. The building unit is configured to construct a growth time-series data model characterizing the growth dynamics of the facility cucumber based on the fused feature vector sequence; The processing and detection unit is configured to perform smoothing and change point detection on the key growth parameters of the facility cucumber based on the growth time series data model, so as to generate multiple growth stage identifiers and corresponding multiple growth state parameter sets. The generation unit is configured to dynamically generate a visualization interface for the facility cucumber based on the plurality of growth stage identifiers and the plurality of growth state parameter sets. The visualization interface includes: growth animation, parameter curves, and stage labeling information. The update unit is configured to update the visualization interface in response to receiving an interaction instruction from a target user.

7. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

Images