Greenhouse environment factor high-dimensional data compression method and device based on self-encoding neural network

Through the self-encoding neural network, the greenhouse environmental factor data is filled and high-order feature extraction, combined with the idea of ​​vector quantization, the problem of low processing efficiency of greenhouse environmental factor data in the existing technology is solved, and efficient data compression and quality improvement is achieved. Greenhouse environmental regulation provides strong support.

CN119026639APending Publication Date: 2024-11-26ZHONGSHAN POLYTECHNIC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411242005.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-05
Publication Date
2024-11-26

Smart Images

  • Figure CN119026639A_ABST
    Figure CN119026639A_ABST
Patent Text Reader

Abstract

The invention discloses a greenhouse environment factor high-dimensional data compression method based on a self-encoding neural network. The method comprises the following steps: performing data filling on collected greenhouse environment factor data; performing high-order feature extraction on the greenhouse environment factor data after data filling based on a pre-trained deep adaptive balanced self-encoding neural network to obtain high-order features corresponding to the greenhouse environment factor data, the deep adaptive balanced self-encoding neural network comprising a plurality of adaptive balanced encoders, the adaptive balance encoder comprises an input layer and an adaptive balance layer, the input layer is used for acquiring greenhouse environment factor data, and the adaptive balance layer is used for performing normalization processing on the greenhouse environment factor data; and mapping the high-order features to a pre-trained codebook to complete compression. The invention provides a greenhouse environment factor high-dimensional data compression method based on a self-encoding neural network, which can realize efficient compression of greenhouse environment factor high-dimensional data, and has higher compression ratio and better data quality.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of greenhouse environment data processing, and in particular to a method and device for compressing high-dimensional data of greenhouse environment factors based on an autoencoder neural network. Background Art

[0002] In modern smart agriculture, precise monitoring and control of greenhouse environmental factors are crucial to improving crop growth conditions, optimizing resource utilization, and increasing agricultural output. However, with the rapid development of the Internet of Things and sensor technology, the amount of greenhouse environmental factor data collected has increased dramatically, and its high dimension, large data volume, and information redundancy have become increasingly prominent. These high-dimensional data not only bring huge challenges to data storage, transmission, and processing, but also affect the accuracy and efficiency of subsequent data analysis. Therefore, studying effective high-dimensional data compression methods has become an urgent need in the current field of agricultural information technology. Through data compression, the dimension and redundancy of data can be reduced, the speed and efficiency of data processing can be improved, and strong support can be provided for the precise control of greenhouse environment.

[0003] The current status of foreign and domestic research on high-dimensional data compression methods is booming, each with its own unique characteristics and advantages. Abroad, the research on high-dimensional data compression has formed a relatively complete system, and many research institutions and scholars are committed to exploring more efficient and accurate data compression algorithms.

[0004] The paper "Dos S, Giovanis D, et al. Grassmannian diffusion maps-based dimension reduction and classification for high-dimensional data [J]. SIAM Journal on Scientific Computing, 2022, 44 (2): B250-B274." proposed a new nonlinear dimensionality reduction technique, Grassmannian diffusion maps (GDMaps). GDMaps consists of two stages. The first is point-wise linear dimensionality reduction, in which each high-dimensional object is mapped to a Grassmann manifold representing the low-dimensional subspace in which it is located. The second stage is dimensionality reduction based on multi-point nonlinear kernels, using diffusion mapping to identify the subspace structure of points on the Grassmann manifold. However, in the dimensionality reduction stage based on multi-point nonlinear kernels, if the kernel function is not selected or the parameters are not set properly, it may lead to inaccurate identification of the subspace structure, thereby further reducing the compressed peak signal-to-noise ratio.

[0005] The paper "Xing S, Islam M. Utilizing differential characteristics of high dimensional data as a mechanism for dimensionality reduction [J]. Pattern recognition letters, 2022, 157: 1-7." proposes a generally applicable dimensionality reduction strategy that analyzes high dimensional data by utilizing the differential characteristics of data relative to a reference data set, and optimizes the distinguishing features between the two sets of data through correlation transformation and covariance matrix scaling. However, the operations of correlation transformation and covariance matrix scaling may be affected by noise or outliers, resulting in unsatisfactory optimization results of distinguishing features.

[0006] The literature "Shao Zheng, Zeng Li, et al. Wavelet method for high-dimensional image compression [J]. Microcomputer Development, 2005, 15 (2): 6-9" proposed a wavelet method for high-dimensional image compression and verified its compression ability through four-dimensional cardiac ultrasound images. However, too many decomposition layers may lead to excessive information loss, while too few decomposition layers may not be able to fully remove redundant information in the data, thus affecting the compression effect.

[0007] The document "Yuan Xiugui, Zhang An. Application of Ridgelet transform in image compression [J]. Computer Engineering and Science, 2005, 27(11): 34-35." proposed a method based on orthogonal finite Ridgelet transform. However, the choice of basis function of Ridgelet transform has an important influence on the compression effect. If the basis function does not match the characteristics of the signal, it may cause the important features of the signal to be lost during the transformation process, thereby reducing the peak signal-to-noise ratio after compression.

[0008] In summary, in the prior art, data processing methods for greenhouse environmental factors generally have the problem of poor results. Summary of the invention

[0009] In order to address the deficiencies in the prior art, the present invention provides a greenhouse environmental factor high-dimensional data compression method based on an autoencoding neural network, which can achieve efficient compression of high-dimensional data of greenhouse environmental factors and has a higher compression ratio and better data quality.

[0010] In order to achieve the above object, the specific scheme adopted by the present invention is:

[0011] The high-dimensional data compression method of greenhouse environmental factors based on autoencoder neural network includes the following steps:

[0012] Fill in the collected greenhouse environmental factor data;

[0013] Based on the pre-trained deep adaptive balanced autoencoder neural network, high-order features of the greenhouse environmental factor data after data filling are extracted to obtain high-order features corresponding to the greenhouse environmental factor data. The deep adaptive balanced autoencoder neural network includes multiple adaptive balanced encoders. The adaptive balanced encoder includes an input layer and an adaptive balancing layer. The input layer is used to obtain the greenhouse environmental factor data, and the adaptive balancing layer is used to normalize the greenhouse environmental factor data.

[0014] High-order features are mapped into pre-trained codebooks to achieve compression.

[0015] Preferably, the specific method for filling the collected greenhouse environmental factor data includes:

[0016] Group greenhouse environmental factor data by category;

[0017] The improved regression equation with random error factor is used to fill each set of greenhouse environmental factor data one by one. The improved regression equation is:

[0018]

[0019] Among them, β n is the greenhouse environmental factor data to be filled, θ0 is the initial regression coefficient, θ g is the regression coefficient of the g-th category, G is the number of missing greenhouse environmental factor data, and λ gn is the nth missing value in the gth greenhouse environmental factor with missing data, and ωn is the random error factor.

[0020] Preferably, the deep adaptive balanced autoencoder neural network includes a hidden layer and an output layer, wherein the hidden layer is used to perform feature extraction and dimensionality reduction processing on the normalized greenhouse environmental factor data to obtain high-order features, and the output layer is used to output high-order features.

[0021] Preferably, the training method of the deep adaptive balanced autoencoder neural network includes:

[0022] Construct a training set based on existing greenhouse environmental factor data;

[0023] Construct multiple basic encoders arranged in order and initialize network parameters, including bias parameters and weight parameters;

[0024] Input the training set into the input layer of the basic encoder, perform unsupervised training on the basic encoder, and input the output result of the output layer of the previous basic encoder into the input layer of the next basic encoder;

[0025] When the number of training basic encoders reaches a preset threshold, the training is terminated to obtain multiple adaptive balanced encoders, and all the adaptive balanced encoders are arranged in sequence to obtain a deep autoencoding neural network.

[0026] Preferably, the training method of the deep adaptive balanced autoencoder neural network also includes:

[0027] Adding an adaptive balancing layer between the input layer and the hidden layer of each trained adaptive balanced encoder to obtain a deep adaptive balanced autoencoder neural network;

[0028] Parameter fine-tuning of deep adaptive balanced autoencoder neural networks.

[0029] Preferably, the specific method for fine-tuning parameters of the deep adaptive balanced autoencoder neural network includes:

[0030] Deriving the gradients of the two affine transformation parameters ψ and ζ in the adaptive balancing layer, and updating ψ and ζ based on the derivative results;

[0031] The specific method for deriving ψ and ζ is:

[0032]

[0033] Among them, U(κ) is the reconstruction error of the batch, u i is the regularization number of the adaptive balancing layer in the i-th adaptive balancing encoder, and n is the number of samples.

[0034] Preferably, the specific method for the adaptive balancing layer to normalize the greenhouse environmental factor data is:

[0035]

[0036] Among them, X N Normalized greenhouse environmental factor data, X is the greenhouse environmental factor data after data filling, X min is the minimum value of X, X max is the maximum value of X.

[0037] Preferably, the specific method of high-order feature extraction includes:

[0038] Based on the expected mean and standard deviation, balanced regularization is performed on the high-dimensional data in the greenhouse environmental factors:

[0039] R[α]=R A [ A ];

[0040]

[0041] Among them, R[α] is the expected standard deviation of high-dimensional data, R A is the regularization function, χ A is the data mean of each mini-batch set in the adaptive balancing layer, γ A is the data variance of each mini-batch set in the adaptive balancing layer;

[0042] The high-dimensional data X' in the processed greenhouse environmental factors is mapped to a nonlinear space to obtain high-order features:

[0043]

[0044] in, is the extracted high-order feature, ν is a constant, and X' is the processed high-dimensional data.

[0045] Preferably, when mapping the high-order features into a predetermined codebook, the high-order features are converted into index codes of codewords in the codebook.

[0046] The present invention further provides a greenhouse environmental factor high-dimensional data compression device based on an autoencoder neural network, which is used to implement the greenhouse environmental factor high-dimensional data compression method based on an autoencoder neural network as described above, and the device comprises:

[0047] A data acquisition module, used to obtain the greenhouse environmental factors;

[0048] A first data processing module, for extracting high-order features from greenhouse environmental factors based on the deep adaptive balanced autoencoder neural network;

[0049] The second data processing module is used to compress the high-order features.

[0050] The present invention realizes the extraction of high-order features of greenhouse environmental factors by constructing a DSBAE model. Based on the idea of ​​vector quantization, a high-dimensional data compression method is designed to realize high-dimensional data compression of greenhouse environmental factors, reduce the dimension and redundancy of the data, and also has a higher compression ratio and better data quality, retains key information, reduces data dimension, and provides strong support for subsequent greenhouse environmental regulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 is a flow chart of the method of the present invention;

[0053] Figure 2 is a block diagram of the adaptive balanced encoder;

[0054] Figure 3 It is the probability distribution diagram of the file after high-dimensional features are compressed;

[0055] Figure 4 It is a test result diagram of the compression ratio of the present invention and two prior arts;

[0056] Figure 5 It is a test result diagram of the peak signal-to-noise ratio of the present invention and two prior arts. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] like Figure 1 As shown, the high-dimensional data compression method of greenhouse environmental factors based on autoencoder neural network includes S1 to S3.

[0059] S1. Fill the collected greenhouse environmental factor data. In the process of collecting greenhouse environmental factor data, it is easy to cause the loss of greenhouse environmental factor data due to factors such as sensor failure, data transmission error or unreasonable setting of data collection time interval, which leads to the decrease of data integrity and accuracy of greenhouse environmental factor data, and in severe cases, it will affect the compression process of greenhouse environmental factor high-dimensional data. Therefore, the present invention first fills the greenhouse environmental factor data to improve the integrity and accuracy of the greenhouse environmental factor data, thereby ensuring the accuracy of subsequent processing. In S1, the specific method of filling the collected greenhouse environmental factor data includes S11 to S12.

[0060] S11, grouping greenhouse environmental factor data by category. Greenhouse environmental factor data includes many different types of data, such as humidity data, temperature data, and light data. Different types of data have different data structures and different value ranges. Therefore, when filling in data, different types of data need to be processed separately. Therefore, the greenhouse environmental factor data are first grouped based on categories, and each group of data corresponds to one category.

[0061] S12. Use the improved regression equation with random error factor to fill each set of greenhouse environmental factor data one by one. The improved regression equation is:

[0062]

[0063] Among them, β n is the greenhouse environmental factor data to be filled, θ0 is the initial regression coefficient, θ g is the regression coefficient of the g-th category, G is the number of missing greenhouse environmental factor data, λ gn is the nth missing value in the gth greenhouse environmental factor with missing data, ω n is the random error factor.

[0064] Regression equation is a commonly used data filling method. However, the conventional regression equation has shortcomings, mainly manifested in the problem of sample distribution distortion, resulting in a high difference between the filled data and the real data. Therefore, the present invention introduces a random error factor ω in S12 n , through ω n Flexible adjustments can make the estimated values ​​of the regression equation different, thus solving the problem of possible sample distribution distortion.

[0065] S2. Based on the pre-trained deep adaptive balanced autoencoder neural network, high-order features are extracted from the greenhouse environmental factor data after data filling to obtain high-order features corresponding to the greenhouse environmental factor data. The deep adaptive balanced autoencoder neural network includes multiple adaptive balanced encoders, and the adaptive balanced encoder includes an input layer and an adaptive balancing layer. The input layer is used to obtain the greenhouse environmental factor data, and the adaptive balancing layer is used to normalize the greenhouse environmental factor data. The deep adaptive balanced autoencoder neural network includes a hidden layer and an output layer, wherein the hidden layer is used to extract features and reduce the dimension of the normalized greenhouse environmental factor data to obtain high-order features, and the output layer is used to output high-order features.

[0066] like Figure 2As shown, in the greenhouse environmental factor data, different types of data may have large differences and imbalances, and the overall situation and change trend of the greenhouse environment are not well characterized. In order to alleviate the negative impact of these problems, the present invention is based on a deep autoencoder neural network (DAE), and by adding an adaptive balancing layer between the input layer and the hidden layer of the encoder, a deep adaptive balanced autoencoder neural network (DSBAE) is constructed, and then the high-order feature extraction of the greenhouse environmental factor data is implemented by using DSBAE, which can more efficiently characterize the balance of the greenhouse environmental factor data and more accurately describe the state and change of the greenhouse environment. Specifically, the conventional deep autoencoder neural network has the phenomenon of internal covariate migration. As the number of network layers increases, this migration phenomenon will become more significant, making the gradient diffusion problem more serious. The present invention adds an adaptive balancing layer to the encoder of the conventional deep autoencoder neural network to obtain an adaptive balanced encoder. Based on the adaptive balancing layer, a batch normalization algorithm is used to keep the greenhouse environmental factor data input into the encoder in a nonlinear distribution, thereby effectively solving the shortcomings of the conventional deep autoencoder neural network.

[0067] More specifically, the specific method for the adaptive balance layer to normalize the greenhouse environmental factor data is:

[0068]

[0069] Among them, X N Normalized greenhouse environmental factor data, X is the greenhouse environmental factor data after data filling, X min is the minimum value of X, X max is the maximum value of X.

[0070] The specific method of high-order feature extraction includes S21 to S22.

[0071] S21. Balanced regularization is performed on high-dimensional data in greenhouse environmental factors based on expected mean and expected standard deviation:

[0072] R[α]=R A [ A ];

[0073]

[0074] Among them, R[α] is the expected standard deviation of high-dimensional data, R A is the regularization function, χ A is the data mean of each mini-batch set in the adaptive balancing layer, γ Ais the data variance of each small batch set of the adaptive balancing layer. It should be noted that the expected mean and expected standard deviation are both calculated based on high-dimensional data, and the specific calculation method belongs to the conventional technology in this field and will not be repeated here.

[0075] S2. Map the high-dimensional data X' in the processed greenhouse environmental factors to a nonlinear space to obtain high-order features:

[0076]

[0077] in, is the extracted high-order feature, ν is a constant, and X' is the processed high-dimensional data.

[0078] The deep adaptive balanced autoencoder neural network is the core of the present invention and needs to be trained in advance. When the greenhouse environmental factor data is actually processed, the trained deep adaptive balanced autoencoder neural network can process more efficiently. In the present invention, the training method of the deep adaptive balanced autoencoder neural network includes the following steps.

[0079] First, a training set is constructed based on the existing greenhouse environmental factor data. The greenhouse environmental factor data in the training set can be obtained by filling in the data of S1 based on the greenhouse environmental factor data actually collected.

[0080] Secondly, construct a plurality of basic encoders arranged in order, and initialize the network parameters, including bias parameters and weight parameters. The basic encoder adopts the encoder structure of a conventional deep autoencoder neural network, that is, a structure mainly including an input layer, a hidden layer, and an output layer. In addition, in addition to the network parameters including bias parameters and weight parameters, after constructing the basic encoder, it is also necessary to determine the initial values ​​such as batch size, number of neurons, number of neurons, number of iterations, etc. The role of each parameter in the deep autoencoder neural network is common knowledge in the art and will not be repeated here.

[0081] After that, the training set is input into the input layer of the basic encoder, the basic encoder is trained unsupervised, and the output result of the output layer of the previous basic encoder is input into the input layer of the next basic encoder. In this process, the encoding calculation from the input layer to the hidden layer is completed first, and then the decoding calculation from the hidden layer to the output layer is realized. In order to evaluate the effect of the encoding and decoding process, it is necessary to calculate the reconstruction error of the input data and the output data, and then use the back propagation algorithm to obtain the partial derivatives of the corresponding parameters in turn, and finally update the network parameters.

[0082] Subsequently, when the number of training basic encoders reaches a preset threshold, the training is terminated to obtain multiple adaptive balanced encoders, and all the adaptive balanced encoders are arranged in sequence to obtain a deep autoencoding neural network.

[0083] Next, an adaptive balancing layer is added between the input layer and the hidden layer of each trained adaptive balanced encoder to obtain a deep adaptive balanced autoencoder neural network.

[0084] Finally, in order to fully improve the accuracy of the deep adaptive balanced autoencoder neural network, it is also necessary to fine-tune the parameters of the deep adaptive balanced autoencoder neural network. The specific method of fine-tuning the parameters of the deep adaptive balanced autoencoder neural network includes fine-tuning the parameters of the adaptive balancing layer.

[0085] The gradients of the two affine transformation parameters ψ and ζ in the adaptive balancing layer are derived, and ψ and ζ are updated based on the derivative results.

[0086] The specific method for deriving ψ and ζ is:

[0087]

[0088] Among them, U(κ) is the reconstruction error of the batch, u i is the regularization number of the adaptive balancing layer in the i-th adaptive balancing encoder, and n is the number of samples.

[0089] S3, mapping the high-order features to a pre-trained codebook to complete compression. Specifically, when mapping the high-order features to a pre-determined codebook, the high-order features are converted into index codes of codewords in the codebook.

[0090] The codebook is similar to the deep adaptive balanced autoencoder neural network and needs to be determined in advance so that the high-order features can be compressed more efficiently when the greenhouse environmental factor data is actually processed. The specific method for determining the codebook is as follows.

[0091] The first step is to use C i Represents the original greenhouse environmental factor data, whose dimension is L, and c i Represents the extracted high-order features, whose dimension is l, C i With c i One to one correspondence.

[0092] The second step is to set the number of iterations k to 0 and set the average distortion W (-1) Set to W (-1) →∞.

[0093] The third step is to initialize the codebook. i Randomly select Z vectors from and represent them as O j(0) (j=1,2,…,Z).

[0094] The fourth step is to use the nearest neighbor principle as the basis, that is, to give the codebook training vector c i Mapped to the codeword with the closest distance m, then c i The extracted Z vectors O j (0) The best mapping codeword partition result And the following conditions are met:

[0095]

[0096] Where D is a vector mapping function, and for each c i , record its corresponding Index(i)=j.

[0097] The fifth step is to calculate the average distortion. The specific calculation method is:

[0098]

[0099] Step 6: Determine whether the relative error meets the conditions Where ρ is a predetermined relative error threshold. According to the judgment result, the seventh step or the eighth step is selected.

[0100] Step 7: When the relative error meets the condition, the high-order features are mapped to a limited codebook to achieve discrete representation and compression of the data, thereby effectively reducing the demand for storage space and transmission bandwidth. At this time, the mapping dimension reduction calculation is implemented, and the specific calculation method is:

[0101]

[0102] Among them, O j is the low-dimensional code book data of greenhouse environmental factors, Index(i) is the result of the quantization processing of the i-th high-order feature vector, and V(j) is the quantization result of the i-th high-order feature vector corresponding to j.

[0103] Step 8: When the relative error does not meet the above conditions, retain the original codebook.

[0104] Step 9: Complete codebook training to obtain a better set of codewords

[0105] Through the above process, a code book that fully meets the needs can be obtained. When compressing high-dimensional features in S3, in each iteration, an attempt is made to assign data points to the nearest codewords with minimal distortion and update the positions of the codewords. Thus, the original high-dimensional data with high-order features is converted into a series of index codes of low-dimensional data points, thus achieving high-dimensional data compression.

[0106] A high-dimensional data compression device for greenhouse environmental factors based on an autoencoding neural network is used to implement the high-dimensional data compression method for greenhouse environmental factors based on an autoencoding neural network as described above. The device includes a data acquisition module, a first data processing module and a second data processing module.

[0107] The data acquisition module is used to obtain the greenhouse environmental factors.

[0108] The first data processing module is used to extract high-order features from greenhouse environmental factors based on the deep adaptive balanced autoencoder neural network, that is, to execute the process from S1 to S2.

[0109] The second data processing module is used to compress the high-order features, that is, to execute the process of S3.

[0110] In one embodiment of the present invention, the data acquisition module includes a plurality of sensors corresponding to the data categories in the greenhouse environmental factor data. Both the first data processing module and the second data processing module can adopt a computer device with an ARM architecture, an X86 architecture or a CISC architecture processor as the core, which will not be repeated here.

[0111] In order to verify the present invention, a tomato planting greenhouse was selected as the test scene. In this test scene, a large amount of environmental factor data related to tomato growth was collected, such as temperature, humidity, light intensity, carbon dioxide concentration, soil moisture, soil pH value, etc. In order to accurately control the tomato growth environment, a variety of sensors were installed to monitor environmental factors, and 5 PT100 temperature sensors, 4 DHT11 humidity sensors, 3 BH1750 light intensity sensors and 3 MH-Z14A carbon dioxide concentration sensors, 10 YL-69 soil moisture sensors, etc. were laid out, and buried in different soil areas in the greenhouse. These sensors collect data in real time and send it to the data center for storage and analysis through a data transmission system. All sensors collect data every 5 minutes, and each sensor collects one data point each time, and the dimension of the experimental data collected each time is 25 dimensions. The experimental data is stored in the data center in CSV format. Each data point includes a timestamp and the corresponding 25 environmental factor values, including 70GB of data of 25 dimensions of 5 types of temperature, humidity, light, carbon dioxide concentration, and soil moisture.

[0112] First, the above data is filled, and the filling result example is shown in Table 1.

[0113] Table 1. Example of filled data

[0114]

[0115] As shown in Table 1, the number of neurons in the input layer of the adaptive balanced encoder is 10, the number of neurons in the hidden layer is 5, the number of neurons in the output layer is 8, the optimizer is Adam, the loss function is MSE, the batch size is 0.1, and the Epochs is 100. Each sample in the data set is described by multiple features. In high-dimensional data, data points are usually very sparse, that is, the file probability of most features may be 0 or close to 0. After compression, high-dimensional features will be converted into low-dimensional data with fewer features, and the feature bytes appear in a concentrated form.

[0116] The probability distribution of files after high-dimensional feature compression is as follows: Figure 3 As shown, it can be seen that after compression, the file probability distribution is relatively uniform, and the file probability after each byte is compressed is relatively high, indicating that the present invention can effectively improve the utilization rate of storage space.

[0117] The prior art disclosed in the document "Dos S, Giovanis D, et al. Grassmannian diffusion maps-based dimension reduction and classification for high-dimensional data [J]. SIAM Journal on Scientific Computing, 2022, 44 (2): B250-B274." is recorded as method A, and the prior art disclosed in the document "Xing S, Islam M. Utilizing differential characteristics of high dimensional data as a mechanism for dimensionality reduction [J]. Pattern recognition letters, 2022, 157: 1-7." is recorded as method B. The comparison results of the compression rate of the present invention with method A and method B are as follows: Figure 4 As shown, Figure 4 The present invention is recorded as a design method. It can be seen that when the data volume is above 50GB, the compression ratio of the present invention begins to decline to a certain extent, and the decline is relatively low. The compression ratio of method A begins to decline when the data volume is above 30GB, and the compression ratio of method B begins to decline significantly when the data volume is above 40GB. Therefore, the compression ratio of the present invention is the highest and the compression performance is the best. Its robustness performance is also better than the two comparison methods because it performs layer-by-layer regularization on the encoding process of the deep autoencoder neural network, by finding the codeword corresponding to each index, and reconstructing the original data point, assigning the data point to the nearest codeword, and updating the position of the codeword to minimize the quantization error to ensure the compression effect.

[0118] Still using method A and method B as a comparison, the peak signal-to-noise ratio comparison results of the three methods are as follows: Figure 5 As shown, Figure 5 The present invention is still recorded as a design method. It can be seen that the peak signal-to-noise ratio of method A and method B both decreases significantly as the sampling rate increases, while the peak signal-to-noise ratio of the method in this paper does not decrease significantly as the sampling rate increases, and only decreases slightly. The peak signal-to-noise ratio of the present invention is the highest, because it combines the results of high-order feature vector quantization, maps high-dimensional data to a limited set of codebooks, and realizes discrete representation and compression of data. When selecting a set of data points as the initial codewords, it is not random. Once the codebook training is completed, it can be used directly to compress the data. The dimension and redundancy of the data are optimized to ensure the quality of the peak signal-to-noise ratio.

[0119] In summary, the present invention realizes the extraction of high-order features of greenhouse environmental factors by constructing a DSBAE model, designs a high-dimensional data compression method based on the idea of ​​vector quantization, realizes high-dimensional data compression of greenhouse environmental factors, reduces the dimension and redundancy of the data, and also has a higher compression ratio and better data quality, retains key information, reduces data dimension, and provides strong support for subsequent greenhouse environmental regulation.

[0120] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0121] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A high-dimensional data compression method for greenhouse environmental factors based on an autoencoder neural network, characterized in that: The method comprises the following steps: filling the collected greenhouse environmental factor data; Based on the pre-trained deep adaptive balanced autoencoder neural network, high-order features of the greenhouse environmental factor data after data filling are extracted to obtain high-order features corresponding to the greenhouse environmental factor data. The deep adaptive balanced autoencoder neural network includes multiple adaptive balanced encoders. The adaptive balanced encoder includes an input layer and an adaptive balancing layer. The input layer is used to obtain the greenhouse environmental factor data, and the adaptive balancing layer is used to normalize the greenhouse environmental factor data. High-order features are mapped into pre-trained codebooks to achieve compression.

2. The greenhouse environmental factor high-dimensional data compression method based on the autoencoder neural network according to claim 1, characterized in that: The specific methods for filling the collected greenhouse environmental factor data include: Group greenhouse environmental factor data by category; The improved regression equation with random error factor is used to fill each set of greenhouse environmental factor data one by one. The improved regression equation is: Among them, β n is the greenhouse environmental factor data to be filled, θ0 is the initial regression coefficient, θ g is the regression coefficient of the g-th category, G is the number of missing greenhouse environmental factor data, and λ gn is the nth missing value in the gth greenhouse environmental factor with missing data, ω n is the random error factor.

3. The greenhouse environmental factor high-dimensional data compression method based on the autoencoder neural network according to claim 1, characterized in that: The deep adaptive balanced autoencoder neural network includes a hidden layer and an output layer, wherein the hidden layer is used to extract features and perform dimension reduction processing on the normalized greenhouse environmental factor data to obtain high-order features, and the output layer is used to output high-order features.

4. The greenhouse environmental factor high-dimensional data compression method based on the autoencoder neural network according to claim 3, characterized in that: The training method of the deep adaptive balanced autoencoder neural network includes: Construct a training set based on existing greenhouse environmental factor data; Construct multiple basic encoders arranged in order and initialize network parameters, including bias parameters and weight parameters; input the training set into the input layer of the basic encoder, perform unsupervised training on the basic encoder, and input the output result of the output layer of the previous basic encoder into the input layer of the next basic encoder; When the number of training basic encoders reaches a preset threshold, the training is terminated to obtain multiple adaptive balanced encoders, and all the adaptive balanced encoders are arranged in sequence to obtain a deep autoencoding neural network.

5. The greenhouse environmental factor high-dimensional data compression method based on the autoencoder neural network according to claim 4, characterized in that: The training method of the deep adaptive balanced autoencoder neural network also includes: Adding an adaptive balancing layer between the input layer and the hidden layer of each trained adaptive balanced encoder to obtain a deep adaptive balanced autoencoder neural network; Parameter fine-tuning of deep adaptive balanced autoencoder neural networks.

6. The greenhouse environmental factor high-dimensional data compression method based on autoencoder neural network according to claim 5, characterized in that: The specific methods for fine-tuning parameters of the deep adaptive balanced autoencoder neural network include: Deriving the gradients of the two affine transformation parameters ψ and ζ in the adaptive balancing layer, and updating ψ and ζ based on the derivative results; The specific method for deriving ψ and ζ is: Among them, U(κ) is the reconstruction error of the batch, u i is the regularization number of the adaptive balancing layer in the i-th adaptive balancing encoder, and n is the number of samples.

7. The greenhouse environmental factor high-dimensional data compression method based on an autoencoder neural network according to claim 1, characterized in that: The specific method of normalizing greenhouse environmental factor data by the adaptive balance layer is: Among them, X N Normalized greenhouse environmental factor data, X is the greenhouse environmental factor data after data filling, X min is the minimum value of X, X max is the maximum value of X.

8. The greenhouse environmental factor high-dimensional data compression method based on autoencoder neural network according to claim 1, characterized in that: Specific methods for high-order feature extraction include: Based on the expected mean and standard deviation, balanced regularization is performed on the high-dimensional data in the greenhouse environmental factors: R[a]=R A [x A ]; Among them, R[α] is the expected standard deviation of high-dimensional data, R A is the regularization function, χ A is the data mean of each mini-batch set in the adaptive balancing layer, γ A is the data variance of each mini-batch set in the adaptive balancing layer; The high-dimensional data X' in the processed greenhouse environmental factors is mapped to a nonlinear space to obtain high-order features: in, is the extracted high-order feature, ν is a constant, and X' is the processed high-dimensional data.

9. The greenhouse environmental factor high-dimensional data compression method based on autoencoder neural network according to claim 1, characterized in that: When mapping high-order features to a predetermined codebook, the high-order features are converted into index codes of codewords in the codebook.

10. A greenhouse environmental factor high-dimensional data compression device based on an autoencoder neural network, used to implement a greenhouse environmental factor high-dimensional data compression method based on an autoencoder neural network as claimed in any one of claims 1 to 9, characterized in that: The device comprises: A data acquisition module, used to obtain the greenhouse environmental factors; A first data processing module, for extracting high-order features from greenhouse environmental factors based on the deep adaptive balanced autoencoder neural network; The second data processing module is used to compress the high-order features.