On-tree peach hardness inversion method based on finite element simulation and transfer learning

Abstract

Disclosed in the present invention is an on-tree peach hardness inversion method based on finite element simulation and transfer learning, comprising the following steps: step 1, using a laser Doppler vibrometer to collect vibration signals of an on-tree peach at different growth stages, using preprocessed signals as inputs to a model, using a texture analyzer to measure a peach hardness reference value as an output of the model, and constructing an experimental data set; step 2, establishing a peach vibration finite element model having a high degree of biomimicry, and calculating a simulated vibration response; step 3, changing the material properties of the finite element model in step 2 to obtain peach vibration responses having different hardness, and constructing a simulation data set; and step 4, constructing a domain adversarial neural network, and implementing domain adaptation and hardness inversion by means of adversarial training. In the present invention, simulated vibration data is generated by using a finite element method, and the domain adversarial neural network is constructed for feature transfer, thereby improving the accuracy of on-tree peach hardness inversion under a limited number of samples.

Description

A peach fruit hardness inversion method based on finite element simulation and transfer learning TECHNICAL FIELD

[0001] The present application relates to the field of fruit quality detection, and particularly relates to a peach fruit hardness inversion method based on finite element simulation and transfer learning. BACKGROUND

[0002] Hardness is related to the taste, maturity and storage resistance of peaches, and is a key quality indicator of peaches. Hardness measurement is required throughout the peach industry chain. Peaches harvested within a certain hardness range usually have higher resistance to damage, longer shelf life, and better taste after complete softening. Therefore, pre-harvest hardness monitoring is of great significance for determining the optimal harvesting time of peaches.

[0003] Traditional fruit hardness detection methods are destructive and cannot meet the needs of large-scale detection. The acoustic vibration method analyzes the vibration response of the vibrating peach fruit, extracts the vibration features, and then obtains the hardness of the peach fruit, which can reduce the mechanical damage to the peach fruit. However, in the application before harvesting, due to the interference of natural wind, the vibration response of the peach fruit is full of noise, and even workers with prior knowledge have difficulty in accurately extracting vibration features from the signal. At present, the feature extraction method based on deep learning has good performance in vibration signal processing, which can unsupervisedly extract features related to the target task from a large number of signals, and is widely used in mechanical fault diagnosis, electrocardiogram analysis, agricultural product quality detection and other fields. However, traditional deep learning methods require a large number of data samples, and assume that the training and testing data have the same feature distribution, which requires a large number of samples and high quality. Collecting more samples not only increases the investment of manpower, material resources and funds, but also makes workers make various deviations due to fatigue. Especially in agricultural field applications, it is almost impossible to obtain so many high-quality and evenly distributed samples. Numerical simulation provides an effective solution to this problem. Through numerical simulation technology, the vibration of the object can be simulated, and a large amount of simulation data can be obtained. Finite element analysis method is a commonly used numerical simulation method. However, because the shape and structure of fruits are complex and the material properties are difficult to measure, the shape, internal structure or material properties of existing fruit finite element models are oversimplified, so that there is a large difference between the simulation data of the finite element model and the experimental data. Constructing an accurate finite element model can improve the similarity between the simulation data and the experimental data, but the measurement environment is complex and variable, and the size and shape of the fruit are different, which inevitably leads to some differences in the feature distribution of the simulation data and the experimental data, which reduces the prediction accuracy of the deep learning prediction model. Therefore, how to accurately invert the pre-harvest hardness of peach fruit under the condition of small sample size is a problem to be solved. SUMMARY

[0004] The purpose of the present application is to solve the above problems, and a peach fruit hardness inversion method based on finite element simulation and transfer learning is proposed, which solves the problem of low accuracy caused by the oversimplification of traditional fruit finite element model, overcomes the problem of small sample size and large feature distribution difference, and provides an effective solution for realizing accurate pre-harvest hardness inversion of peach fruit.

[0005] To solve the above technical problems, the technical scheme provided by the present application is as follows: a peach fruit hardness inversion method based on finite element simulation and transfer learning, comprising the following steps:

[0006] Step 1: Use a laser Doppler vibration meter to collect vibration signals of peach fruits at different growth stages, and use a texture analyzer to measure the peach fruit hardness reference value as the output of the model after preprocessing, and construct an experimental data set;

[0007] Step 2: Establish a peach fruit vibration finite element model with high bionics and calculate the simulated vibration response;

[0008] Step 2.1: Use a handheld laser three-dimensional scanner to obtain the three-dimensional point cloud of the surface of the peach fruit, kernel and nut, and construct a three-dimensional geometric model with accurate shape and structure of the peach fruit through reverse modeling;

[0009] Step 2.2: Use a texture analyzer and a universal mechanical testing machine to test the compression elastic modulus of peach flesh, kernel and nut and the tensile elastic modulus of fruit skin as the material properties of the finite element model;

[0010] Step 2.3: According to the three-dimensional geometric model of the peach fruit, the material properties measured in step 2.2, the experimental excitation and the fruit constraint conditions, a peach fruit vibration finite element model with high bionics is established, and the simulated vibration response of the response point is simulated and calculated;

[0011] Step 3: Change the material properties of the finite element model in step 2 to obtain the simulated vibration response of peach fruits with different hardness;

[0012] Step 4: Construct a domain adversarial neural network, use the simulated data with hardness value as the source domain and the experimental data without hardness value as the target domain for training, and realize domain transfer and hardness inversion through adversarial training;

[0013] Step 4.1: Construct a feature extraction network based on multi-scale convolution and channel attention mechanism to realize adaptive feature extraction;

[0014] Step 4.2: Construct a predictor to predict the hardness of the features through a fully connected layer. And select mean square error (MSE) as the loss function of the predictor:

[0015] In the formula, L is the loss function of the predictor; N is the number of samples; y i is the hardness value; is the predicted hardness value.

[0016] Step 4.3, constructing the domain discriminator. This step maximizes the domain classification loss by a gradient reversal layer that automatically reverses the gradient direction during backpropagation, confusing the target domain data with the source domain data. The cross-entropy loss is used as the loss function of the domain discriminator:

[0017] In the formula, L is the loss function of the predictor; N is the number of samples; y i is the hardness value; is the predicted hardness value.

[0018] Step 4.4, the loss function of the network is composed of the predictor loss function and the domain discriminator loss function: L = L p -λL d

[0019] In the formula, L is the total loss function; λ is the parameter for adjusting the strength of the confrontation.

[0020] As a further improvement of the present application, a band-pass filter is used for filtering in the signal preprocessing process, and the power spectral density of the vibration response is calculated as the model input.

[0021] As a further improvement of the present application, in step 1, the force-displacement curve of the puncture process is obtained through the puncture experiment, and the initial slope of the curve is used as the reference value of the peach fruit hardness.

[0022] As a further improvement of the present application, in step 2.1, the peach fruit point cloud is denoised and curved in Geomagic software, and then imported into Solidworks software to combine each part according to the physiological structure of the peach fruit, and construct a complete three-dimensional model of the peach fruit.

[0023] As a further improvement of the present application, in step 2.2, the flesh is prepared into a 15x15x15mm cube sample, and a texture analyzer is used for compression test. After obtaining the force-displacement curve, the elastic modulus is calculated according to the following formula:

[0024] In the formula, E is the elastic modulus, σ is the stress, ε is the strain, F is the compression force, L is the initial length of the sample, A is the cross-sectional area of the sample, and ΔL is the deformation.

[0025] As a further improvement of the present application, in step 2.2, the complete kernel and nut are placed on the texture analyzer for compression test. After obtaining the force-displacement curve, the elastic modulus is calculated according to the following formula:

[0026] where E is the elastic modulus, F is the force, D is the deformation, K U and K L are constants determined by the curvature of the upper and lower surface contact points, R U and R′ U are the maximum and minimum radii of curvature of the upper surface contact points, R L and R′ L are the maximum and minimum radii of curvature of the lower surface contact points.

[0027] As a further improvement of the present application, in step 2.2, the peel is prepared into a rectangular sample, and a tensile test is performed on the peel using a universal mechanical testing machine, and the elastic modulus thereof is calculated according to the following formula:

[0028] where E is the elastic modulus, σ is the stress, ε is the strain, F is the compression force, L is the initial length of the sample, A is the cross-sectional area of the sample, and ΔL is the deformation.

[0029] As a further improvement of the present application, in step 2.3, a fixed constraint is provided at the stem part of the fruit, and an excitation force is applied at the equatorial part, all modes within 2000 Hz are calculated, and the vibration response output is obtained through transient analysis.

[0030] Compared with the prior art, the present application has the following advantages:

[0031] The bionic peach fruit vibration finite element model of the present application fully considers the complex shape, internal structure and material properties of the peach fruit, accurately constructs a three-dimensional geometric model of the peach fruit through three-dimensional scanning and reverse modeling, and determines the elastic modulus of the peel, pulp, core and kernel of the peach fruit through experiments as the material properties of the finite element, thereby improving the accuracy of the finite element simulation. The present application obtains a simulation data set through the method of finite element simulation for transfer learning, and can realize the inversion of the hardness of the peach fruit on the tree based on the acoustic vibration method under the condition of a small sample size. BRIEF DESCRIPTION OF DRAWINGS

[0032] Fig. 1 is a flowchart of the embodiment of the present application.

[0033] Fig. 2 is a diagram of the structure of a domain adversarial neural network.

[0034] Fig. 3 is a diagram of the structure of a peach fruit vibration response feature extraction network.

[0035] Fig. 4 is a feature visualization diagram of different network outputs. DETAILED DESCRIPTION

[0036] The present application will be further described below with reference to the accompanying drawings.

[0037] With reference to Figures 1-4, a peach fruit hardness inversion method based on finite element simulation and transfer learning, the method comprising the following steps:

[0038] Step 1, collect the vibration response data of the peach fruit on the tree and measure the hardness to construct the experimental data set;

[0039] Step 1.1, use a laser Doppler vibration meter to collect the vibration signals of the peach fruit at different growth stages to obtain the vibration response data of the peach fruit with different hardness and size, and use a band-pass filter to reduce noise, with a cutoff frequency of 5-2000Hz, and use an autoregressive model method to calculate the power spectral density of the vibration response as the model input;

[0040] Step 1.2, use a texture analyzer to perform a penetration test on the peach fruit, with a probe diameter of 5mm, a loading speed of 0.5mm / s, and a loading distance of 8mm, and calculate the initial slope of the force-displacement curve to obtain the hardness reference value as the output of the model;

[0041] Step 2, establish a peach fruit vibration finite element model with high bionics and calculate the simulated vibration response;

[0042] Step 2.1, use a handheld laser three-dimensional scanner to obtain the three-dimensional point cloud of the outer surface of 10 different peach fruits, kernels and nuts, denoise the point cloud of each part of the peach fruit in Geomagic software, and then import it into Solidworks software to combine each part according to the physiological structure of the peach fruit to construct a complete three-dimensional model of the peach fruit;

[0043] Step 2.2, select 15 peach fruits at different growth stages, prepare the flesh into 15x15x15mm cubic samples, and use a texture analyzer to perform a compression test, select a cylindrical probe with a diameter of 100mm, set the loading speed to 0.1mm / s, and the loading distance to 8mm, obtain the force-displacement curve, and calculate the elastic modulus according to the following formula:

[0044] Where E is the elastic modulus, σ is the stress, ε is the strain, F is the compression force, L is the initial length of the sample, A is the cross-sectional area of the sample, and ΔL is the deformation.

[0045] Place the complete kernel and nut on the texture analyzer for compression test, select a cylindrical probe with a diameter of 100mm, set the loading speed to 0.1mm / s, and the loading distance to 3mm, obtain the force-displacement curve, and calculate the elastic modulus according to the following formula:

[0046] Where E is the elastic modulus, F is the force, D is the deformation, K U and K LR is a constant determined by the curvature of the upper and lower surface contact points U and R' U R is the maximum and minimum radius of curvature of the upper surface contact point L and R' L R is the maximum and minimum radius of curvature of the lower surface contact point, the radius of curvature of the kernel, the kernel top is given by the three-dimensional model.

[0047] The peel is prepared into a rectangular sample, and a tensile test is performed on the peel using a universal mechanical testing machine. The elastic modulus thereof is calculated according to the following formula:

[0048] In the formula, E is the elastic modulus, σ is the stress, ε is the strain, F is the compression force, L is the initial length of the sample, A is the cross-sectional area of the sample, and ΔL is the deformation.

[0049] Step 2.3, according to the three-dimensional geometric model of the peach fruit, the material properties measured in step 2.2, the experimental excitation and the fruit constraint conditions, a peach fruit finite element model with high bionics is established in ANSYS finite element simulation software, fixed constraints are set at the fruit stem part to simulate the external constraints of the peach fruit, and all modes below 2000 Hz are calculated; a 0.5N excitation force is applied at the equatorial part to simulate the gas excitation force in the experiment, and the vibration response output is obtained through transient analysis;

[0050] Step 3, change the material properties of the flesh of the finite element model in step 2, according to the range of the elastic modulus of the flesh measured by the test, uniformly select 100 values in the range of 0.5MPa-2.5MPa as the elastic modulus of the flesh, to obtain the simulation vibration response of peach fruits with different hardness;

[0051] Step 4, as shown in Figure 2, a domain adversarial neural network (DANN) is constructed, which mainly includes a feature extraction network, a hardness predictor, a domain discriminator and a gradient reversal layer. The gradient of the domain discriminator is automatically reversed through the gradient reversal layer to maximize the domain classification error and extract domain-invariant features. In the experiment, the simulation data with hardness values are used as the source domain, and the experimental data without hardness values are used as the target domain for training, and domain transfer and hardness inversion are realized through adversarial training;

[0052] Step 4.1, a feature extraction network as shown in Figure 3 is constructed based on multi-scale convolution and channel attention mechanism, to realize multi-scale feature extraction. It mainly includes two convolution layers, an Inception module, a Squeeze-and-Excitation (SE) module, a Dropout layer, and a Batch Normalization layer and a ReLU activation function and a maximum pooling layer are added after each convolution layer to speed up the convergence speed and improve the nonlinear ability of the network;

[0053] Step 4.2, constructing a predictor, predicting the hardness of the features through a fully connected layer. And selecting mean square error (MSE) as the loss function of the predictor:

[0054] In the formula, L is the loss function of the predictor; N is the number of samples; y i is the hardness value; is the predicted hardness value.

[0055] Step 4.3, constructing a domain discriminator. This step maximizes the domain classification loss by automatically reversing the gradient direction during backpropagation through a gradient reversal layer, confusing the target domain data with the source domain data. Cross-entropy loss is used as the loss function of the domain discriminator:

[0056] In the formula, L is the loss function of the predictor; N is the number of samples; y i is the hardness value; is the predicted hardness value.

[0057] Step 4.4, the loss function of the network is composed of the predictor loss function and the domain discriminator loss function: L = L p -λL d

[0058] In the formula, L is the total loss function; λ is a parameter for adjusting the strength of the confrontation.

[0059] 1000 vibration response data obtained by simulation are used as the source domain, and 300 vibration response data of peach fruits measured by experiment are used as the target domain for training. The learning rate is set to 0.001, the penalty parameter λ is set to 1, the iteration number is 100 times, and the batch processing amount is 64 for training.

[0060] In order to verify the effectiveness of the present application, an Inception-SE network with the same network structure and an Inception-SE+DANN network are selected for comparison, and the results of feature visualization using t-SNE are shown in (a) and (b) of FIG. 4. The results show that the Inception-SE+DANN network in the present application can effectively extract domain-invariant features. The root mean square error (RMSE) and the determination coefficient (R 2 ) are used as evaluation indexes to evaluate the effect of the method, and the results are shown in Table 1. From the comparison results, it can be seen that the feature extraction network proposed by the present application can effectively extract features related to the hardness of peach fruits, and the method of obtaining simulation vibration data by finite element method and performing transfer learning can improve the accuracy of peach fruit hardness inversion on the tree under the condition of limited sample size.

[0061] Table 1

Claims

1. A method for peach fruit firmness inversion on tree based on finite element simulation and transfer learning, characterized in that, The method comprises the following steps: Step 1, using a laser Doppler vibrometer to collect vibration signals of peach fruits at different growth stages on the tree, after pretreatment, as the input of the model, and using a texture analyzer to measure the peach fruit hardness reference value, an experimental data set is constructed; Step 2, a peach fruit vibration finite element model with bionics is established and the simulation vibration response is calculated; Step 3, the material properties of the peach fruit vibration finite element model with bionics obtained in step 2 are changed to obtain the vibration response of peach fruits with different hardness, and a simulation data set is constructed; Step 4, a domain adversarial neural network is constructed, the simulation data set obtained in step 3 is used as the source domain, and the experimental data set obtained in step 1 is used as the target domain for training, and a peach fruit hardness inversion model is obtained through adversarial training; Step 5, input the vibration response of the peach fruit on the tree into the peach fruit hardness inversion model to obtain the peach fruit hardness value, and guide the fruit picking.

2. The finite element simulation and transfer learning based peach fruit firmness inversion method on tree according to claim 1, characterized in that, In step 1, the pretreatment adopts the method of band-pass filtering and power spectrum density estimation.

3. The finite element simulation and transfer learning based peach fruit firmness inversion method on tree according to claim 1, characterized in that, In step 2, the peach fruit vibration finite element model with bionics is established and the simulation vibration response is calculated, which specifically includes: Step 2.1, use a handheld laser three-dimensional scanner to obtain the three-dimensional point cloud of the outer surface of the peach fruit, kernel and kernel, and construct a three-dimensional geometric model with accurate shape structure of the peach fruit through reverse modeling; Step 2.2, use a texture analyzer and a universal mechanical testing machine to test the compression elastic modulus of peach flesh, kernel and kernel, and the tensile elastic modulus of the peel as material properties; Step 2.3, according to the three-dimensional geometric model with accurate shape structure of the peach fruit and the material properties measured in step 2.2, a peach fruit vibration finite element model with bionics is established, and the simulation vibration response of the response point is simulated and calculated under the experimental excitation and fruit constraint conditions.

4. The finite element simulation and transfer learning based peach fruit firmness inversion method on tree according to claim 3, characterized in that, In step 2.3, the experimental excitation is the gas excitation force given to the peach fruit in the experiment.

5. The finite element simulation and transfer learning based peach fruit firmness inversion method on tree according to claim 3, characterized in that, In step 2.3, the fruit constraint condition is that the peach fruit is constrained by the constraint force at the stem part.

6. The finite element simulation and transfer learning based peach fruit firmness inversion method on tree according to claim 1, characterized in that, In step 4, the domain adversarial neural network is constructed, which specifically includes: Step 4.1, a feature extraction network is constructed based on multi-scale convolution and channel attention mechanism; Step 4.2, building the predictor, the hardness prediction is made by a fully connected layer on the features and the mean squared error, MSE, is chosen as the loss function of the predictor: In the formulae, loss function for the predictor; N p is the number of samples; y p is the hardness value; For predicting hardness value; Step 4.3, construct domain discriminator, make the gradient direction automatically reverse during back propagation by a gradient reversal layer, and take cross-entropy loss as the loss function of the domain discriminator: In the formula, L d is a loss function of the discriminator; N d is the number of samples; y d is the hardness value; For predicting hardness value; Step 4.4, the loss function of the network is composed of the loss function of the predictor and the loss function of the domain discriminator: L = L p - λL d In the formula, L is the total loss function; λ is a parameter for adjusting the intensity of the confrontation.

Images