Tomato planting decision-making method and computer device based on double-branch network

Abstract

The present invention relates to the field of tomato planting decision-making, and specifically to a tomato planting decision-making method and computer device based on a dual-branch network. The scheme includes: multimodal data collection and fusion; enhancing the fused data through a self-attention generative adversarial network; feature extraction through a Transformer-CNN dual-branch network; and reinforcement learning and decision-making through DDPG. When processing complex structure and texture images, the present invention can better grasp the image region relationship and generate more accurate natural data through the self-attention layer, make full use of the self-attention mechanism to capture global semantic information and long-distance dependencies, and CNN extracts local features and spatial information to achieve effective fusion and feature extraction of multimodal data. Then, the DDPG algorithm is used to solve the continuous action space problem. Combined with the deterministic strategy and the experience playback mechanism, the accuracy of tomato planting decisions is greatly improved. The present invention is suitable for tomato planting decisions.

Description

Technical Field

[0001] The present invention relates to the field of tomato planting decision-making, and in particular to a tomato planting decision-making method based on a double-branch network and a computer device. Background Art

[0002] Tomato planting decision is a crucial part of agricultural production, which is directly related to the yield, quality and economic benefits of agricultural tomatoes. With the development of science and technology, especially the integration of information technology and agricultural science, tomato planting decision has changed from traditional empirical judgment to scientific decision based on data-driven. However, although modern technology provides strong support for tomato planting decision, there are still some challenges and shortcomings.

[0003] At present, tomato planting decisions involve knowledge and technology from multiple fields, including but not limited to soil science, plant physiology, meteorology, remote sensing technology, and geographic information systems. Knowledge in these fields provides a scientific basis for tomato planting decisions, but still faces some problems in practical applications. For example, the collection and analysis of soil data often requires a lot of time and resources, and the complexity of soil conditions makes it difficult to accurately assess soil fertility and pH. In addition, there is uncertainty in the prediction of meteorological data. The above-mentioned multiple reasons may lead to misjudgment in choosing the planting time and tomato varieties. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a tomato planting decision method and a computer device based on a double-branch network, thereby improving the accuracy of tomato planting decisions.

[0005] The present invention adopts the following technical solutions to achieve the above-mentioned purpose. In a first aspect, the present invention provides a tomato planting decision method based on a double-branch network, comprising:

[0006] S1, multimodal data acquisition;

[0007] S2, multimodal data fusion, the multimodal data including image, meteorological and soil data;

[0008] S3, enhance the fusion data through self-attention generative adversarial network;

[0009] S4, feature extraction through Transformer-CNN dual-branch network;

[0010] The input sequence obtained by fusing digital images, meteorological and soil data through the Transformer branch passes through the encoder of the Transformer branch, and uses the self-attention mechanism and multi-head attention to capture global semantic information and long-distance dependencies to obtain an output vector containing global features;

[0011] The image, meteorological and soil data are fused in different ways as input through the CNN branch, and local features and spatial information are extracted through multiple convolutional layers. The dimensions are then reduced through the pooling layer, and finally an output vector containing local features and spatial information is obtained;

[0012] The output vectors of the two branches are fused by concatenation, weighted fusion or attention fusion, and then processed by the fully connected layer, adding Dropout and regularization terms to obtain the final comprehensive feature vector for subsequent tasks;

[0013] S5. Reinforcement learning and decision making through DDPG (Deep Deterministic Policy Gradient)

[0014] When the comprehensive feature vector is input, in DDPG, the Actor and Critic network parameters and the target network are randomly set in the initialization phase, and a playback buffer is created. In the interaction phase with the environment, actions are generated through the Actor network based on the input vector and applied to the environment. The environment feeds back the next state, reward, and end flag, and stores the current interaction experience in the playback buffer.

[0015] In the training phase, samples are extracted from the buffer, the Critic network loss function is calculated and the parameters are updated by stochastic gradient descent, the Actor network policy gradient is calculated and the parameters are updated, and the target network parameters are slowly updated;

[0016] In the decision-making stage, during testing, the Actor network generates actions based on the input vector and applies them to the environment. This process is repeated until the task is completed, thereby achieving decision-making based on multimodal data feature input.

[0017] Furthermore, step S1 specifically includes:

[0018] Get a variety of meteorological data, including maximum temperature, minimum temperature, average temperature, relative humidity, wind speed, precipitation, air pressure and dew point temperature;

[0019] Collect soil, analyze the soil, and obtain data on the content of various inorganic salts in the soil;

[0020] Measure the conductivity and pH of the soil, obtain the data of the conductivity and pH of the soil, and process the data of the inorganic salt content, the conductivity and pH of the soil through ArcGIS software and interpolation technology to obtain the soil analysis result image;

[0021] The soil moisture content data is obtained through a capacitive soil moisture sensor, and the resistivity of the soil is measured to obtain the resistivity of the soil;

[0022] A bird's-eye view image of a tomato field is captured as image data.

[0023] Furthermore, step S2 specifically includes:

[0024] Arrange various meteorological data into a meteorological data array, and arrange soil-related data into a soil data array;

[0025] Normalize various meteorological data and soil-related data;

[0026] Create a data matrix (n, height, width, 2), where height and width are the height and width of the image, and 2 represents two channels, one for meteorological data and the other for soil-related data.

[0027] Then it is merged with the 3-channel RGB image to become a fusion matrix of (n, height, width, 5).

[0028] In a second aspect, the present invention provides a computer device, including a memory, wherein the memory stores program instructions, and when the program instructions are executed, the above-mentioned tomato planting decision method based on the double-branch network is executed.

[0029] The beneficial effects of the present invention are:

[0030] The data of the present invention is detailed and covers a variety of key indicators, providing a rich meteorological information basis for subsequent analysis.

[0031] After on-site sampling, the present invention analyzes the inorganic salt content, electrical conductivity, pH value, etc. in the laboratory. At the same time, it uses advanced sensors to monitor soil moisture content and resistivity, collects data related to grain and rice planting, and comprehensively reflects soil characteristics and agricultural production conditions.

[0032] The present invention processes soil data using interpolation technology through ArcGIS software to make the data more complete and accurate, and presents soil analysis diagrams in a visual form, providing an intuitive basis for decision-making.

[0033] The present invention introduces a self-attention generative adversarial network in data enhancement. Especially when processing complex structure and texture images, the self-attention layer can better grasp the relationship between image regions and generate more accurate natural data.

[0034] The present invention makes full use of the self-attention mechanism of Transformer to capture global semantic information and long-distance dependencies, and CNN to extract local features and spatial information, to achieve effective fusion and feature extraction of multimodal data, and then uses the DDPG deep reinforcement learning algorithm to solve the continuous action space problem, combined with deterministic strategies and experience replay mechanism, to greatly improve the accuracy of planting decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of a tomato planting decision method based on a double-branch network provided by an embodiment of the present invention;

[0036] Figure 2 It is a DDPG structure diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0038] The present invention provides a tomato planting decision method based on a double-branch network, such as Figure 1 As shown, the following steps are included:

[0039] S1, multimodal data acquisition;

[0040] The meteorological data is taken from the authoritative China Meteorological Data Network. On the China Meteorological Data Network, it is very convenient to directly find the detailed daily meteorological data of the target area. In order to better meet the needs of subsequent in-depth analysis and research, these data cover a wide range, including the maximum temperature of air and soil, which reflects the peak value of air temperature or soil temperature in a specific time period; the minimum temperature, that is, the lower limit of temperature in the corresponding time period; and the average temperature. The present invention comprehensively considers the temperature change over a period of time and provides a relatively stable reference value for the overall temperature condition. In addition, it also includes the specific value of relative humidity, which is crucial for understanding the humidity of the air and the impact on various environmental factors. The size of the wind speed is also included in these data. The change of wind speed has an important impact on agricultural production, meteorological forecasting and people's daily life. The amount of precipitation records the precipitation in the target area within a specific time, which is of great significance to water resources management, agricultural irrigation and other aspects. The level of air pressure reflects the pressure of the atmosphere, which has a key impact on weather changes and aviation. The dew point temperature can help people understand the saturation of water vapor in the air and possible weather phenomena.

[0041] The soil is carefully sampled on site, and then the content of each inorganic salt is comprehensively and detailedly analyzed in a professional laboratory environment. These inorganic salts cover the content of nitrogen (N), phosphorus (P), potassium (K), magnesium (Mg), sodium (Na) and other elements. These elements play a vital role in soil fertility and plant growth and development. At the same time, the electrical conductivity (EC) and pH (PH) of the soil are accurately measured. Electrical conductivity can reflect the salt content and ion activity in the soil, while pH directly affects the effectiveness of nutrients in the soil and the activity of microorganisms. Through the powerful ArcGIS software, these soil data are processed meticulously using advanced interpolation technology. Interpolation technology can infer the data value of unknown areas based on known data points, making the soil data more complete and accurate. Then the soil analysis results are presented in a vivid and intuitive visual form, providing people with clear, intuitive and easy-to-understand soil analysis maps. These soil analysis maps can help agricultural producers, land managers and scientific researchers better understand the characteristics and distribution of soils, and provide a scientific basis for formulating reasonable land use and agricultural production strategies.

[0042] In order to obtain soil moisture data, the present invention uses several advanced capacitive soil moisture sensors. These sensors are carefully selected and arranged, and are cleverly distributed in the soil at different depths to comprehensively and accurately monitor the moisture content of different layers of the soil. In this way, the distribution and changes of soil moisture in the vertical direction can be better understood, providing strong data support for precision irrigation and water resource management. The sensor and the computer transmit data through a stable and reliable connection method to ensure the timeliness and accuracy of data transmission. The computer can efficiently receive data from the sensor and store it safely and reliably. The stored data can be called and analyzed at any time, providing valuable historical data for long-term soil moisture monitoring and research.

[0043] Regarding the determination of soil resistivity, the present invention uses a portable ARP instrument for collection. The instrument is exquisitely designed, easy to operate, and can perform fast and accurate measurements under different site conditions. It accurately calculates the conductivity of the soil based on a scientific and reasonable calculation formula by accurately measuring the voltage difference generated by the front wheel and the three rear wheels at different depths. This measurement method can effectively reflect the conductivity of the soil and provide an important reference for understanding the physical properties and fertility of the soil.

[0044] At the same time, the present invention also collects a large amount of observation data, which are rich and varied, including the water content of grains, the flow rate of grains, wet yield and dry yield, and the specific coordinates corresponding thereto. After these data are comprehensively sorted and analyzed, they are plotted together, thereby successfully plotting a seasonal ground yield map that can reflect the characteristics of different seasons. The seasonal ground yield map can intuitively show the yield changes of agricultural tomatoes in different seasons, providing an important reference basis for agricultural production planning and decision-making. In addition, the power of each machine and its fuel consumption data during the rice planting process are specially collected. Through detailed statistics and analysis of these data, the energy consumption during the rice planting process is accurately obtained. Understanding the energy consumption during the rice planting process is of great significance for optimizing the agricultural production process, reducing production costs, and improving the sustainability of agricultural production.

[0045] The present invention takes a bird's-eye view image of a tomato field as image data.

[0046] S2, multimodal data fusion;

[0047] Arrange various meteorological index data into a meteorological data array, and arrange soil-related data into a soil data array;

[0048] Normalize various meteorological index data and soil-related data so that different types of data have similar numerical ranges to avoid some data having too much impact on subsequent processing. Use common normalization methods, such as minimum-maximum normalization, to map the data to the [0, 1] interval.

[0049] Create a data matrix (n, height, width, 2), where height and width are the height and width of the image, and 2 represents two channels, one for meteorological data and the other for soil-related data. Because the overhead view of the tomato field is a 3-channel RGB image, the data matrix is ​​merged with the image to become a fusion matrix of (n, height, width, 5).

[0050] S3, fusion data enhancement;

[0051] The generator of the self-attention generative adversarial network is mainly composed of multiple convolutional layers and upsampling layers. Its core function is to gradually transform random noise into images. In this gradual transformation process, each layer of the generator extracts and processes the input data to gradually construct an image with rich details and semantic information.

[0052] The self-attention layer is introduced in the middle layer of the generator, which is of vital importance. By introducing the self-attention layer, the generator can have a deeper understanding of the intrinsic connection and relationship between different regions in the image. This high attention to regional relationships enables the generator to more accurately grasp the overall structure and local details of the image when generating images, thus helping to generate clearer, more detailed, and high-quality images.

[0053] Especially when facing images with complex structures and textures, the advantages of the self-attention layer are more obvious. It enables the generator to better understand the complex relationships between the various parts of the image, including the relationship between shape, color, texture, etc. By accurately grasping these relationships, the generator can generate more accurate and natural images, both in terms of the overall composition of the image and the local details, to a higher level.

[0054] For example, when generating natural images with complex textures, the self-attention layer can help the generator better handle texture changes and transitions, making the generated image texture more delicate and realistic. When generating images with multiple objects and complex scenes, the self-attention layer can better coordinate the relationship between different objects, making the image composition more reasonable and natural.

[0055] Generator working process, initial layer processing The generator first receives this 5-channel matrix input. The initial layer performs preliminary feature extraction and transformation on the input data, mapping the input data to a high-dimensional latent space. This process may include the application of some linear transformations and nonlinear activation functions to enhance the expressiveness of the data. Intermediate layer processing As the network goes deeper, the intermediate layer gradually processes and refines the data through a series of convolutional layers, deconvolutional layers, and self-attention layers. The convolutional layer can extract local features, while the deconvolutional layer gradually increases the size of the image and restores the details of the image. The self-attention layer can capture the long-distance dependencies between different positions in the image, making the generated image more coherent and realistic. Discriminator working process, input and feature extraction The discriminator receives both fake images and real image data from the generator (if in the training phase). It first extracts features from the input image, and can also use operations such as convolutional layers to extract various features of the image. For a 5-channel input image, the discriminator analyzes the features of each channel and their relationship with each other. Intermediate layer judgment The intermediate layer further processes these features to judge the authenticity of the image. The discriminator learns to distinguish between real images and fake images generated by the generator, and improves the accuracy of judgment by continuously adjusting its own parameters. In this process, the discriminator may use some advanced feature representation methods, such as global average pooling, fully connected layers, etc., to compress the features of the image into a single judgment value, indicating the probability that the image is real. Output decision Finally, the discriminator outputs a judgment result, that is, the probability that the input image is a real image. If the input is a fake image from the generator, the discriminator will try to give a lower probability value; if it is a real image, it will give a higher probability value. Training and optimization process, adversarial training SAGAN continuously improves performance through adversarial training between the generator and the discriminator. The goal of the generator is to generate fake images that are as realistic as possible to deceive the discriminator; while the goal of the discriminator is to accurately distinguish between real images and fake images. During the training process, the parameters of the generator and the discriminator are continuously updated, and they compete with each other until they reach a state of equilibrium, at which point the generator can generate very realistic images, while the discriminator has difficulty distinguishing between real images and fake images. This generates more data and achieves the purpose of data enhancement.

[0056] S4, Transformer-CNN dual-branch network feature extraction;

[0057] The Transformer-CNN dual-branch network consists of two branches, the Transformer branch and the CNN branch. The Transformer branch consists of an encoder and a decoder, and its core is the self-attention mechanism, which allows the model to dynamically consider the importance of all other elements in the input sequence when processing each element and assign different weights to them. In this way, the model can capture long-distance dependencies, that is, no matter how far apart two elements are in the sequence, as long as they are semantically related, the model can give them corresponding attention. For images and meteorological and soil data in the new array, the self-attention mechanism can help the model understand the relationship between different image regions and the association between images and meteorological and soil data. The convolution layer is the core component of CNN. It performs convolution operations on the input data by using a set of learnable filters (also called convolution kernels). For image data in the new array, the convolution layer can extract local features of the image, such as edges, textures, color changes, etc. Different convolution kernels can detect different feature patterns. As the network layer deepens, the convolution layer can extract more and more complex and abstract features. For meteorological and soil data channels, they can be used as input together with image data, or fused in a specific convolutional layer so that the convolutional layer can learn the features of image and non-image data at the same time.

[0058] In the Transformer-CNN dual-branch network, the Transformer branch passes the input sequence after the image is fused with the meteorological and soil data through the encoder, and uses the self-attention mechanism and multi-head attention to capture the global semantic information and long-distance dependencies to obtain an output vector containing global features. The CNN branch uses the image as input after fusing with the meteorological and soil data in different ways, extracts local features and spatial information through multiple convolutional layers, and then reduces the dimension through the pooling layer to finally obtain an output vector containing local features and spatial information. Then, the output feature vectors of the two branches are fused through splicing, weighted fusion or attention fusion, and then processed through the fully connected layer, adding Dropout and regularization terms to prevent overfitting, and finally obtain a comprehensive feature representation for subsequent tasks.

[0059] S5, DDPG reinforcement learning and decision making;

[0060] DDPG is a deep reinforcement learning algorithm for solving continuous action space problems. It combines the ideas of deterministic strategies and experience replay. In the decision-making field, it can make correct predictions about the existing environment. DDPG is based on the Actor-Critic framework. The actor learns the deterministic strategy to output the action value, and the critic learns the value function. DDPG uses deterministic strategies to help the continuous action space learn strategies. It introduces the experience replay mechanism, uses the target network to estimate the target value and strategy, and the target network parameters are softly updated. The deterministic strategy outputs actions that lack exploration. In the training phase, the Actor network outputs actions that add noise to allow the agent to explore.

[0061] Experience replay is a technique that makes the probability distribution of experience stable, which can improve the stability of training. Experience replay mainly consists of two key steps: "storage" and "replay". Storage is to store experience in the experience pool, and replay is to retrieve one or more samples from the experience pool.

[0062] The update process includes the update process of the Critic network, the update process of the Actor network, the update process of the Target Actor network, and the update process of the Target Critic network. The algorithm update mainly updates the parameters of the Actor and Critic networks, where the Actor network is updated by maximizing the cumulative expected return, and the Critic network is updated by minimizing the error between the evaluation value and the target value.

[0063] The overall update flow chart is as follows Figure 2 shown.

[0064] During input, in DDPG, the initialization phase randomly sets the parameters of the Actor and Critic networks and the target network, and creates a playback buffer. During the interaction phase with the environment, the agent generates actions through the Actor network based on the input state vector and applies them to the environment. The environment feeds back the next state, reward, and end flag, and stores the current interaction experience in the playback buffer. During the training phase, samples are extracted from the buffer, the Critic network loss function is calculated and the parameters are updated through stochastic gradient descent, the Actor network policy gradient is calculated and the parameters are updated, and the target network parameters are slowly updated. During the decision-making phase, during testing, the agent generates actions through the Actor network based on the input state vector and applies them to the environment. This process is repeated until the task is completed, thereby achieving decision-making based on multimodal data feature input.

[0065] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A tomato planting decision-making method based on a double-branch network, characterized in that: include: S1, multimodal data acquisition; Get a variety of meteorological data, including maximum temperature, minimum temperature, average temperature, relative humidity, wind speed, precipitation, air pressure and dew point temperature; Collect soil, analyze the soil, and obtain data on the content of various inorganic salts in the soil; The conductivity and pH of the soil were measured to obtain the data of the conductivity and pH of the soil, and the data of the inorganic salt content, the conductivity and pH of the soil were processed by ArcGIS software and interpolation technology to obtain the soil analysis result image; The soil moisture content data is obtained through a capacitive soil moisture sensor, and the resistivity of the soil is measured to obtain the resistivity of the soil; Taking a bird's-eye view image of a tomato field as image data; S2, multimodal data fusion, the multimodal data including image, meteorological and soil data; Arrange various meteorological data into a meteorological data array, and arrange soil-related data into a soil data array; Normalize various meteorological data and soil-related data; Create a data matrix (n, height, width, 2), where height and width are the height and width of the image, and 2 represents two channels, one for meteorological data and the other for soil-related data. Then merge it with the 3-channel RGB image to become a fusion matrix of (n, height, width, 5); S3, enhance the fusion data through self-attention generative adversarial network; S4, feature extraction through Transformer-CNN dual-branch network; The input sequence obtained by fusing the image, meteorological and soil data through the Transformer branch passes through the encoder of the Transformer branch, and uses the self-attention mechanism and multi-head attention to capture the global semantic information and long-distance dependencies, and obtain an output vector containing global features; The image, meteorological and soil data are fused in different ways as input through the CNN branch, and local features and spatial information are extracted through multiple convolutional layers. The dimensions are then reduced through the pooling layer, and finally an output vector containing local features and spatial information is obtained; The output vectors of the two branches are fused by concatenation, weighted fusion or attention fusion, and then processed by the fully connected layer, adding Dropout and regularization terms to obtain the final comprehensive feature vector for subsequent tasks; S5. Reinforcement learning and decision making through DDPG; When the comprehensive feature vector is input, in DDPG, the Actor and Critic network parameters and the target network are randomly set in the initialization phase, and a playback buffer is created. In the interaction phase with the environment, actions are generated through the Actor network based on the input vector and applied to the environment. The environment feeds back the next state, reward, and end flag, and stores the current interaction experience in the playback buffer. In the training phase, samples are extracted from the buffer, the Critic network loss function is calculated and the parameters are updated by stochastic gradient descent, the Actor network policy gradient is calculated and the parameters are updated, and the target network parameters are slowly updated; In the decision-making stage, during testing, the Actor network generates actions based on the input vector and applies them to the environment. This process is repeated until the task is completed, thereby achieving decision-making based on multimodal data feature input.

2. A computer device comprising a memory, wherein the memory stores program instructions, characterized in that: When the program instructions are executed, the tomato planting decision method based on the double-branch network as claimed in claim 1 is executed.

Images