Farm intelligent reflecting surface deployment method and system based on multiple modes and storage medium

Abstract

The invention discloses a multi-mode-based farm intelligent reflecting surface deployment method and system and a storage medium, and the method comprises the following steps: S1, collecting farm image data, and sensor data and meteorological data set in a farm, and labeling crop type labels; s2, extracting visual feature vectors in the farm graphic data; constructing a farm topological graph according to sensor data set in the farm; constructing a meteorological time sequence vector according to the meteorological data; s3, fusing all vectors obtained in the step S2 to obtain a joint feature vector; s4, adopting reinforcement learning to adjust the joint feature vector in combination with the crop type, and calculating to-be-selected deployment coordinates of the intelligent reflecting surface according to the adjusted joint feature vector; s5, to-be-selected deployment coordinates with signal gains meeting requirements are screened, and phase parameters of the intelligent emitting surface are calculated according to the selected deployment coordinates and the adjusted joint feature vectors; the device can comprehensively consider the factors of farm terrain, crop types and climatic factors to improve the signal transmission quality.

Description

Technical Field

[0001] The present invention relates to the deployment of intelligent reflectors on farms, and in particular to a multi-modal based method, system and storage medium for deploying intelligent reflectors on farms. Background Art

[0002] There are various devices in the agricultural Internet of Things for monitoring the growth status of farm crops, such as sensors and cameras. The signals collected by these devices need to be effectively transmitted to the terminal to achieve effective monitoring. Intelligent reflectors can be deployed in the farmland for signal transmission. However, in the deployment of traditional farm communication networks, the existing deployment schemes of intelligent reflectors do not comprehensively consider the terrain of the farmland, the influence of different crop types on signal attenuation, and the influence of factors such as climate on signal transmission, resulting in unstable signal transmission quality. Summary of the Invention

[0003] Object of the Invention: The object of the present invention is to provide a multi-modal based method, system and storage medium for deploying intelligent reflectors on farms that can comprehensively consider the terrain of the farmland, crop types and climate factors to improve the signal transmission quality.

[0004] Technical Solution: The multi-modal based method for deploying intelligent reflectors on farms according to the present invention includes the following steps:

[0005] S1. Collect farm image data, sensor data set in the farm and meteorological data, and label type tags for the crops in the farm image data;

[0006] S2. Extract visual feature vectors from the farm graphic data; construct a topological embedding vector of the farm according to the sensor data set in the farm; construct a meteorological time series vector according to the meteorological data;

[0007] S3. Fuse the visual feature vectors, meteorological time series vectors and topological embedding vectors to obtain a joint feature vector;

[0008] S4. Adjust the joint feature vector by using reinforcement learning in combination with the crop type tags, and calculate the candidate deployment coordinates of the intelligent reflector in the farm according to the adjusted joint feature vector;

[0009] S5. Screen the candidate deployment coordinates whose signal gain meets the requirements, and calculate the phase parameters of the intelligent transmitter according to the selected deployment coordinates and the adjusted joint feature vector.

[0010] Based on the above technical solution, by collecting farm image data, soil conductivity sensor data, and meteorological data, the visual feature vector of the farm can be extracted to reflect the terrain and obstacle information of the farm. A farm topology map is constructed to reflect the environmental conditions such as soil conductivity, temperature, and humidity of the farm, and a meteorological time series feature is constructed to reflect the meteorological environment of the farm. Then, these three vectors are fused to obtain a joint feature vector, so that the joint feature vector can reflect the above three types of information of the farm at the same time; when collecting data, the type labels of the crops in the farm are marked, and the joint feature vector is optimized and adjusted through reinforcement learning in combination with the crop type, so as to make some features in the joint feature vector more prominent under the set goal. For example, under the set goal, it is necessary to make the terrain feature more prominent, so that this part of the feature will be considered more in the subsequent calculation of the deployment coordinates and phase parameters of the intelligent reflecting surface. In this way, multiple factors of the farm can be comprehensively considered through the joint feature vector, avoiding defects in the layout of the intelligent reflecting surface caused by ignoring some factors and affecting signal transmission; through the optimization and adjustment of the joint feature vector by reinforcement learning, some features that may be ignored can be highlighted, further improving the accuracy of the layout of the intelligent reflecting surface. And the more accurate setting of the position and phase parameters of the intelligent transmitting surface actually means that more interference factors are considered more accurately during the layout, so that the layout position and phase of the intelligent reflecting surface can better avoid these influencing factors and avoid attenuation during signal transmission, thereby making the signal transmission quality better and more stable.

[0011] The multi-modal based farm intelligent reflecting surface deployment system described in the present invention includes:

[0012] Data acquisition module: used to collect farm image data, sensor data set in the farm, and meteorological data, and label the type labels of the crops in the farm image data;

[0013] Feature construction module: used to extract the visual feature vector from the farm graphic data; construct a farm topology map according to the sensor data set in the farm; construct a meteorological time series vector according to the meteorological data;

[0014] Feature fusion module: used to fuse the visual feature vector, meteorological time series vector, and topological vector in the farm topology map to obtain a joint feature vector;

[0015] Coordinate calculation module: used to adjust the joint feature vector by reinforcement learning, and calculate the candidate deployment coordinates of the intelligent reflecting surface in the farm according to the adjusted joint feature vector;

[0016] Coordinate screening and phase parameter calculation module: used to screen the candidate deployment coordinates whose signal gain meets the requirements, and calculate the phase parameters of the intelligent transmitting surface according to the selected deployment coordinates and the adjusted joint feature vector.

[0017] A computer-readable storage medium storing one or more programs, the one or more programs including instructions which, when executed by a computing device, cause the computing device to execute any of the above methods.

[0018] Advantages: Compared with the prior art, the remarkable effect of the present invention is that by extracting or constructing the visual feature vector, topological embedding vector and meteorological time series vector of the farm from the collected data, and then fusing these vectors to obtain a joint feature vector, and combining the crop type to optimize and adjust the joint feature vector through reinforcement learning, the deployment coordinates and phase parameters of the intelligent reflecting surface are obtained from the adjusted joint feature vector. Considering multiple factors comprehensively and avoiding some factors being ignored, the intelligent reflecting surface can be laid out more accurately, reducing the attenuation caused by multiple factors during the signal transmission process and improving the signal transmission quality. Description of the Drawings

[0019] Figure 1 is a schematic flowchart of this application;

[0020] Figure 2 is the input farm image data;

[0021] Figure 3 is the output deployment coordinate heat map; Detailed Embodiments

[0022] As shown in the figure, the multi-modal based farm intelligent reflecting surface deployment method of the present invention includes the following steps:

[0023] S1. Collect the farm image data, the sensor data set in the farm and the meteorological data, and label the crop type tags for the crops in the farm image data;

[0024] The farm image data can use satellite remote sensing images, and images with a resolution of not less than 0.5 meters can be screened to better obtain the terrain and obstacle information of the farm; the sensors set in the farm can include soil conductivity sensors, temperature and humidity sensors, etc., and the meteorological data can collect the meteorological data of all farm locations within a certain time interval. Different crop types, such as rice, wheat, fruit trees, correspond to different predefined signal attenuation coefficients; the signal attenuation coefficients of different crops affect the priority and optimization process of reflecting surface deployment; different crop types also have different effects on the water saving rate of unit signal gain, so it is necessary to label the crop type tags.

[0025] After the data is collected, preprocessing can also be performed, such as:

[0026] Image calibration processing: Align the geographic coordinate system through affine transformation (WGS84 to UTM);

[0027] Data normalization: Map the sensor data to the interval [0, 1], and the formula is:

[0028]

[0029] where x norm is the normalized sensor data, μ is the mean of the sensor data, and σ is the standard deviation of the sensor data.

[0030] S2. Extract the visual feature vector from the farm graphic data; construct the farm topology graph based on the sensor data set in the farm; construct the meteorological time series vector based on the meteorological data;

[0031] Use Vision Transformer to extract the visual features of the farm image data. For example, for an input image of 512×512 pixels with a patch size of 16×16, a 256-dimensional feature vector is output.

[0032]

[0033] V is the 256-dimensional visual feature vector, ViT is the applied Vision transformer model, and I is the input image.

[0034] Generation process of the visual feature vector: Input image patching (16×16 pixels) → Linear projection into an embedding vector → Transformer encoder processing → Global average pooling to generate V. The input image size is 512×512 pixels and the patch size is 16×16.

[0035] Use the graph attention network to construct the farm topology graph. The nodes represent the sensor positions, and the edge weights are the correlation coefficients (0 - 1) of the corresponding sensor parameters; output a 128-dimensional topology embedding vector, and the calculation formula of the topology embedding vector is

[0036]

[0037] G is the 128-dimensional topology embedding vector used to represent the soil parameter association pattern between nodes, and GAT(·) is the topology vector generation function.

[0038] e ij is the sensor correlation coefficient between the i-th and j-th nodes in the farm, and the calculation formula is

[0039]

[0040] where EC i and EC j are the sensor parameters of the i-th and j-th nodes in the farm respectively.

[0041] Encode 24-hour meteorological data into a 24-dimensional meteorological time series vector through temporal convolution processing.

[0042] S3. Combine the visual feature vector, the meteorological time series vector, and the topological vector in the farm topology map to obtain a joint feature vector; calculate the joint feature vector through the following formula

[0043] F joint = Align(V, G, T)

[0044] F joint is the joint feature vector, V is the visual feature vector, G is the topological embedding vector, and T is the meteorological time series vector.

[0045] S4. Use reinforcement learning to adjust the joint feature vector in combination with the crop type label, and calculate the candidate deployment coordinates of the intelligent reflecting surface in the farm according to the adjusted joint feature vector;

[0046] The reinforcement learning has a reward function R for balancing signal gain and energy consumption. The calculation formula of R is

[0047] R = α·RSSI gain + β·Water saved - γ·||H pred - H phys '||2

[0048] where R represents the immediate reward value of the reinforcement learning, RSSI gain is the received signal strength, Water saved is the irrigation water saving amount corresponding to the unit signal gain, H pred is the channel matrix predicted by the neural network, H phys ' is the electromagnetic equation simulation value; α is the signal strength weight coefficient, β is the water saving benefit weight coefficient, and γ is the physical consistency penalty coefficient, satisfying α + β + γ = 1;

[0049] where β·Water saved The specific calculation formula is

[0050]

[0051] where βc is the water saving weight of crop type c; ΔWc is the irrigation water saving amount of crop type c; C is the total number of all crop types in the farm; ΔRSSI is the change in the received signal strength.

[0052] The calculation formula of Hphys is

[0053]

[0054] Among them, f is the signal operating frequency, d is the straight-line distance between the signal transmitting end (base station) and the receiving end (sensor); α c is the attenuation coefficient of crop type c; h c is the height of crop type c; the fixed value 32.4 is calculated from 20log10(4πx10e9 / c) (c is the speed of light, and the units of frequency and distance are GHz and km respectively); log 10 the 20 before (f) is a fixed coefficient for the path loss to increase with frequency and distance, log 10 the 20 before (d) is the logarithmic scaling factor derived from the inverse square law of electromagnetic wave propagation.

[0055] The objective function of reinforcement learning is

[0056]

[0057] Among them, N t is the number of reflecting surfaces enabled in the t-th time period, C t is the signal coverage rate, T is the time interval for data collection; λ1 and λ2 are set constants, satisfying λ1 + λ2 = 1, and in this embodiment, λ1 and λ2 take values of 0.6 and 0.4 respectively

[0058] The physical constraint of reinforcement learning is

[0059]

[0060] In the formula, P j is the power consumption of the j-th intelligent reflecting surface, M is the total number of intelligent reflecting surfaces, P max is the upper limit of the total power consumption of the system.

[0061] Reinforcement learning optimizes and adjusts the joint feature vector based on the above settings to highlight the features that meet the requirements.

[0062] Perform a deconvolution operation on the adjusted joint feature vector to output a heat map with the same resolution as the input image, reflecting the deployment probability of intelligent reflecting surfaces at each location in the farm. A probability threshold can be set, and the position coordinates not less than the probability threshold are used as candidate deployment coordinates.

[0063] S5. Screen the candidate deployment coordinates whose signal gain meets the requirements, and calculate the phase parameters of the intelligent transmitting surface according to the selected deployment coordinates and the adjusted joint feature vector. Specifically, it includes the following sub-steps:

[0064] S51. Screen the candidate deployment coordinates whose signal gain meets the requirements; calculate the signal gain of each candidate deployment coordinate in step S4, and retain the candidate deployment coordinates with signal gain > 3dB.

[0065] S52. Extract the local feature vector F corresponding to the selected deployment coordinates from the adjusted joint feature vector. local ;

[0066] S53. Input the local feature vector F local into the phase decoder network to obtain the phase parameters of the intelligent reflecting surface deployed at this position;

[0067] Specifically, calculate the deployment phase parameters according to the following formula:

[0068] φ init = Sigmoid(W2·ReLU(W1·F local + b1)+b2)×360°

[0069] φ init is the deployment phase parameter, W1 is the first-layer fully connected weight of the phase decoder network, and b1 is the first-layer bias of the phase decoder network; W2 is the second-layer fully connected weight of the phase decoder network, and b2 is the second-layer bias of the phase decoder network.

[0070] The training method of the phase decoder network includes:

[0071] (a) Virtual data generation: Generate virtual farm scenes containing terrain parameters, crop parameters, and meteorological conditions through electromagnetic simulation software, and calculate the optimal phase parameters for signal coverage ≥ 85% in each scenario:

[0072]

[0073] φ ideal is the optimal phase parameter calculated by electromagnetic simulation.

[0074] (b) Network construction: Adopt an architecture with an input layer (408 dimensions), a fully connected layer (256 nodes, ReLU activation), and an output layer (64 nodes, Sigmoid activation), and train through the following periodic loss function:

[0075]

[0076] where L is the periodic loss function; is the i-th phase value predicted by the phase decoder network; is the i-th ideal phase value calculated by electromagnetic simulation.

[0077] (c) Pre-training stage: Use the Adam optimizer (learning rate 5e-4) to train the network for 100 rounds, and inject Gaussian noise σ (σ = 0.05) and the characteristics of the shielding crop type to enhance generalization;

[0078] (d) Meta - learning adaptation: Freeze the parameters of the visual and topological modules, and fine - tune the second fully - connected layer of the decoder (learning rate 1e - 5) based on 3 - 5 sets of scenario data from the new farm until the validation loss converges, that is, ΔLoss < 0.01, where ΔLoss is the change in the validation loss, namely the absolute value of the change in L. When ΔLoss < 0.01, it can be considered that the model converges and the training is completed.

[0079] After deploying the intelligent transmitting surface, the blind beamforming technology can also be used to calculate and adjust the phase parameters of the intelligent reflecting surface in real - time according to the received signal strength to better adapt to the real - time changes in the farm environment. Among them, the phase control accuracy of the blind beamforming technology is 6 - bit, and it supports 360° omnidirectional coverage.

[0080] The specific calculation formula is

[0081] φ fin =φ new ·mod 360°

[0082] φ fin is the final phase parameter, and φ new is the preliminary phase parameter calculated according to the following formula

[0083]

[0084] Among them, η is the gradient ascent learning rate, is the phase gradient, and the calculation formula is

[0085]

[0086] Among them, RSSI(φ) and RSSI(φ + Δφ) respectively represent the signal strength values measured at the receiving end when the current phase parameters are Φ and Φ+ΔΦ, and Δφ is the set phase change amount.

[0087] After the deployment is completed, the deployment effect of the intelligent reflecting surface can be verified by combining virtual verification and on - site verification. The on - site verification can be tested by an unmanned aerial vehicle to ensure that the error of the received signal strength indication (RSSI) is controlled within 2 dB to verify the actual effect of the system.

[0088] Within historical time, the farm can complete the pre-training of multiple multi-modal learning according to the above method to complete the deployment of the intelligent reflecting surface. It can adjust the corresponding parameters of the model based on the previously pre-trained multi-modal model through meta-learning to quickly adapt to the new farm and complete the deployment of the intelligent reflecting surface in the new farm. Of course, if there is no pre-trained multi-modal learning model completed in history, pre-training can also be carried out through a virtual scenario. After training, the satellite image data of the new farm is input, the parameters of VisionTransformer and GAT are frozen, and only the fully connected adaptation layer (learning rate 5e-5) is fine-tuned to complete the rapid adjustment of the model to output the deployment coordinates and phase parameters of the intelligent reflecting surface.

[0089] The multi-modal-based intelligent reflecting surface deployment system for a farm according to the present invention, the system includes:

[0090] Data acquisition module: used to collect farm image data, sensor data and meteorological data set in the farm, and label type tags for crops in the farm image data;

[0091] Feature construction module: used to extract visual feature vectors from farm graphic data; construct a farm topology map according to the sensor data set in the farm; construct a meteorological time series vector according to the meteorological data;

[0092] Feature fusion module: used to fuse visual feature vectors, meteorological time series vectors and topological vectors in the farm topology map to obtain a joint feature vector;

[0093] Coordinate calculation module: used to adjust the joint feature vector by reinforcement learning in combination with the crop type, and calculate the candidate deployment coordinates of the intelligent reflecting surface in the farm according to the adjusted joint feature vector;

[0094] Coordinate screening and phase parameter calculation module: used to screen the candidate deployment coordinates whose signal gain meets the requirements, and calculate the phase parameters of the intelligent transmitting surface according to the selected deployment coordinates and the adjusted joint feature vector.

[0095] The computer-readable storage medium storing one or more programs according to the present invention includes one or more programs including instructions, and when the instructions are executed by a computing device, the computing device executes any of the above methods.

Claims

1. A multi-modal farm intelligent reflective surface deployment method, characterized in that: The following steps are involved: S1, collect farm image data, sensor data and meteorological data set in the farm, and label the crop type in the farm image data; S2, extract visual feature vectors from farm graphic data; construct a topological embedding vector of the farm based on sensor data set in the farm; construct a meteorological time series vector based on meteorological data; S3, fuse the visual feature vector, meteorological time series vector and topological embedding vector to obtain the joint feature vector; S4. Reinforcement learning is used to adjust the joint feature vector in combination with the crop type label, and the candidate deployment coordinates of the smart reflective surface on the farm are calculated according to the adjusted joint feature vector; S5. Screen the candidate deployment coordinates whose signal gain meets the requirements, and calculate the deployment phase parameters of the smart transmitting surface according to the selected deployment coordinates and the adjusted joint eigenvector.

2. The multi-modal farm intelligent reflective surface deployment method according to claim 1, characterized in that: In the step S2, Vision Transformer is used to extract the visual feature vector of the farm image data; a graph attention network is used to construct the topological embedding vector of the farm; and temporal convolution processing is used to construct the meteorological data into a meteorological time series vector.

3. The multi-modal farm intelligent reflective surface deployment method according to claim 2, characterized in that: The graph attention network generates a topological embedding vector by G=GAT(e ij ) Where G is the topological embedding vector, GAT(·) is the topological vector generating function; ij is the sensor correlation coefficient between the i-th and j-th nodes in the farm, e ij The calculation formula is as follows G = GAT (e ij ) Among them, EC i and EC j are the sensor parameters of the i-th and j-th nodes in the farm, respectively.

4. The multi-modal farm intelligent reflective surface deployment method according to claim 1, characterized in that: The step S3 calculates the joint feature vector by the following formula F joint =Align(V,G,T) F joint is the joint feature vector, V is the visual feature vector, G is the topological embedding vector, and T is the meteorological time series vector.

5. The multi-modal farm intelligent reflective surface deployment method according to claim 1, characterized in that: In step S4, the reinforcement learning is provided with a reward function R calculation formula: R=α·RSSI gain +b·Water saved -γ·||H pred -H phys ′||2 Among them, R represents the immediate reward value of reinforcement learning, RSSI gain To receive signal strength, Water saved is the irrigation water saving corresponding to unit signal gain, H pred is the channel matrix predicted by the neural network, H phys ' is the simulation value of the electromagnetic equation; α is the signal strength weight coefficient, β is the water-saving benefit weight coefficient, and γ is the physical consistency penalty coefficient, satisfying α+β+γ=1; Where β·Water saved The specific calculation formula is: Among them, βc is the water-saving weight of type c crop; ΔWc is the irrigation water saving of type c crop; C is the total number of all crop types in the farm; ΔRSSI is the change in received signal strength. The objective function of reinforcement learning is Among them, N t The number of reflective surfaces enabled in time period t, C t is the signal coverage, T is the time interval of data collection; λ1 and λ2 are set constants, satisfying λ1+λ2=1; The physical constraints for reinforcement learning are Where, is P j The power consumption of the jth smart reflector, M is the total number of smart reflectors, P max The total power consumption limit of the system.

6. The multi-modal farm intelligent reflective surface deployment method according to claim 5, characterized in that: The H phys The calculation formula of (d)' is Among them, f is the signal operating frequency, d is the straight-line distance between the signal transmitter and the receiver; αc is the attenuation coefficient of type c crops; hc is the height of type c crops.

7. The multi-modal farm intelligent reflective surface deployment method according to claim 1, characterized in that: The step S4 performs deconvolution on the adjusted joint feature vector to output a heat map of the coordinates to be deployed; the step S5 includes the following sub-steps: S51, screening candidate deployment coordinates whose signal gain meets the requirements; S52, extracting the local feature vector F corresponding to the selected deployment coordinates from the adjusted joint feature vector local ; S53, the local feature vector F local Input into the phase decoder network to obtain the phase parameters of the smart reflector deployed at the location; The deployment phase parameters are calculated according to the following formula: φ init =Sigmoid(W2·ReLU(W1·F local +b1)+b2)×360° φ init is the deployment phase parameter, W1 is the first layer fully connected weight of the phase decoder network, b1 is the first layer bias of the phase decoder network; W2 is the second layer fully connected weight of the phase decoder network, and b2 is the second layer bias of the phase decoder network.

8. The multi-modal farm intelligent reflective surface deployment method according to claim 1, characterized in that: After the deployment limit parameters are obtained in step S5, the blind beamforming technology is used to calculate and adjust the deployment phase parameters in real time according to the received signal strength. The specific calculation formula is φ fin =φ new · Mod 360° φ fin is the final phase parameter, φ new The preliminary phase parameters are calculated according to the following formula Where η is the gradient ascent learning rate, is the phase gradient, and the calculation formula is Among them, RSSI(φ) and RSSI(φ+Δφ) respectively represent the signal strength values ​​measured by the receiving end when the current phase parameters are Φ and Φ+ΔΦ, and Δφ is the set phase change.

9. A multi-modal farm intelligent reflective surface deployment system, characterized in that: The system comprises: Data collection module: used to collect farm image data, sensor data and meteorological data set in the farm, and label the crop type in the farm image data; Feature construction module: used to extract visual feature vectors from farm graphic data; construct the farm's topological embedding vector based on the sensor data set in the farm; and construct the meteorological time series vector based on meteorological data; Feature fusion module: used to fuse visual feature vectors, meteorological time series vectors and topological embedding vectors to obtain joint feature vectors; Coordinate calculation module: used to adjust the joint feature vector by using reinforcement learning in combination with the crop type label, and calculate the candidate deployment coordinates of the intelligent reflective surface on the farm according to the adjusted joint feature vector; Coordinate screening and phase parameter calculation module: used to screen candidate deployment coordinates whose signal gain meets the requirements, and calculate the phase parameters of the smart transmitting surface based on the selected deployment coordinates and the adjusted joint eigenvector.

10. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions which, when executed by a computing device, cause the computing device to perform any one of the methods according to claims 1 to 8.

Images