Graph neural network agricultural remote sensing image semantic segmentation method based on frequency domain
By converting agricultural remote sensing image data into Fourier domain and using Fourier basis for training, traditional graph neural networks are solved for slow speed and memory overflow when processing large-scale data, achieving more efficient and accurate image processing.
Patent Information
- Application Number
- CN202510063385.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-15
- Publication Date
- 2025-06-03
AI Technical Summary
Traditional graph neural networks are slow to process large-scale agricultural remote sensing images, and are prone to memory overflow problems during training.
The graph neural network method based on frequency domain is used to convert the graph data into Fourier domain, trained through Fourier basis, and the data is processed in batches to avoid memory overflow.
It improves the speed of large-scale data processing, improves the efficiency and accuracy of agricultural remote sensing image processing, and avoids memory overflow problems.
Smart Images
Figure BDA0005243541910000021 
Figure BDA0005243541910000022 
Figure BDA0005243541910000027
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision image processing, and in particular to a method for semantic segmentation of agricultural remote sensing images based on a graph neural network in the frequency domain. Background Art
[0002] For traditional agriculture, a large amount of manpower, material resources and financial resources are required to achieve farmland yield estimation and farmland area statistics, and a large amount of time is also required. The development of remote sensing technology has put forward new ideas for solving these problems. By performing semantic segmentation processing on agricultural remote sensing images, information such as farmland or plant cultivation and estimated yields and growth trends can be obtained. Therefore, graph neural networks and semantic segmentation technologies have achieved many results in agricultural applications, and unassociated pixels can be constructed into an associated graph structure.
[0003] However, traditional graph neural networks have a slow processing speed when dealing with large-scale data, and training a graph neural network usually requires saving the entire graph data and the intermediate states of all nodes in memory, and its full-batch processing training algorithm will cause serious memory overflow problems. Summary of the Invention
[0004] In order to solve the deficiencies existing in the prior art, the present application proposes a method for semantic segmentation of agricultural remote sensing images based on a graph neural network in the frequency domain, which combines a graph neural network and semantic segmentation to improve the processing speed of large-scale data, and can effectively improve the efficiency and accuracy of agricultural remote sensing image processing.
[0005] The technical solution adopted by the present invention is as follows:
[0006] A method for semantic segmentation of agricultural remote sensing images based on a graph neural network in the frequency domain, comprising the following steps:
[0007] Step 1, obtaining a remote sensing image, converting the remote sensing image into graph data, and taking the graph data in batches as input
[0008] Step 2, for the graph data input in batches, calculating the eigenvalues of the graph data to obtain an eigenvalue matrix.
[0009] Step 3, based on the eigenvalue matrix obtained in Step 2, sorting the eigenvalues in the eigenvalue matrix from largest to smallest, extracting the subscripts of the top k eigenvalues; then extracting the vectors with the same subscripts from the Fourier matrix, and forming a Fourier basis from the extracted vectors;
[0010] Step 4, construct a loss function, and iteratively optimize the weight parameters of the semantic segmentation model based on the Fourier basis and the loss function; if the iteration termination condition is met, obtain the optimal weight parameters of the semantic segmentation model; if the iteration termination condition is not met, go back to Step 1 to input the next batch of image data for continued training;
[0011] Step 5, based on the obtained optimal weight parameters, use the semantic segmentation model to perform semantic segmentation on agricultural remote sensing images and output the segmentation results.
[0012] Furthermore, the steps to obtain the eigenvalue matrix in Step 2 are as follows:
[0013] Step 2.1, construct the Lagrangian function, denoted as:
[0014]
[0015] Step 2.2, based on the constructed Lagrangian function, obtain all the eigenvalues of the input batch of image data to get the eigenvalue matrix;
[0016] The calculation method of the eigenvalues is as follows:
[0017]
[0018] where F is the Fourier matrix, diag is the diagonal matrix, is the i-th diagonal element of the diagonal matrix, F H is the conjugate transpose matrix of the Fourier matrix, is the conjugate of , ⊙ represents element-wise multiplication, D ii is the element in the i-th row and i-th column of the degree matrix, L ij is the element in the i-th row and j-th column of the Laplacian matrix, and the obtained Laplacian matrix is a symmetric matrix, Λ is the eigenvalue of the image data, is the eigenvector corresponding to the eigenvalue.
[0019] Furthermore, the constructed loss function l' f is denoted as:
[0020]
[0021] where b is the batch number of the input image data, S ij is the affinity matrix, g(z i ; θ) is the mapping function, θ is the parameter set in the proposed network, denoted as θ = {W 1 , b 1 , W 2 , b 2 , W 3 , b 3}, W1 , W 2 , W 3 are the weights of the first, second, and third layers in the network, respectively. b 1 , b 2 , b 3 are the biases of the first, second, and third layers in the network, respectively; d i is the sum of the i-th row of the degree matrix, d j is the sum of the j-th row of the degree matrix, z i is the i-th batch of the input, z j is the j-th batch of the input. t is the temperature parameter used to control the sensitivity of similarity calculation, and ε is the similarity threshold.
[0022] Furthermore, the method for iteratively optimizing the weight parameters of the semantic segmentation model based on the Fourier basis and the loss function in step 4 is as follows: Use the Fourier basis obtained in step 3 as the weight W of the third layer of the network 3 and substitute it into the loss function l' f , and determine whether the loss function l' f converges. If it converges, perform backpropagation iteration update through the following formula to obtain the optimal weight parameters.
[0023] Furthermore, the method for backpropagation iteration update of parameters is denoted as:
[0024] For W 2 , b 2 , the update method is as follows:
[0025]
[0026] For W 1 , b 1 , the update method is as follows:
[0027]
[0028] where, F k is the k-th Fourier basis, F is the Fourier basis, is the activation output of the first layer, is the activation output of the second layer, s 2 is the affinity matrix of the second layer, and g is the activation function.
[0029] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned method for semantic segmentation of agricultural remote sensing images based on a frequency-domain graph neural network.
[0030] A computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the above-mentioned semantic segmentation method of agricultural remote sensing images based on a graph neural network in the frequency domain is implemented.
[0031] Advantages of the present invention:
[0032] (1) In the method of the present invention, after obtaining data input, graph data is constructed and then the data is input in batches. During the training process, the clustering of the graph neural network is Fourier-transformed, the data is transformed into the Fourier domain, the Fourier basis of the batch data is obtained during the training process, and the stability of the Fourier basis is judged to determine whether the training is completed according to its stability. This method can effectively improve the training efficiency of parameters.
[0033] (2) Since matrix calculations in the frequency domain can make the solution of many complex problems in the time domain more direct and simple. For example, the convolution operation in the time domain only requires element-wise multiplication in the frequency domain. And the present invention transforms matrix calculations into the frequency domain, so the present invention can improve the speed of large-scale data processing and effectively improve the efficiency and accuracy of agricultural remote sensing image processing. Description of the drawings
[0034] Figure 1 is a flowchart of a semantic segmentation method of agricultural remote sensing images based on a graph neural network in the frequency domain according to the present invention.
[0035] Figure 2 is an effect diagram of the prediction result of the present invention. Detailed implementation manners
[0036] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0037] Embodiment 1
[0038] According to Figure 1 the flowchart shown, a semantic segmentation method of agricultural remote sensing images based on a graph neural network in the frequency domain proposed by the present invention includes the following steps:
[0039] Step 1, obtain a remote sensing image, convert the remote sensing image into graph data, and use the graph data as input in batches;
[0040] Step 2, for the graph data input in batches, use the Lagrange function to calculate the eigenvalues of the graph data to obtain an eigenvalue matrix. The specific steps are as follows:
[0041] Step 2.1, construct the Lagrangian function according to the following mathematical properties of the circulant matrix, specifically as follows:
[0042] The mathematical properties of the circulant matrix are denoted as:
[0043]
[0044] where f(x i ) represents the characterization information of the i-th data sample x i , F is the Fourier matrix, diag is the diagonal matrix, is the i-th diagonal element of the diagonal matrix, and F H is the conjugate transpose matrix of the Fourier matrix.
[0045] The constructed Lagrangian function is denoted as:
[0046]
[0047] where is the conjugate of , ⊙ represents element-wise multiplication, D ii is the element in the i-th row and i-th column of the degree matrix, L ij is the element in the i-th row and j-th column of the Laplacian matrix, and the obtained Laplacian matrix is a symmetric matrix, Λ is the eigenvalue of the graph data, is the eigenvector corresponding to the eigenvalue.
[0048] Step 2.2, based on the constructed Lagrangian function, according to the properties of the circulant matrix and the properties of the Fourier matrix, all eigenvalues of the input batch of graph data can be obtained to get the eigenvalue matrix.
[0049] The calculation method of the eigenvalues is as follows:
[0050]
[0051] In this embodiment, by the above processing, the data is transformed into the Fourier domain, and the spectral information of the data can be effectively characterized in the Fourier domain.
[0052] In this embodiment, the properties of the above Fourier matrix are denoted as:
[0053] FF H =I
[0054] where I is the identity matrix.
[0055] Step 3: Based on the eigenvalue matrix obtained in Step 2, sort the eigenvalues in the eigenvalue matrix from largest to smallest, and extract the subscripts of the top k eigenvalues. According to the extracted k subscripts, extract the vectors with the same subscripts from the Fourier matrix, and form the Fourier basis with the extracted vectors.
[0056]
[0057] where is the vector extracted from the Fourier matrix, representing the k-th Fourier basis of the j-th batch input.
[0058] In the present invention, by selecting eigenvalues from the eigenvalue matrix and extracting the corresponding vectors from the Fourier matrix for the selected eigenvalues to form the Fourier basis, the time-consuming eigenvalue decomposition can be converted into the selection of pre-determined Fourier bases.
[0059] In this embodiment, the Fourier matrix mentioned in Step 3 is as follows:
[0060]
[0061] where ω = e -2πi / m and ω m = 1.
[0062] Step 4: Construct the loss function l' f , and iteratively optimize the weight parameters of the semantic segmentation model based on the Fourier basis and the loss function l' f ; if the iteration termination condition is satisfied, obtain the optimal weight parameters of the semantic segmentation model; otherwise, return to Step 1 to input the next batch of graph data for continued training.
[0063] Step 4.1: To improve the optimization efficiency of the model, the present invention defines a loss function l' f based on graph data in the Fourier domain, which is expressed as follows:
[0064]
[0065] where b is the batch number of the graph data input, S ij is the affinity matrix, d i is the sum of the i-th row of the degree matrix, denoted as d i = Σ j S ji ; g(z i ; θ) is the mapping function, θ is the set of parameters in the proposal network, denoted as θ = {W 1 , b 1 , W 2 , b 2 , W 3 , b 3}, W 1 , W 2 , W 3 are the weights of the 1st, 2nd, and 3rd layers in the network respectively, and b 1 , b 2 , b 3 are the biases of the 1st, 2nd, and 3rd layers in the network respectively; d j is the sum of the j-th row of the degree matrix, z i is the i-th batch of the input, z j is the j-th batch of the input, t is the temperature parameter used to control the sensitivity of similarity calculation, and ε is the similarity threshold.
[0066] Step 4.2, Take the Fourier basis obtained in Step 3 as the weight W of the 3rd layer of the network 3 and substitute it into the loss function l' f , and determine whether the loss function l' f converges. If it converges, perform backpropagation iteration update through the following formula to obtain the optimal weight parameters.
[0067] The method for backpropagation iteration update of other parameters is denoted as:
[0068] For W 2 , b 2 , the update method is:
[0069]
[0070] For W 1 , b 1 , the update method is:
[0071]
[0072] where, F k is the k-th Fourier basis, F is the Fourier basis, is the activation output of the 1st layer, is the activation output of the 2nd layer, s 2 is the affinity matrix of the 2nd layer, and g is the activation function.
[0073] Step 5, Based on the obtained optimal weight parameters, use the semantic segmentation model to perform semantic segmentation on agricultural remote sensing images and output the segmentation results.
[0074] To verify the effect of the proposed method of the present invention, the Agriculture-Vision dataset is selected as the data input for the clustering task. Combined with the attached Figure 2 , where Input is the input original image, GT is the corresponding ground truth label, and the rest are the prediction results of the corresponding model. It can be seen that compared with the original SCG model, the prediction results of the FSCG model using this method are closer to the ground truth label.
[0075] Example 2
[0076] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the above-mentioned method for semantic segmentation of agricultural remote sensing images based on a frequency-domain graph neural network is implemented.
[0077] Example 3
[0078] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the above-mentioned method for semantic segmentation of agricultural remote sensing images based on a frequency-domain graph neural network is implemented.
[0079] The above embodiments are only used to illustrate the design concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made according to the principles and design concepts disclosed by the present invention are within the protection scope of the present invention.
Claims
1. A frequency domain-based graph neural network agricultural remote sensing image semantic segmentation method, characterized in that: The steps include: Step 1: Obtain remote sensing images, convert them into graph data, and use the graph data as input in batches. Step 2: For the graph data input in batches, calculate the eigenvalues of the graph data to obtain an eigenvalue matrix. Step 3, based on the eigenvalue matrix obtained in step 2, sort the eigenvalues in the eigenvalue matrix from large to small, and extract the subscripts of the top k eigenvalues; Then, vectors with the same subscript are extracted from the Fourier matrix, and the extracted vectors form a Fourier basis; Step 4: construct a loss function, and iteratively optimize the weight parameters of the semantic segmentation model based on the Fourier basis and the loss function; if the iteration termination condition is met, the optimal weight parameters of the semantic segmentation model are obtained; if the iteration termination condition is not met, go to step 1 to input the next batch of graph data to continue training; Step 5: Based on the obtained optimal weight parameters, the semantic segmentation model is used to perform semantic segmentation on the agricultural remote sensing image and output the segmentation results.
2. According to claim 1, a frequency domain-based graph neural network agricultural remote sensing image semantic segmentation method is characterized in that: The steps to obtain the eigenvalue matrix in step 2 are as follows: Step 2.1, construct the Lagrangian function, denoted as: Step 2.2, based on the constructed Lagrangian function, all eigenvalues of the input batch image data are obtained to obtain an eigenvalue matrix; The eigenvalues are calculated as follows: Where F is the Fourier matrix, diag is the diagonal matrix, is the i-th diagonal element of the diagonal matrix, F H is the conjugate transposed matrix of the Fourier matrix, for The conjugate of , ⊙ represents the multiplication of the elements at the corresponding positions, D ii is the element in the i-th row and i-th column of the degree matrix, L ij is the i-th row and j-th column element of the Laplace matrix, and the obtained Laplace matrix is a symmetric matrix, Λ is the eigenvalue of the graph data, is the eigenvector corresponding to the eigenvalue.
3. The method for semantic segmentation of agricultural remote sensing images based on a frequency domain graph neural network according to claim 1 is characterized in that: The constructed loss function l' f Denoted as: Among them, b is the batch number of the image data input, S ij is the affinity matrix, g(z i ; θ) is the mapping function, θ is the parameter set in the proposed network, denoted as θ = {W1, b1, W2, b2, W3, b3}, W1, W2, W3 are the weights of the 1st, 2nd and 3rd layers in the network respectively, b1, b2, b3 are the biases of the 1st, 2nd and 3rd layers in the network respectively; d i is the sum of the i-th row of the degree matrix, d j is the sum of the jth row of the degree matrix, z i is the i-th batch of input, z j is the j-th batch of input, t is the temperature parameter used to control the sensitivity of similarity calculation, and ε is the similarity threshold.
4. The method for semantic segmentation of agricultural remote sensing images based on a frequency domain graph neural network according to claim 3 is characterized in that: The method for iteratively optimizing the weight parameters of the semantic segmentation model based on the Fourier basis and the loss function in step 4 is: Substitute the Fourier basis obtained in step 3 as the weight W3 of the third layer of the network into the loss function l' f , determine the loss function l' f Whether it converges, if it converges, it will be iteratively updated through the following back propagation to obtain the optimal weight parameters.
5. According to claim 4, a frequency domain-based graph neural network agricultural remote sensing image semantic segmentation method is characterized in that: The method of back propagation iteratively updating parameters is recorded as: The update method for W2 and b2 is: The update method for W1 and b1 is: Among them, F k is the kth Fourier basis, F is the Fourier basis, is the activation output of layer 1, is the activation output of the second layer, s2 is the affinity matrix of the second layer, and g is the activation function.
6. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the frequency domain-based graph neural network agricultural remote sensing image semantic segmentation method described in claim 1.
7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the frequency domain-based graph neural network agricultural remote sensing image semantic segmentation method described in claim 1.