Farmland parallax calculation method and calculation device

Abstract

The invention provides a farmland parallax calculation method and a farmland parallax calculation device which are applied to an agricultural scene, and the method comprises the steps: obtaining a first parallax value of a foreground and a second parallax value of a background of a farmland according to a binocular image of the farmland; obtaining a first Laplacian parameter and a second Laplacian parameter according to the first parallax value and the second parallax value; through the first Laplacian parameter, constructing a first Laplacian model; constructing a second Laplacian model through the second Laplacian parameter; and determining a third parallax value at the intersection of the foreground and the background by comparing the first probability density with the second probability density. The objective of the invention is to accurately obtain a parallax value at the intersection of a foreground and a background by constructing two Laplacian models.

Description

Technical Field

[0001] The present application relates to the field of agriculture, and more specifically, to a method and device for calculating farmland parallax. Background Art

[0002] In the agricultural operation process, the disparity map of the binocular image is obtained through the stereo matching algorithm. After the depth is calculated, the stereo image of the crop is obtained through 3D reconstruction to detect the growth status of the crop. In the process of calculating the disparity map of the binocular image, since there will be obvious edges between crops and crops, and between crops and soil during the field planting process, the edge points usually cannot reconstruct the boundaries of the object well. The process of obtaining edge disparity by fitting interpolation and other methods is too smooth, which will cause artifacts, and it is not conducive to subsequent 3D reconstruction after conversion into point cloud.

[0003] In view of this, how to provide a method to avoid over-smoothing of parallax calculation at edges during crop detection is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a method and device for calculating farmland disparity, which aims to use a bimodal Laplace distribution for modeling in the process of calculating disparity values ​​through binocular images, so that it can directly change from one disparity to another in discontinuous areas, avoid the problem of over-smoothness, and more accurately predict the disparity value, and further accurately obtain stereo images to detect the growth status of crops.

[0005] In a first aspect, a method for calculating farmland disparity is provided, the method comprising: obtaining a first disparity value of a foreground and a second disparity value of a background of the farmland according to a binocular image of the farmland, wherein the foreground is crops in the binocular image, and the background is ridges in the binocular image; obtaining a first Laplace parameter and a second Laplace parameter according to the first disparity value and the second disparity value; constructing a first Laplace model through the first Laplace parameter, the first Laplace model being used to characterize the relationship between the first disparity value and the first probability density; constructing a second Laplace model through the second Laplace parameter, the second Laplace model being used to characterize the relationship between the second disparity value and the second probability density; and determining a third disparity value at the intersection of the foreground and the background by comparing the first probability density and the second probability density.

[0006] Based on the above scheme, two Laplace models are established according to the disparity values ​​of the foreground and background, so that the disparity can be changed directly from one disparity to another in the discontinuous area of ​​the foreground and background (edge ​​points), avoiding the problem of over-smoothness in the existing technology through fitting, and then more accurately predicting the disparity value, and further accurately obtaining stereo images to detect the growth status of crops.

[0007] In combination with the first aspect, in certain implementations of the first aspect, a first disparity value of the foreground of the farmland and a second disparity value of the background are obtained based on a binocular image of the farmland, including: the binocular image is passed through a feature extraction network to obtain a left feature map and a right feature map; the left feature map and the right feature map are subjected to matching cost operations and cost aggregation operations to obtain a first disparity value and a second disparity value.

[0008] It should be understood that the left feature map and the right feature map are obtained through matching cost operation and cost aggregation operation to obtain a cost matrix (Cost Volume), and the first disparity value and the second disparity value are obtained through the cost matrix (Cost Volume).

[0009] In combination with the first aspect, in certain implementations of the first aspect, a third disparity value at the intersection of the foreground and the background is determined by comparing the first probability density and the second probability density, including: when the first probability density is greater than the second probability density, determining the first disparity value corresponding to the first probability density as the third disparity value; when the first probability density is less than the second probability density, determining the second disparity value corresponding to the second probability density as the third disparity value.

[0010] In combination with the first aspect, in some implementations of the first aspect, obtaining a first Laplace parameter and a second Laplace parameter according to the first disparity value and the second disparity value includes: the first disparity value and the second disparity value pass through a first disparity network to obtain the first Laplace parameter and the second Laplace parameter.

[0011] Optionally, the cost matrix (Cost Volume) may be input into the first disparity network to obtain the first Laplacian parameter and the second Laplacian parameter. Alternatively, the cost matrix (Cost Volume), the left feature map, and the right feature map may be input into the first disparity network to obtain the first Laplacian parameter and the second Laplacian parameter.

[0012] In combination with the first aspect, in certain implementations of the first aspect, the first disparity value and the second disparity value pass through the first disparity network to obtain the first Laplace parameter and the second Laplace parameter, including: through the first disparity network, the first disparity value and the vector size of the first feature point corresponding to the first disparity value, the second disparity value and the vector size of the second feature point corresponding to the second disparity value are mapped into a five-dimensional vector, the five-dimensional vector is used to characterize the vector size of the first Laplace parameter and the second Laplace parameter, the first feature point is a point in the left feature map and the right feature map in the foreground, and the second feature point is a point in the left feature map and the right feature map in the background.

[0013] In combination with the first aspect, in some implementations of the first aspect, the first Laplacian parameter includes a disparity distribution weight of a foreground, a peak value and a scale of the first Laplacian model.

[0014] In combination with the first aspect, in some implementations of the first aspect, the second Laplace parameters include a disparity distribution weight of the background, a peak value and a scale of the second Laplace model.

[0015] It should be understood that in the present application, the first Laplacian parameter and the second Laplacian parameter are five parameters in total because the relationship between the disparity distribution weight of the foreground and the disparity weight of the background is π and 1-π.

[0016] In combination with the first aspect, in some implementations of the first aspect, a first Laplace model is constructed by using a first Laplace parameter, and the first Laplace model is used to characterize the relationship between the first disparity value and the first probability density, including: characterizing the relationship between the first disparity value and the first probability density by using a first formula, wherein the first formula is:

[0017]

[0018] Among them, π is the disparity distribution weight of the foreground, b 1 is the scale of the first Laplace model, μ 1 is the peak value of the first Laplace model, d 1 is the first disparity value, p(d 1 ) is the first probability density.

[0019] In combination with the first aspect, in some implementations of the first aspect, a second Laplace model is constructed by a second Laplace parameter, and the second Laplace model is used to characterize the relationship between the second disparity value and the second probability density, including: characterizing the relationship between the second disparity value and the second probability density by a second formula, wherein the second formula is:

[0020]

[0021] Among them, (1-π) is the disparity distribution weight of the foreground, b 2 is the scale of the second Laplace model, μ 2 is the peak value of the second Laplace model, d 2 is the second disparity value, q(d 2 ) is the second probability density.

[0022] In combination with the first aspect, in certain implementations of the first aspect, a third disparity value at the intersection of the foreground and the background is determined by comparing the sizes of the first probability density and the second probability density, including: the first probability density and the second probability density are compared through a comparison model, and the sizes of the first probability density and the second probability density are compared to obtain the third disparity value at the intersection of the foreground and the background.

[0023] In combination with the first aspect, in some implementations of the first aspect, the first Laplace model, the second Laplace model, and the comparison model constitute a second parallax network, and the first parallax network and the second parallax network constitute a target parallax network.

[0024] In combination with the first aspect, in certain implementations of the first aspect, the target disparity network is trained by: obtaining first sample data and label data; training the initial network according to the first sample data and the label data to obtain the target disparity network; wherein the first sample data includes multiple groups of binocular image data of farmland, and the label data includes foreground data and background data in the multiple groups of binocular images.

[0025] In combination with the first aspect, in certain implementations of the first aspect, the target disparity network is obtained by training the initial network based on the first sample data and the label data, including: inputting the first sample data and the label into the initial network to obtain a fourth disparity value at the intersection of the foreground and background of multiple sets of binocular images; and adjusting the parameters of the initial network based on the fourth disparity value to train the target disparity network.

[0026] In combination with the first aspect, in some implementations of the first aspect, adjusting the parameters of the initial network according to the fourth disparity value includes: performing a first function operation on the fourth disparity value to adjust the parameters of the initial network, the first function operation including a cross entropy function operation.

[0027] In a second aspect, a device for calculating farmland parallax is provided, the device comprising: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute: a calculation method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic architecture diagram of a farmland to which an embodiment of the present application is applicable is shown.

[0029] Figure 2 A schematic flowchart of the parallax of crops provided in an embodiment of the present application is shown.

[0030] Figure 3 A schematic flowchart of a method for calculating farmland parallax provided in an embodiment of the present application is shown.

[0031] Figure 4 A schematic flow chart of a network model provided in an embodiment of the present application is shown.

[0032] Figure 5 A schematic flow chart of a target parallax network provided in an embodiment of the present application is shown.

[0033] Figure 6 A schematic flow chart of a device for calculating farmland parallax provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0035] The present application relates to the field of crop growth status detection, that is, by calculating the disparity value of the binocular image to obtain the depth value, and then based on the depth value and the geometric relationship of the camera, the crops are reconstructed in three dimensions to generate a three-dimensional point cloud or a three-dimensional model, and the growth status of the crops is detected through the stereo image formed by the three-dimensional point cloud or the three-dimensional model. Among them, the binocular image is fixed at a certain distance by two identical or similar cameras, and the angle and position of the camera are adjusted so that their fields of view overlap and the imaging planes are parallel, and a synchronization trigger or other synchronization mechanism is used to ensure that the two cameras capture the image at the same time.

[0036] Figure 1 A schematic architecture diagram of a farmland 100 to which an embodiment of the present application is applicable is shown.

[0037] like Figure 1 As shown, the farmland 100 includes crops 110 and ridges 120 for dividing the fields. The ridges 120 divide the farmland 100 into several small plots for easy zoning management and operation. During irrigation, the ridges 120 can guide water flow to ensure that the water flow is evenly distributed to each piece of farmland 100.

[0038] In the farmland 100 , since there are obvious edges between the crops 110 and between the crops 110 and the ridges 120 , in the process of calculating the disparity through binocular images, the edge points usually cannot reconstruct the boundaries of the objects well.

[0039] Figure 2 A schematic flowchart of the parallax of crops provided in an embodiment of the present application is shown.

[0040] like Figure 2 As shown in , the left side of x1 is the foreground of the crop, and the right side of x1 is the background of the crop. The edge point is where the disparity value of the foreground and background suddenly changes and becomes discontinuous. In the prior art, the disparity of the edge point is usually obtained by fitting interpolation, such as Figure 2From the black circles shown, it can be seen that the disparity value of the edge point obtained by the fitting interpolation method is too smooth and cannot accurately predict the disparity value of the edge point.

[0041] In view of this, an embodiment of the present application proposes a method for calculating farmland disparity, which aims to use a bimodal Laplace distribution for modeling in the process of calculating disparity values ​​through binocular images, so that it can directly change from one disparity to another in discontinuous areas, avoiding the problem of over-smoothness, and thus more accurately predict the disparity value, and further accurately obtain stereo images to detect the growth status of crops.

[0042] Figure 3 A schematic flowchart of a method 300 for calculating farmland parallax provided in an embodiment of the present application is shown.

[0043] like Figure 3 As shown, the method 300 for calculating farmland parallax includes the following steps.

[0044] S310: Obtaining a first disparity value of a foreground of a farmland and a second disparity value of a background according to the binocular image, wherein the foreground is the crops in the binocular image, and the background is the ridges in the binocular image.

[0045] S320: Obtain a first Laplace parameter and a second Laplace parameter according to the first disparity value and the second disparity value.

[0046] S330: constructing a first Laplace model using the first Laplace parameter, where the first Laplace model is used to characterize the relationship between the first disparity value and the first probability density.

[0047] S340: constructing a second Laplace model using the second Laplace parameter, where the second Laplace model is used to characterize the relationship between the second disparity value and the second probability density.

[0048] S350: Obtain a third disparity value at the intersection of the foreground and the background by comparing the first probability density and the second probability density.

[0049] Specifically, the execution subject of the embodiment of the method may be a cloud, a server or a computer device.

[0050] It should be understood that the above steps can also refer to Figure 4 It can be understood that the first disparity value and the second disparity value are passed through the first disparity network to obtain the first Laplace parameter and the second Laplace parameter, and the first Laplace parameter and the second Laplace parameter are sent to the first Laplace model and the results obtained by the second Laplace model are sent to the comparison model to obtain the third disparity value.

[0051] In the above step S310, the binocular image is passed through a feature extraction network to obtain a left feature map and a right feature map. The obtained left feature map and right feature map are subjected to matching cost calculation and cost aggregation calculation to obtain a first disparity value and a second disparity value.

[0052] It should be understood that a cost matrix (Cost Volume) is obtained by matching the cost and aggregating the cost, and the first disparity value and the second disparity value are obtained through the cost matrix (Cost Volume).

[0053] In the present application, the cost matrix (Cost Volume) can also be input into the first disparity network to obtain the first Laplace parameters and the second Laplace parameters, or the cost matrix (Cost Volume) and the left and right feature maps can be spliced ​​and input into the first disparity network to obtain the first Laplace parameters and the second Laplace parameters through more features.

[0054] Specifically, the binocular images (left and right) are first sent to the feature extraction network to obtain the left feature map and the right feature map, respectively, that is, the feature representation of each pixel is obtained through feature extraction. Among them, the feature extraction network selects the twin convolutional neural network, which is composed of two identical sub-networks, each of which is a convolutional neural network. The two sub-networks share the same weights and structures to ensure that the same features are extracted from the input binocular images. Each input image passes through its corresponding sub-network. In each sub-network, the left or right image passes through a series of convolutional layers and pooling layers to gradually extract the high-level features of the binocular image; and after processing by the convolutional layer and the pooling layer, each image is converted into a feature vector of a fixed length.

[0055] Among them, the left feature map and the right feature map both include feature maps with three different resolutions: downsampled 4 times, 8 times, and 16 times. Feature maps with different resolutions are determined by the convolution kernels of different layers of the convolutional neural network. The convolution kernel slides on the input image to generate a new feature map. Due to the different convolution kernels, the resolution of the final feature map is different.

[0056] Optionally, the input of the feature extraction network is the corrected left and right eye images. By correcting the left and right eye images in advance, the subsequent prediction accuracy is further improved.

[0057] In this application, a twin convolutional neural network can be used to extract features from binocular images, ensuring that the features of the left and right views are represented in the same feature space, making subsequent matching and comparison more accurate and consistent.

[0058] It should be understood that in the present application, feature extraction may use other methods, such as traditional feature extraction methods, and the present application does not impose any limitation on this.

[0059] In the matching cost calculation process, for a pixel in the left image, the matching cost between it and all possible corresponding pixels in the right image is calculated. The feature similarity between the left pixel and each pixel in the right image is calculated. Commonly used similarity measurement methods include Absolute Difference (AD), Squared Difference (SD), Mutual Information (MI), etc. For each pixel in the left image, the similarity measurement results of all pixels in the right image are accumulated to form a cost matrix (Cost Volume).

[0060] During the matching aggregation calculation process, for each pixel point, cost aggregation is performed within its neighborhood (such as a cross or diamond window) to reduce the impact of noise and outliers. A global optimization algorithm, such as belief propagation or graph cut, is applied to the entire cost matrix (Cost Volume) to consider the global information of the image. According to the aggregated cost matrix (Cost Volume), the optimal disparity value is selected for each left image pixel point, usually by finding the path with the minimum cost. This disparity value is the disparity that minimizes the matching cost, that is, the first disparity value and the second disparity value are finally obtained.

[0061] In this application, the matching cost calculation is implemented through a convolutional neural network. Specifically, the process is the same as the convolution operation. Through the convolutional neural network, the correlation between the left and right images is calculated to obtain a three-dimensional matrix of size w*h*d, which is as follows:

[0062]

[0063] Among them, w, h are the width and height of the left or right feature map, d is the search range of disparity, and s is the scale of the left or right feature map.

[0064] In this application, cost aggregation can be divided into two cases.

[0065] Case 1

[0066] The cost aggregation calculation formula at the same resolution is as follows:

[0067]

[0068] in is the aggregated matching cost at pixel q with disparity d, where pixel q belongs to the neighboring pixel of point p; C(d,q) is the original matching cost at pixel q with disparity d, and w(p,q) is the aggregation weight.

[0069] Case 2

[0070] The cost aggregation calculation formula at different resolutions is as follows:

[0071]

[0072] Among them, C ∧s is the cost matrix (Cost Volume) after aggregation at different resolutions, and C ～k is the cost volume after aggregation at the same resolution at the kth scale, f k It is a custom function representation. k Specifically, when the cost matrix (Cost Volume) of the kth scale is aggregated with other sth scales. When k is less than s, the cost matrix (Cost Volume) of other sth scales is downsampled and fused into the cost matrix (Cost Volume) of the kth scale. On the contrary, when k is greater than s, the cost matrix (Cost Volume) of the sth scale is upsampled and fused into the cost matrix (Cost Volume) of the resolution of this layer.

[0073] In the above step S320, the first disparity value and the second disparity value are passed through the first disparity network to obtain the first Laplacian parameter and the second Laplacian parameter.

[0074] Optionally, through the first disparity network, the first disparity value and the vector size of the first feature point corresponding to the first disparity value, the second disparity value and the vector size of the second feature point corresponding to the second disparity value are mapped into a five-dimensional vector, and the five-dimensional vector is used to characterize the vector size of the first Laplace parameter and the second Laplace parameter. The first feature point is a point in the left feature map and the right feature map in the foreground, and the second feature point is a point in the left feature map and the right feature map in the background.

[0075] Optionally, the first Laplace parameters include a disparity distribution weight of the foreground, a peak value and a scale of the first Laplace model. The second Laplace parameters include a disparity distribution weight of the background, a peak value and a scale of the second Laplace model.

[0076] In the above step S330, a first Laplace model is constructed by using the first Laplace parameter, and the first Laplace model is used to characterize the relationship between the first disparity value and the first probability density, including: characterizing the relationship between the first disparity value and the first probability density by using a first formula, wherein the first formula is:

[0077]

[0078] Among them, π is the disparity distribution weight of the foreground, b 1 is the scale of the first Laplace model, μ 1is the peak value of the first Laplace model, d 1 is the first disparity value, p(d 1 ) is the first probability density.

[0079] In the above step S340, a second Laplace model is constructed by using the second Laplace parameter, and the second Laplace model is used to characterize the relationship between the second disparity value and the second probability density. Specifically, the relationship between the second disparity value and the second probability density is characterized by a second formula, wherein the second formula is:

[0080]

[0081] Among them, (1-π) is the disparity distribution weight of the foreground, b 2 is the scale of the second Laplace model, μ 2 is the peak value of the second Laplace model, d 2 is the second disparity value, q(d 2 ) is the second probability density.

[0082] It should be understood that the five-dimensional vectors obtained by the first disparity network are (π, b 1 , μ 1 , b 2 , μ 2 ).

[0083] In the above step S350, the first probability density and the second probability density are compared by using a comparison model to obtain a third disparity value at the intersection of the foreground and the background.

[0084] Optionally, the comparison model can be expressed in the following manner: when the first probability density is greater than the second probability density, the first disparity value corresponding to the first probability density is determined as the third disparity value; when the first probability density is less than the second probability density, the second disparity value corresponding to the second probability density is determined as the third disparity value.

[0085] The first formula is used to model the foreground in the crop binocular image, and the second formula is used to model the background in the crop binocular image. For a specific pixel, a pattern with a higher probability is selected to determine whether the pixel belongs to the foreground or the background, to capture the discontinuous value of the disparity, and to obtain a sharp boundary, so that the point at the edge position will not be too smooth.

[0086] Optional, such as Figure 5 As shown, the first Laplace model, the second Laplace model and the comparison model constitute a second parallax network, and the first parallax network and the second parallax network constitute a target parallax network.

[0087] Among them, the first disparity network can be a multi-layer perceptron. Taking three layers as an example, the number of channels can be set to 74, 32, and 5. It should be understood that no matter how many layers the perceptron has and how many channels are set, the number of channels of the last layer can be set to 5.

[0088] The target disparity network in the above embodiment is trained in the following way:

[0089] First sample data and label data are obtained, and according to the first sample data and the label data, an initial network is trained to obtain a target disparity network; wherein the first sample data includes multiple groups of binocular image data of farmland, and the label data includes foreground data and background data in the multiple groups of binocular images.

[0090] The training process and the use process can be performed on different devices or clouds, or on the same device or cloud, and this application does not impose any limitations on this.

[0091] Specifically, the first sample data and the label are input into the initial network to obtain the fourth disparity value at the intersection of the foreground and the background of the multiple sets of binocular images; according to the fourth disparity value, the parameters of the initial network are adjusted to train and obtain the target disparity network.

[0092] It should be understood that during the training process of the neural network, the label is used to guide the network to learn the final result. That is, it tells the network what kind of result should be output for each input data point. By comparing the predicted output of the network with the true label, we can calculate the loss function and use this loss function to adjust the weight of the network to make the network's prediction more accurate.

[0093] Optionally, during the disparity prediction network training process, label setting can be divided into the following steps:

[0094] Step 1: Sample the left image in the binocular image, and the sampling points are N points composed of edge areas and non-edge areas in equal proportion.

[0095] Step 2: Calculate the disparity values ​​of N points respectively.

[0096] Step 3: Determine whether the point on the left image is an edge point according to the size of the disparity value, and then sample the edge points and non-edge points of all disparity values ​​according to a certain ratio to generate a disparity label.

[0097] It should be understood that in the above step three, edge points and non-edge points are sampled for all disparity values ​​at a certain ratio to balance the number of edge points and non-edge points in the disparity label, so that the initial network can better distinguish the foreground and background of the crop binocular image.

[0098] Optionally, when setting the label, a mask corresponding to the label is generated, and the weight of the label of the edge point is increased by the mask. In addition, the present application does not impose any limitation on the specific setting of the mask value.

[0099] In addition, in the binocular images of crops, since non-edge points (i.e., background and foreground) are easy to capture and the disparity values ​​of edge points (i.e., the intersection of background and foreground) are difficult to calculate, the weight of the edge point label can be increased through the mask during the label setting process.

[0100] Optionally, adjusting the parameters of the initial network according to the fourth disparity value includes: performing a first function operation on the fourth disparity value to adjust the parameters of the initial network, the first function operation including a cross entropy function operation.

[0101] Specifically, during the initial network training process, it is necessary to learn the disparity at the edge points. Since in the binocular image, the information at the edge points is not balanced with the information at the non-edge points, that is, in the scene of crop detection, although the ridges and crops alternate, the number of non-edge points of crops is still greater than the number of edge points. In the process of selecting the loss function, it is necessary to consider processing the unbalanced classification data. In this application, the cross entropy loss function is used to solve the above problem. The formula is as follows:

[0102]

[0103] Wherein, P(d) is the probability density value corresponding to the disparity value, and the probability density value includes the probability density value corresponding to the disparity value of the non-edge point.

[0104] Through the technical solution of the embodiment of the present application, in the process of calculating the disparity value through binocular images, a bimodal Laplace distribution is used for modeling, so that it is possible to directly change from one disparity to another in discontinuous areas, avoiding the problem of over-smoothness, and thus more accurately predict the disparity value, and further accurately obtain stereo images to detect the growth status of crops.

[0105] Combination of the above Figures 1 to 5 , describes the method embodiment of the present application in detail, and the following is combined with Figure 6 , describing the device embodiments of the present application, the device embodiments correspond to the method embodiments, so the parts that are not described in detail can refer to the method embodiments in the previous parts.

[0106] Figure 6A schematic flow chart of a farmland parallax calculation device 600 provided in an embodiment of the present application, the calculation device 600 includes: a processor 610 and a memory 620, the memory 620 is used to store computer programs, and the processor 610 is used to call and run the computer program stored in the memory 620 to execute: the calculation method as in the above embodiment.

[0107] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0108] Optionally, the computer-readable storage medium can be applied to the method for calculating farmland parallax in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes in each method of the embodiments of the present application, which will not be described in detail here for the sake of brevity.

[0109] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0110] Optionally, the computer program product can be applied to the method for calculating farmland parallax in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes in each method of the embodiments of the present application, which will not be described in detail here for the sake of brevity.

[0111] The embodiment of the present application also provides a computer program.

[0112] Optionally, the computer program can be applied to the method for calculating farmland parallax in the embodiments of the present application. When the computer program runs on a computer, the computer executes the corresponding processes in the various methods in the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0113] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0114] It should be understood that the specific examples in this article are only intended to help those skilled in the art better understand the embodiments of the present application, rather than to limit the scope of the embodiments of the present application.

[0115] It should also be understood that the various implementation methods described in this specification can be implemented individually or in combination, and the embodiments of the present application are not limited to this.

[0116] Unless otherwise stated, all technical and scientific terms used in the embodiments of the present application are the same as the meanings generally understood by those skilled in the art of the technical field of the application. The terms used in the present application are only for the purpose of describing specific embodiments, and are not intended to limit the scope of the present application. The singular forms "a kind of", "above" and "the" used in the embodiments of the present application and the appended claims are also intended to include majority forms, unless the context clearly indicates other meanings. In addition, the terms "first", "second" etc. are only used for description purposes, and cannot be interpreted as indicating or suggesting relative importance.

[0117] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0120] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0121] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for calculating farmland parallax, characterized in that: include: According to the binocular image of the farmland, a first disparity value of the foreground and a second disparity value of the background of the farmland are obtained, wherein the foreground is the crops in the binocular image, and the background is the ridge in the binocular image; Obtaining a first Laplace parameter and a second Laplace parameter according to the first disparity value and the second disparity value; constructing a first Laplace model using the first Laplace parameter, where the first Laplace model is used to characterize a relationship between the first disparity value and a first probability density; constructing a second Laplace model by using the second Laplace parameter, where the second Laplace model is used to characterize the relationship between the second disparity value and the second probability density; By comparing the first probability density and the second probability density, a third disparity value at the intersection of the foreground and the background is determined.

2. The calculation method according to claim 1, characterized in that: The method of obtaining a first disparity value of a foreground and a second disparity value of a background of the farmland according to the binocular image of the farmland comprises: The binocular image is passed through a feature extraction network to obtain a left feature map and a right feature map; The left feature map and the right feature map are subjected to matching cost operation and cost aggregation operation to obtain the first disparity value and the second disparity value.

3. The calculation method according to claim 1, characterized in that: The determining a third disparity value at the intersection of the foreground and the background by comparing the first probability density and the second probability density comprises: When the first probability density is greater than the second probability density, determining the first disparity value corresponding to the first probability density as the third disparity value; When the first probability density is less than the second probability density, the second disparity value corresponding to the second probability density is determined as the third disparity value.

4. The calculation method according to claim 2, characterized in that: The obtaining, according to the first disparity value and the second disparity value, a first Laplace parameter and a second Laplace parameter comprises: The first disparity value and the second disparity value are passed through a first disparity network to obtain the first Laplace parameter and the second Laplace parameter.

5. The calculation method according to claim 4, characterized in that: The first disparity value and the second disparity value are passed through a first disparity network to obtain the first Laplace parameter and the second Laplace parameter, including: Through the first disparity network, the first disparity value and the vector size of the first feature point corresponding to the first disparity value, and the second disparity value and the vector size of the second feature point corresponding to the second disparity value are mapped into a five-dimensional vector. Among them, the five-dimensional vector is used to characterize the vector size of the first Laplace parameter and the second Laplace parameter, the first feature point is the point in the left feature map and the right feature map in the foreground, and the second feature point is the point in the left feature map and the right feature map in the background.

6. The calculation method according to any one of claims 1 to 5, characterized in that: The first Laplacian parameters include a disparity distribution weight of the foreground, a peak value and a scale of the first Laplacian model.

7. The calculation method according to any one of claims 1 to 5, characterized in that: The second Laplacian parameters include a disparity distribution weight of the background, a peak value and a scale of the second Laplacian model.

8. The calculation method according to any one of claims 1 to 5, characterized in that: The step of constructing a first Laplace model by using the first Laplace parameter, wherein the first Laplace model is used to characterize the relationship between the first disparity value and the first probability density, includes: The relationship between the first disparity value and the first probability density is characterized by a first formula, wherein the first formula is: Among them, π is the disparity distribution weight of the foreground, b1 is the scale of the first Laplace model, μ1 is the peak value of the first Laplace model, d1 is the first disparity value, and p(d1) is the first probability density.

9. The calculation method according to any one of claims 1 to 5, characterized in that: The second Laplace model is constructed by using the second Laplace parameter, and the second Laplace model is used to characterize the relationship between the second disparity value and the second probability density, including: The relationship between the second disparity value and the second probability density is characterized by a second formula, wherein the second formula is: Among them, (1-π) is the disparity distribution weight of the foreground, b2 is the scale of the second Laplace model, μ2 is the peak value of the second Laplace model, d2 is the second disparity value, and q(d2) is the second probability density.

10. The calculation method according to claim 1, characterized in that: The determining a third disparity value at the intersection of the foreground and the background by comparing the first probability density and the second probability density comprises: The first probability density and the second probability density are compared by a comparison model to obtain a third disparity value at the intersection of the foreground and the background.

11. The calculation method according to claim 10, characterized in that: The first Laplace model, the second Laplace model, and the comparison model constitute a second parallax network, and the first parallax network and the second parallax network constitute a target parallax network.

12. The calculation method according to claim 11, characterized in that: The target disparity network is trained in the following way: Obtaining first sample data and label data; According to the first sample data and the label data, training the initial network to obtain the target disparity network; The first sample data includes multiple groups of binocular image data of the farmland, and the label data includes foreground data and background data in the multiple groups of binocular images.

13. The calculation method according to claim 12, characterized in that: The step of training the initial network according to the first sample data and the label data to obtain the target disparity network includes: Inputting the first sample data and the label into the initial network to obtain a fourth disparity value at the intersection of the foreground and the background of the multiple sets of binocular images; According to the fourth disparity value, the parameters of the initial network are adjusted to obtain the target disparity network through training.

14. The calculation method according to claim 13, characterized in that: The adjusting the parameters of the initial network according to the fourth disparity value includes: A first function operation is performed on the fourth disparity value to adjust parameters of the initial network, where the first function operation includes a cross entropy function operation.

15. A device for calculating farmland parallax, characterized in that include: A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute: the computing method according to any one of claims 1 to 14.

Images