Image data processing method, device, equipment and storage medium

Abstract

The present invention provides an image data processing method, device, equipment and storage medium, including: obtaining image features of a first image and a second image, wherein the image features include texture components and structural features, the first image is an image of a three-dimensional sand dune taken by a walker at a first viewing angle, and the second image is an image of a three-dimensional sand dune taken by the walker at a second viewing angle; performing feature matching after feature extraction on the texture components of the first image and the texture components of the second image, respectively, to obtain a first matching point group; performing dense matching and forward intersection based on the first matching point group, to obtain a three-dimensional point cloud of the three-dimensional sand dune in the first field of view of the walker in three-dimensional coordinates; interpolating the three-dimensional point cloud to obtain a digital elevation model. The technical solution of the present invention can improve the three-dimensional reconstruction effect of sand dune images.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an image data processing method, device, electronic device and non-transitory computer-readable storage medium. Background Art

[0002] In deep space exploration missions, topographic mapping of the rover landing area is essential basic data for long-distance route design and safe driving, and can also provide three-dimensional topographic information for scientific research on planetary surface targets. Rover stereo images are widely used for mapping small areas within a few dozen meters around each station within the rover's travel range. The mapping results can provide detailed topographic information to formulate short-term travel exploration plans and conduct research on scientific targets.

[0003] In the Mars exploration mission, image matching is difficult because the dune images have similar colors, repeated textures, and fuzzy edge contour information. This makes it difficult to use commonly used feature extraction and feature matching methods to meet the needs of fine mapping and to achieve high-precision three-dimensional reconstruction of the Mars rover dune images. Summary of the invention

[0004] The present invention provides an image data processing method, device, electronic device and non-transitory computer-readable storage medium, which are used to solve the problem of difficult dune image matching and achieve high-precision three-dimensional reconstruction.

[0005] The present invention provides an image data processing method, comprising: obtaining image features of a first image and a second image, wherein the image features include texture components and structural features, the first image is an image of a three-dimensional sand dune taken by a walker at a first viewing angle, and the second image is an image of a three-dimensional sand dune taken by the walker at a second viewing angle; performing feature matching after respectively performing feature extraction on the texture components of the first image and the texture components of the second image to obtain a first matching point group; performing dense matching and forward intersection based on the first matching point group to obtain a three-dimensional point cloud of the three-dimensional sand dune in the first field of view of the walker under three-dimensional coordinates; and performing interpolation processing on the three-dimensional point cloud to obtain a digital elevation model.

[0006] According to an image data processing method provided by the present invention, before acquiring the image features of the first image and the second image, the method further includes: performing texture enhancement on the first image and the second image respectively to obtain corresponding texture components and structural features.

[0007] According to an image data processing method provided by the present invention, the feature extraction of the texture components of the first image and the texture components of the second image are respectively performed and then feature matching is performed, comprising: using the ASIFT algorithm to extract the features of the texture components of the first image and the texture components of the second image respectively and then perform feature matching.

[0008] According to an image data processing method provided by the present invention, the feature extraction of the texture components of the first image and the texture components of the second image are respectively followed by feature matching, comprising: extracting the features of the texture components of the first image and the texture components of the second image respectively and then performing feature matching to obtain an initial matching point group; using the RANSAC algorithm to eliminate gross errors of erroneous matching points in the initial matching point group to obtain the first matching point group.

[0009] According to an image data processing method provided by the present invention, dense matching and forward intersection are performed based on the first matching point group, including: after constructing a triangulated network using the first matching point group, obtaining a feature vector composed of a multi-window structure tensor of the to-be-matched point; obtaining an angle between different feature vectors of the to-be-matched point based on the feature vector; densely matching the to-be-matched points based on the angle to obtain dense matching points; and performing a forward intersection based on the dense matching points and the internal and external orientation elements of the image to obtain a three-dimensional point cloud under the three-dimensional coordinates.

[0010] According to an image data processing method provided by the present invention, before performing interpolation processing on the three-dimensional point cloud, the method further includes: removing gross errors from the three-dimensional point cloud.

[0011] According to an image data processing method provided by the present invention, before texture enhancement is performed on the first image and the second image respectively, the method further includes: reading a stereoscopic sand dune image taken by the walker, the stereoscopic sand dune image including stereoscopic images taken by the walker at various viewing angles and internal and external orientation elements of a single image; and acquiring the first image and the second image based on the stereoscopic sand dune image.

[0012] The present invention provides an image data processing device, comprising: an acquisition unit, used to acquire image features of a first image and a second image, wherein the image features include texture components and structural features, the first image is an image of a three-dimensional sand dune taken by a walker at a first viewing angle, and the second image is an image of a three-dimensional sand dune taken by the walker at a second viewing angle; a feature matching unit, used to perform feature matching after performing feature extraction on the texture components of the first image and the texture components of the second image respectively, to obtain a first matching point group; a dense matching unit, used to perform dense matching and forward intersection based on the first matching point group, to obtain a three-dimensional point cloud of the three-dimensional sand dune in the first field of view of the walker under three-dimensional coordinates; and an interpolation unit, used to perform interpolation processing on the three-dimensional point cloud, to obtain a digital elevation model.

[0013] According to an image data processing device provided by the present invention, the device also includes a texture enhancement unit, which is used to perform texture enhancement on the first image and the second image respectively to obtain corresponding texture components and structural features.

[0014] According to an image data processing device provided by the present invention, the feature matching unit is further used to use an ASIFT algorithm to extract features from the texture components of the first image and the texture components of the second image respectively and then perform feature matching.

[0015] According to an image data processing device provided by the present invention, the feature matching unit is also used to perform feature matching after feature extraction on the texture components of the first image and the texture components of the second image respectively to obtain an initial matching point group; use the RANSAC algorithm to eliminate gross errors of erroneous matching points in the initial matching point group to obtain the first matching point group.

[0016] According to an image data processing device provided by the present invention, the dense matching unit is also used to obtain a feature vector composed of a multi-window structure tensor of the to-be-matched points in the first matching point group after constructing a triangulated network using the first matching point group; obtain the angle between the feature vectors of different to-be-matched points according to the feature vector; perform dense matching on the to-be-matched points according to the angle to obtain dense matching points; and perform forward intersection with the dense matching points and the internal and external orientation elements of the image to obtain a three-dimensional point cloud under the three-dimensional coordinates.

[0017] According to an image data processing device provided by the present invention, the device further comprises a gross error elimination unit for eliminating gross errors from the three-dimensional point cloud.

[0018] According to an image data processing device provided by the present invention, the device also includes a selection unit for reading a stereoscopic sand dune image taken by the walker, wherein the stereoscopic sand dune image includes stereoscopic images taken by the walker at various viewing angles and internal and external orientation elements of a single image; and the first image and the second image are obtained based on the stereoscopic sand dune image.

[0019] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned image data processing methods are implemented.

[0020] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned image data processing methods are implemented.

[0021] The image data processing method, device, electronic device and non-transitory computer-readable storage medium provided by the present invention respectively extract the image features of the dune images from different perspectives and then perform feature matching, and further perform dense matching and forward intersection on the basis of the first matching point group to obtain a three-dimensional point cloud of the matching points, and then obtain a digital elevation model. Compared with the existing technology, a digital elevation model with higher accuracy can be obtained.

[0022] The technical solution of the embodiment of the present invention combines the texture components of the original image to extract feature points, increases the feature points, and combines the characteristic vectors of the structure tensor to densely match the dune image to obtain a dense point result, which has better applicability, higher accuracy and better reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 It is one of the flowcharts of the image data processing method provided by the present invention;

[0025] Figure 2 This is the second flow chart of the image data processing method provided by the present invention;

[0026] Figure 3 It is a structural schematic diagram of the image data processing device provided by the present invention;

[0027] Figure 4It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The terms used in one or more embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present invention. The singular forms of "a", "said" and "the" used in one or more embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present invention refers to and includes any or all possible combinations of one or more associated listed items.

[0030] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present invention, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present invention, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0031] Exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0032] like Figure 1 FIG. 1 is a flowchart of an image data processing method according to an embodiment of the present invention. The method provided by the embodiment of the present invention can be executed by any electronic device with computer processing capabilities, such as a terminal or a server. Figure 1 As shown, the image data processing method includes:

[0033] Step 102, obtaining image features of the first image and the second image, wherein the image features include texture components and structural features, the first image is an image of a three-dimensional sand dune taken by the walker at a first perspective, and the second image is an image of a three-dimensional sand dune taken by the walker at a second perspective.

[0034] Specifically, the walker may be a Mars rover equipped with a camera device, and the three-dimensional sand dunes may be sand dunes on Mars.

[0035] Step 104 , extracting features from the texture components of the first image and the texture components of the second image respectively and then performing feature matching to obtain a first matching point group.

[0036] Step 106 , performing dense matching and forward intersection based on the first matching point group to obtain a three-dimensional point cloud of the three-dimensional sand dune in the first field of view of the walker in three-dimensional coordinates.

[0037] Specifically, the forward intersection is to determine the object space coordinates of the point by the image coordinate values ​​of the same-name image point P and the internal and external orientation elements of the first and second field of view images. The first field of view can be the field of view of the first direction range of the walker, and the image in the first field of view is the overlapped part of the first image and the second image.

[0038] Step 108: interpolate the three-dimensional point cloud to obtain a digital elevation model.

[0039] The technical solution in the embodiment of the present invention makes full use of images from different regions to complete the mapping of the Mars rover, thereby broadening the terrain mapping capability of the Mars rover and better meeting engineering needs. The technical method is based on the dune images and attitude information of the Mars rover, performs feature matching on the basis of extracting texture components, and then completes dense matching by combining the feature points and the feature vectors of other points to be matched, and finally generates a digital elevation model based on the coordinates of the matching points and the attitude information, thereby achieving dense matching of the dune images on the basis of making full use of the dune images obtained by the Mars rover, and finally obtaining high-precision mapping results.

[0040] Before step 102, it is necessary to read the stereoscopic dune image taken by the walker, which includes the stereoscopic images taken by the walker at various viewing angles and the internal and external orientation elements of a single image, and obtain the first image and the second image based on the stereoscopic dune image.

[0041] The first field of view may be the front field of view of the Mars rover, and the first image and the second image may be images captured from a left front perspective and a right front perspective of the Mars rover, respectively.

[0042] Before step 102, the first image and the second image are respectively subjected to texture enhancement using a variational method to obtain corresponding texture components and structural features.

[0043] Specifically, the variational method is a method for solving the extreme value function of a functional and the functional extreme value corresponding to the extreme value function.

[0044] In step 102, an adaptive variational method may be used to perform texture enhancement on the Mars rover stereo dune image to obtain corresponding texture components and structural components. The specific process includes:

[0045] The data term E that defines the energy function dataIt is the following formula (1):

[0046] E data =(I p -S p ) 2 (1)

[0047] Among them, the original image is I, the structural component of the image is S, I p and S p That is, the original image and structural component corresponding to p pixels.

[0048] According to the two formulas in formula (2), the corresponding lateral total variation value D in each pixel neighborhood window is calculated respectively: x (p) and the total longitudinal variation D y (p):

[0049]

[0050] Among them, q is the pixel in the neighborhood with p as the center of the window, g p,q is the weight coefficient corresponding to the pixel, and R(p) represents the neighborhood with p as the window center.

[0051] The calculation formula is as follows (3):

[0052]

[0053] σ is the scale parameter of the window.

[0054] According to the two formulas in the following formula (4), the total variation H of all horizontal directions in the image is calculated respectively: x (p), the total variation value of all longitudinal directions H y (p):

[0055]

[0056] The spatial term of the energy function is established according to the following formula (5):

[0057]

[0058] Here, ε is a very small positive value.

[0059] The gradient constraint of the energy function is established according to the following formula (6):

[0060]

[0061] The final energy function E can be expressed by the following formula (7):

[0062]

[0063] Among them, β is the weight coefficient of the spatial term, and λ is the weight coefficient of the gradient constraint term. The minimum value of the energy function is iteratively calculated using the gradient descent method. When the difference between the previous and next energy functions meets the iteration threshold, the calculation ends and the structural component S is obtained.

[0064] According to the formula T=IS and the original image and the structural component S, the texture component T can be obtained.

[0065] In step 104, the ASIFT algorithm may be used to extract features from the texture components of the first image and the texture components of the second image and then perform feature matching.

[0066] The method of feature matching is to first find the feature points with significant features (Feature Detect), then describe the two feature points separately (Feature Descriptor), and finally compare the similarity of the two descriptions to determine whether they are the same feature (Feature Match). If the direction of the feature can be determined before feature description, rotation invariance can be achieved (Rotation invariant), and if the scale can be determined, scale invariance can be achieved (Scale invariant).

[0067] Specifically, the ASIFT algorithm can simulate two parameters of the affine matrix, namely the tilt angle and rotation angle of the camera and the image normal, so as to accurately simulate all images deformed by the change in the direction of the camera optical axis, and then use the SIFT algorithm to extract and match the features of these simulated images. Among them, the SIFT (Scale Invariant Feature Transform) algorithm is a local feature description algorithm in the field of image processing. The ASIFT algorithm is a completely affine invariant image feature matching algorithm.

[0068] In step 104, feature extraction can be performed on the texture components of the first image and the texture components of the second image respectively, and then feature matching can be performed to obtain an initial matching point group, and the RANSAC algorithm can be used to eliminate erroneous matching points in the initial matching point group to obtain a first matching point group.

[0069] In the embodiment of the present invention, it is not limited to using the RANSAC algorithm for gross error elimination, and other gross error elimination methods may also be used, such as a difference method, a quantile method, etc. Among them, the initial matching point group is a combination of all matching points that have not been eliminated by gross errors, and the first matching point group is a combination of correct matching points that have been eliminated by gross errors.

[0070] Specifically, the RANSAC (RANdom SAmple Consensus) algorithm can be used to iteratively estimate the parameters of the mathematical model from a set of observed data containing outliers.

[0071] In step 106, a triangulated network needs to be constructed. After the triangulated network is constructed using the first matching point group, a feature vector consisting of a multi-window structure tensor of the to-be-matched points in the first matching point group is obtained, and the angle between the feature vectors of different to-be-matched points is obtained based on the feature vector, and then dense matching is performed based on the angle. The specific process is as follows:

[0072] First, a triangulated network is constructed in the image space through the matched feature points, and interpolation is performed to obtain the disparity range set of point P. The disparity value of point P obtained by interpolation using the plane equation is the following formula (8):

[0073] d p =ax p +by p +c (8)

[0074] Among them, d P is the disparity value of point P obtained by interpolation, (x p ,y P ) is the image point coordinate of P, a, b, c are constants obtained by solving the linear equations, and are the coefficients of the geometric relationship between the image point coordinates and the corresponding disparity values. a, b, c can be solved using the following formula (9):

[0075]

[0076] Where (x A ,y A )、(x B ,y B )、(x C ,y C ) are the coordinates of vertices A, B, and C, respectively, (d A , d B , d C ) is the disparity of vertices A, B, and C. The disparity of feature matching points is regarded as Gaussian distribution, and the twice variance is calculated according to statistics as the search radius r, and the disparity range set is obtained as D = {d|d P -r≤d≤d P +r}.

[0077] According to the following formula (10), the matching point P(x p ,y p ) is the structure tensor T3 corresponding to the 3*3 window centered at:

[0078]

[0079] Among them, G σ is a Gaussian function with standard deviation σ, g x is the horizontal gradient of each pixel, g y is the vertical gradient of each pixel.

[0080] The structure tensor T7 corresponding to the 7*7 window centered at the point to be matched P is calculated according to the following formula (11):

[0081]

[0082] According to the following formula (12), the structure tensor T corresponding to the 11*11 window centered on the matching point P is calculated: 11 :

[0083]

[0084] Calculate the eigenvalues ​​corresponding to each structure tensor in turn

[0085] The structural tensor angle θ3 of the 3*3 window is calculated according to the eigenvalue and eigenvector and the following formula (13):

[0086]

[0087] The tensor angle θ7 corresponding to the 7*7 window is calculated according to the following formula (14):

[0088]

[0089] The tensor angle θ corresponding to the 11*11 window is calculated according to the following formula (15): 11 :

[0090]

[0091] The tensor angle difference is calculated according to the following formula (16) and formula (17):

[0092] =θ 73 =θ7-θ3 (16)

[0093] Δθ 113 =θ 11 -θ3 (17)

[0094] The transformed eigenvectors D1 and D2 are calculated according to the following formulas (18) and (19) and the angle difference:

[0095]

[0096]

[0097] According to the following formula (20) and formula (21) and the eigenvalues ​​obtained from different windows, the scale coefficient S corresponding to the eigenvector is calculated: 73 and S 113 :

[0098]

[0099]

[0100] According to formula (22), the feature vector obtained by multiplying the scale coefficient and the angle difference is obtained to obtain the feature vector V composed of the multi-window structure tensor of the first image to be matched point P of the stereo image P :

[0101]

[0102] According to the parallax range D of point P, the candidate same-name point Q of point P in the second image is obtained. i The coordinates Q i (x p -D i ,y p ), use the steps to calculate the eigenvectors V1, V2…V i .

[0103] Calculate the matching point P and each candidate point Q respectively i The cosine of the angle between the eigenvectors of , where the candidate point corresponding to the minimum value of the cosine of the angle is the point of the same name of P.

[0104] After completing the matching of the to-be-matched point P, the other to-be-matched points in the first image are matched in turn, and dense matching can be completed.

[0105] After dense matching, it is necessary to perform forward intersection on the calculated dense matching points combined with the internal and external orientation elements of the image to obtain a three-dimensional point cloud under three-dimensional coordinates.

[0106] When performing forward intersection, first calculate the image space coordinates of the image matching points based on the input dense matching point image point coordinates and the internal orientation elements (x0, y0). Then calculate the rotation matrix R based on the input internal and external orientation elements, where i is a natural number, and the formula is:

[0107]

[0108] a i ,b i ,c i Direction cosines.

[0109] a1=cosφcosκ-sinφsinωsinκ

[0110] a2=-cosφsinκ-sinφsinωcosκ

[0111] a3=-sinφcosω

[0112] b1=cosωsinκ

[0113] b2=cosωcosκ

[0114] b3=-sinω

[0115] c1=sinφcosκ

[0116] c2=-sinφsinκ+cosφsinωcosκ

[0117] c3=cosφcosω

[0118] According to the collinear equation, we can get:

[0119] (x-x0)[a3(XX S )+b3(YY S )+c3(ZZ S )]=

[0120] -f[a1(XX S )+b1(YY S )+c1(ZZ S )]

[0121] (y-y0)[a3(XX S )+b3(YY S )+c3(ZZ S )]=

[0122] -f[a2(XX S )+b2(YY S )+c2(ZZ S )]

[0123] Among them, (x, y) represents the coordinates of the image point in the image plane coordinate system; (X, Y, Z) represents the coordinate value of the target in the object coordinate system; R represents the rotation matrix of the camera; f represents the focal length of the camera; (X S ,Y S ,Z S ) represents the three-dimensional coordinates of the camera image principal point.

[0124] Simplifying gives:

[0125] l1X+l2Y+l3Z-l x =0

[0126] l4X+l5Y+l6Z-l y =0

[0127] in,

[0128] l1=fa1+(x-x0)a3, l2=fb1+(x-x0)b3, l3=fc1+(x-x0)c3

[0129] l x =fa1X S +fb1Y S +fc1Z S +(x-x0)a3X S +(x-x0)b3Y S +(x-x0)c3Z S

[0130] l4=fa2+(y-y0)a3, l5=fb2+(y-y0)b3, l6=fc2+(y-y0)c3

[0131] l y =fa2X S +fb2Y S +fc2Z S +(y-y0)a3X S +(y-y0)b3Y S +(y-y0)c3Z S

[0132] The three-dimensional coordinates (X, Y, Z) of each dense matching point can be obtained using the least squares method.

[0133] Among them, the least squares method is a mathematical optimization technology, which finds the best function matching of data by minimizing the sum of squares of errors. The least squares method can be used to easily obtain unknown data and minimize the sum of squares of errors between the obtained data and the actual data.

[0134] Before step 108, the three-dimensional point cloud may also be subjected to gross error elimination. The main steps are:

[0135] For any point P in the three-dimensional point cloud, the KNN (K-Nearest Neighbor) search algorithm is used to find its k nearest neighbor points. The distances between point P and the k nearest neighbor points are calculated respectively, and the average M and standard deviation S of the distances are obtained. If M>QS, that is, the average distance between P and its k neighboring points is greater than Q times its standard deviation, then point P is considered to be a gross error point. Among them, Q is a pre-set natural number. For example, Q can be taken as 3, that is, point P can be considered to be a gross error point when M>3S. Gross error points need to be eliminated, and non-gross error points are retained. After calculating all points in the three-dimensional point cloud in turn, gross error elimination is completed. Gross error elimination of a three-dimensional point cloud can be understood as filtering of a three-dimensional point cloud.

[0136] In step 108, Kriging interpolation may be performed on the three-dimensional point cloud from which errors have been removed to generate a corresponding digital elevation model.

[0137] Specifically, Kriging is a regression algorithm for spatial modeling and prediction of random processes or random fields based on covariance functions. It has better linear non-estimation.

[0138] like Figure 2 As shown, an image data processing process of an embodiment of the present invention includes the following steps:

[0139] Step 241 , performing texture enhancement on the Mars rover left image 211 to obtain a structural component 212 and a texture component 213 .

[0140] Step 242 , performing texture enhancement on the Mars rover right image 221 to obtain a structural component 222 and a texture component 223 .

[0141] Step 243 , ASFIT feature point matching, implements feature extraction and feature matching of texture components of the Mars rover left image 211 and the Mars rover right image 221 .

[0142] Step 244, gross error elimination, eliminates erroneous matching points in the matching point group obtained by feature matching.

[0143] Step 245: densely match the correctly matched matching points to obtain dense matching points.

[0144] Step 246 , performing forward intersection on the dense matching points combined with the internal and external orientation elements of the image to obtain a three-dimensional point cloud 231 .

[0145] Step 247: point cloud filtering to eliminate gross errors in the three-dimensional point cloud.

[0146] Step 248 , performing Kriging interpolation on the three-dimensional point cloud from which errors have been removed, to obtain the digital elevation model 232 .

[0147] The technical solution of the embodiment of the present invention combines the texture components of the original image to extract feature points, increases the feature points, and combines the characteristic vectors of the structure tensor to densely match the dune image to obtain a dense point result, which has better applicability, higher accuracy and better reliability.

[0148] The image data processing method provided by the present invention performs feature matching after extracting the image features of sand dune images from different viewing angles, and further performs dense matching and forward intersection on the basis of the first matching point group to obtain a three-dimensional point cloud of matching points, and then obtains a digital elevation model. Compared with the prior art, a digital elevation model with higher accuracy can be obtained.

[0149] The image data processing device provided by the present invention is described below. The image data processing device described below and the image data processing method described above can be referenced to each other.

[0150] like Figure 3 As shown, an image data processing device according to an embodiment of the present invention includes:

[0151] The acquisition unit 302 can be used to acquire image features of the first image and the second image, wherein the image features include texture components and structural features, the first image is an image of a three-dimensional sand dune taken by the walker from a first perspective, and the second image is an image of a three-dimensional sand dune taken by the walker from a second perspective.

[0152] The feature matching unit 304 may be used to extract features from the texture components of the first image and the texture components of the second image respectively and then perform feature matching to obtain a first matching point group.

[0153] The dense matching unit 306 can be used to perform dense matching and forward intersection based on the first matching point group to obtain a three-dimensional point cloud of the three-dimensional sand dune in the first field of view of the walker in three-dimensional coordinates.

[0154] The interpolation unit 308 may be used to perform interpolation processing on the three-dimensional point cloud to obtain a digital elevation model.

[0155] The technical solution in the embodiment of the present invention makes full use of images from different regions to complete the mapping of the Mars rover, thereby broadening the terrain mapping capability of the Mars rover and better meeting engineering needs. The technical method is based on the dune images and attitude information of the Mars rover, performs feature matching on the basis of extracting texture components, and then completes dense matching by combining the feature points and the feature vectors of other points to be matched, and finally generates a digital elevation model based on the coordinates of the matching points and the attitude information, thereby achieving dense matching of the dune images on the basis of making full use of the dune images obtained by the Mars rover, and finally obtaining high-precision mapping results.

[0156] In an embodiment of the present invention, the device may also include a selection unit for reading a stereoscopic sand dune image taken by the walker, the stereoscopic sand dune image including stereoscopic images taken by the walker from various perspectives and internal and external orientation elements of a single image; and obtaining a first image and a second image based on the stereoscopic sand dune image.

[0157] In the embodiment of the present invention, the apparatus may further include a texture enhancement unit, which is used to perform texture enhancement on the first image and the second image respectively to obtain corresponding texture components and structural features.

[0158] In the embodiment of the present invention, the feature matching unit may also be used to use the ASIFT algorithm to extract features from the texture components of the first image and the texture components of the second image respectively and then perform feature matching.

[0159] The method of feature matching is to first find the feature points with significant features (Feature Detect), then describe the two feature points separately (Feature Descriptor), and finally compare the similarity of the two descriptions to determine whether they are the same feature (Feature Match). If the direction of the feature can be determined before feature description, rotation invariance can be achieved (Rotation invariant), and if the scale can be determined, scale invariance can be achieved (Scale invariant).

[0160] Specifically, the ASIFT algorithm can simulate two parameters of the affine matrix, namely the tilt angle and rotation angle of the camera and the image normal, so as to accurately simulate all images deformed by the change in the direction of the camera optical axis, and then use the SIFT algorithm to extract and match the features of these simulated images. Among them, the SIFT algorithm is a local feature description algorithm in the field of image processing. The ASIFT algorithm is a completely affine invariant image feature matching algorithm.

[0161] In an embodiment of the present invention, the feature matching unit can also be used to perform feature matching after feature extraction on the texture components of the first image and the texture components of the second image respectively to obtain an initial matching point group; use the RANSAC algorithm to eliminate erroneous matching points in the initial matching point group to obtain a first matching point group.

[0162] In the embodiment of the present invention, it is not limited to using the RANSAC algorithm for gross error elimination, and other gross error elimination methods may also be used, such as a difference method, a quantile method, etc. Among them, the initial matching point group is a combination of all matching points that have not been eliminated by gross errors, and the first matching point group is a combination of correct matching points that have been eliminated by gross errors.

[0163] In an embodiment of the present invention, the dense matching unit can also be used to obtain a feature vector composed of a multi-window structure tensor of the points to be matched in the first matching point group after constructing a triangulated network using the first matching point group; obtain the angle between the feature vectors of different points to be matched based on the feature vector; perform dense matching on the points to be matched based on the angle to obtain dense matching points; and perform forward intersection with the dense matching points and the internal and external orientation elements of the image to obtain a three-dimensional point cloud under three-dimensional coordinates.

[0164] Specifically, the cosine of the angle between the feature vectors of the point to be matched P and each candidate point is calculated, where the candidate point corresponding to the minimum value of the cosine of the angle is the same-name point of P. After completing the matching of the point to be matched P, the other points to be matched in the first image are matched in turn, and dense matching can be completed.

[0165] After dense matching, it is necessary to perform forward intersection on the calculated dense matching points combined with the internal and external orientation elements of the image to obtain a three-dimensional point cloud under three-dimensional coordinates.

[0166] In the embodiment of the present invention, the device may further include a gross error elimination unit for eliminating gross errors from the three-dimensional point cloud.

[0167] Since the various functional modules of the image data processing device of the exemplary embodiment of the present invention correspond to the steps of the exemplary embodiment of the above-mentioned image data processing method, for details not disclosed in the embodiment of the device of the present invention, please refer to the embodiment of the image data processing method of the present invention.

[0168] The technical solution of the embodiment of the present invention combines the texture components of the original image to extract feature points, increases the feature points, and combines the characteristic vectors of the structure tensor to densely match the dune image to obtain a dense point result, which has better applicability, higher accuracy and better reliability.

[0169] The image data processing device provided by the present invention performs feature matching after extracting the image features of the dune images from different perspectives, and further performs dense matching and forward intersection on the basis of the first matching point group to obtain a three-dimensional point cloud of the matching points, and then obtains a digital elevation model. Compared with the existing technology, a digital elevation model with higher accuracy can be obtained.

[0170] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the image data processing method, which includes: obtaining image features of a first image and a second image, wherein the image features include texture components and structural features, the first image is an image of a three-dimensional sand dune taken by a walker at a first perspective, and the second image is an image of a three-dimensional sand dune taken by the walker at a second perspective; extracting features from the texture components of the first image and the texture components of the second image respectively and performing feature matching to obtain a first matching point group; performing dense matching and forward intersection based on the first matching point group to obtain a three-dimensional point cloud of the three-dimensional sand dune in the first field of view of the walker under three-dimensional coordinates; interpolating the three-dimensional point cloud to obtain a digital elevation model.

[0171] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0172] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the image data processing method provided by the above methods, and the method includes: obtaining image features of a first image and a second image, wherein the image features include texture components and structural features, the first image is an image of a three-dimensional sand dune taken by a walker from a first perspective, and the second image is an image of a three-dimensional sand dune taken by the walker from a second perspective; performing feature extraction on the texture components of the first image and the texture components of the second image, respectively, and then performing feature matching to obtain a first matching point group; performing dense matching and forward intersection based on the first matching point group to obtain a three-dimensional point cloud of the three-dimensional sand dunes in the first field of view of the walker in three-dimensional coordinates; interpolating the three-dimensional point cloud to obtain a digital elevation model.

[0173] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned image data processing methods, the methods comprising: obtaining image features of a first image and a second image, wherein the image features include texture components and structural features, the first image being an image of a stereoscopic sand dune taken by a walker from a first perspective, and the second image being an image of a stereoscopic sand dune taken by the walker from a second perspective; performing feature extraction on the texture components of the first image and the texture components of the second image, respectively, and then performing feature matching to obtain a first matching point group; performing dense matching and forward intersection based on the first matching point group to obtain a three-dimensional point cloud of the stereoscopic sand dunes in the first field of view of the walker in three-dimensional coordinates; and interpolating the three-dimensional point cloud to obtain a digital elevation model.

[0174] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0175] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0176] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing image data, characterized in that: include: Acquire image features of a first image and a second image, wherein the image features include texture components and structural features, the first image is an image of a three-dimensional sand dune taken by a walker at a first viewing angle, and the second image is an image of a three-dimensional sand dune taken by the walker at a second viewing angle; Extracting features from the texture components of the first image and the texture components of the second image respectively and then performing feature matching to obtain a first matching point group; Based on the first matching point group, dense matching and forward intersection are performed to obtain a three-dimensional point cloud of the three-dimensional sand dune in the first field of view of the walker in three-dimensional coordinates; Performing interpolation processing on the three-dimensional point cloud to obtain a digital elevation model; The dense matching and forward intersection based on the first matching point group includes: After constructing a triangulated network using the first matching point group, obtaining a feature vector consisting of a multi-window structure tensor of the to-be-matched points in the first matching point group; Acquire the angles between the feature vectors of different points to be matched according to the feature vectors; Densely matching the points to be matched according to the angle to obtain dense matching points; A forward intersection is performed based on the dense matching points and the internal and external orientation elements of the image to obtain a three-dimensional point cloud under the three-dimensional coordinates.

2. The method according to claim 1, characterized in that: Before acquiring the image features of the first image and the second image, the method further includes: Texture enhancement is performed on the first image and the second image respectively to obtain corresponding texture components and structural features.

3. The method according to claim 1, characterized in that The extracting features of the texture components of the first image and the texture components of the second image and then performing feature matching comprises: The ASIFT algorithm is used to extract features from the texture components of the first image and the second image respectively, and then feature matching is performed.

4. The method according to claim 1, characterized in that The extracting features of the texture components of the first image and the texture components of the second image and then performing feature matching comprises: Extracting features from the texture components of the first image and the texture components of the second image respectively and then performing feature matching to obtain an initial matching point group; The RANSAC algorithm is used to eliminate gross errors of erroneous matching points in the initial matching point group to obtain the first matching point group.

5. The method according to claim 1, characterized in that: Before performing interpolation processing on the three-dimensional point cloud, the method further includes: The three-dimensional point cloud is subjected to gross error elimination.

6. The method according to claim 2, characterized in that Before performing texture enhancement on the first image and the second image respectively, the method further includes: Reading the stereoscopic dune image taken by the walker, wherein the stereoscopic dune image includes the stereoscopic images taken by the walker at various viewing angles and the internal and external orientation elements of a single image; The first image and the second image are acquired according to the stereoscopic sand dune image.

7. An image data processing device, characterized in that: include: an acquisition unit, configured to acquire image features of a first image and a second image, wherein the image features include texture components and structural features, the first image is an image of a three-dimensional sand dune taken by the walker at a first viewing angle, and the second image is an image of a three-dimensional sand dune taken by the walker at a second viewing angle; A feature matching unit, configured to extract features from the texture components of the first image and the texture components of the second image and then perform feature matching to obtain a first matching point group; A dense matching unit, configured to perform dense matching and forward intersection based on the first matching point group to obtain a three-dimensional point cloud of the three-dimensional sand dune in the first field of view of the walker in three-dimensional coordinates; An interpolation unit, used for performing interpolation processing on the three-dimensional point cloud to obtain a digital elevation model; The dense matching and forward intersection based on the first matching point group includes: After constructing a triangulated network using the first matching point group, obtaining a feature vector consisting of a multi-window structure tensor of the to-be-matched points in the first matching point group; Acquire the angles between the feature vectors of different points to be matched according to the feature vectors; Densely matching the points to be matched according to the angle to obtain dense matching points; A forward intersection is performed based on the dense matching points and the internal and external orientation elements of the image to obtain a three-dimensional point cloud under the three-dimensional coordinates.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images