A method for generating training samples for remote sensing image target recognition based on grid parameterization

Abstract

A method for generating training samples for remote sensing image target recognition based on grid parameterization is proposed. For a given original image, a grid of corresponding size is constructed according to its size and the grid is triangulated to obtain an initial triangular grid M. The initial triangular grid M is parameterized to obtain a parameterized grid G'. After the parameterized grid G' is normalized, two fixed points γ are selected on it. 1 and γ 2 , through γ 1 and γ 2 The corresponding Möbius transformation is obtained by inverse deduction, and then the parameterized mesh G' is subjected to the Möbius transformation and the transformation result is normalized. Finally, according to the rules of the real-time rendering pipeline, the mesh G' obtained after the Möbius transformation is used as the parameterized coordinate, and the original image is used as the texture. The texture is pasted back to the initial triangular mesh M to obtain the augmented image.

Description

Technical Field

[0001] The invention relates to a method for generating training samples for remote sensing image target recognition based on grid parameterization and Mobius transformation, and belongs to the technical field of computational geometry and deep learning. Background Art

[0002] In the task of remote sensing image target recognition, deep learning methods are usually used to detect targets such as ships, airplanes, and cars. In this process, the parameter training of the entire network often requires massive data as training samples, which is costly.

[0003] Generally, when there is less training data, data augmentation is a common method to expand it and generate new samples. Its purpose is to increase the richness of the training data set and make the training data as diverse as possible, so that the trained network model has stronger generalization ability. According to the timing of processing data, data augmentation can be divided into offline augmentation and online augmentation. Offline augmentation is suitable for smaller data sets, so that even if the amount of data increases several times, the hardware can withstand it. Online augmentation is suitable for larger data sets. Since the hardware often cannot withstand the explosive growth of the capacity of such data sets, only a small batch of data input to the model is augmented each time. Currently commonly used data augmentation methods include: horizontal or vertical flipping, rotation, scaling, shearing, translation, cropping, adjusting contrast, color jittering or adding noise, etc.

[0004] Mesh parameterization refers to the one-to-one mapping between a manifold (surface mesh or volume mesh) embedded in three-dimensional space and a simpler and more regular parameter domain. Its purpose is to transfer the processing operations on the complex manifold to the simple parameter domain, thereby improving the feasibility and efficiency of the operation. Mesh parameterization is one of the core issues studied in computer graphics, computer-aided geometric design, and digital geometry processing. In the past nearly two decades, it has become a ubiquitous basic tool and is widely used in the processing of three-dimensional mesh models, including texture mapping, detail conversion, mesh deformation, mesh editing, mesh re-sectioning, mesh compression, mesh repair, surface fitting, surface deformation, shape analysis, etc.

[0005] In computational geometry, the classic Möbius inversion formula was proposed by August Ferdinand Introduced in 1832. Each Möbius transformation is a one-to-one conformal transformation from the Riemann sphere to itself, that is, every straight line is mapped to a straight line or a circle, and every circle is mapped to a straight line or a circle. Generally speaking, for a mapping from one surface to another, if it maintains the similarity of small areas everywhere, the mapping is called a conformal transformation. The Möbius transformation is a conformal transformation, which is characterized by keeping the local shape unchanged, so that the shapes of each object and target in the new training sample generated by the present invention will not change compared with the original sample. Summary of the invention

[0006] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art and provide a method for generating remote sensing image target recognition training samples based on grid parameterization, which realizes the generation of remote sensing image target recognition training samples by using grid parameterization in computational geometry and Mobius transformation. The training samples can augment the background geographic image information without changing the shapes of recognition targets such as ships, airplanes and cars.

[0007] The technical solution of the present invention is:

[0008] A method for generating training samples for remote sensing image target recognition based on grid parameterization includes the following steps:

[0009] Step 1: Construct a grid of corresponding size in proportion to the size of the original image, and triangulate it to obtain an initial triangular grid M;

[0010] Step 2: Parameterize the initial triangular mesh M to obtain [0,1] 2 The parameterized grid G ​​on

[0011] Step 3: Select two fixed points γ1 and γ2 on the parameterized grid G, and adjust the positions of the fixed points to the two fixed points (0,0) and (1,0) on the circumference;

[0012] Step 4: Use γ1 and γ2 to inversely obtain the four coefficients a, b, c and d corresponding to the Möbius transformation;

[0013] Step 5: Keep the original triangular mesh topology unchanged and perform a Möbius transformation on each vertex of the parameterized mesh G to obtain the mesh G';

[0014] Step 6: Normalize the mesh G' obtained after the Möbius transformation so that each vertex v on the mesh G' i ∈[0,1] 2 ;

[0015] Step 7: Use the mesh G' obtained after the Möbius transformation as the parameterized coordinates and the original image as the texture. According to the rules of the real-time rendering pipeline, the texture is pasted onto the initial triangular mesh M to obtain the augmented image, thus completing the generation of remote sensing image target recognition training samples based on mesh parameterization.

[0016] Furthermore, the step 1 constructs a grid mesh of corresponding size in proportion to the size of the original image, specifically: the size of the grid is smaller than the size of the original image, and the ratio is between 1:2 and 1:4, and the size of the grid is the number of horizontal and vertical grids; after obtaining the grid mesh, the upper left corner and the lower right corner of each square are connected to obtain the initial triangular mesh M.

[0017] Furthermore, in step 2, the initial triangular mesh M is parameterized to obtain [0,1] 2 The parameterized grid G ​​on is:

[0018] Each vertex on the initial triangle mesh M is Normalize to [0,1] 2 And discard the coordinate components perpendicular to the mesh plane to obtain the parameterized mesh G of the initial triangular mesh M; where v x is the x-coordinate value of vertex V, x min is the minimum x-coordinate of all vertices, x max is the maximum x-coordinate of all vertices.

[0019] Furthermore, in step 4, four coefficients a, b, c and d corresponding to the Mobius transformation are obtained by inversely calculating γ1 and γ2, which are specifically:

[0020] For a given Möbius transformation Substituting γ1 and γ2 into the equation f(γ) = γ, we obtain Where γ is a synonym for γ1 and γ2, and the equation is equivalent to the standard hyperbolic equation cγz-az+dγ-b=0 on the zγ plane;

[0021] Take the midpoint of γ1 and γ2 As the third fixed point, the process of inverse Möbius transformation is equivalent to finding the four coefficients a, b, c and d of the mapping f(z) from the triple (z1, z2, z3) to (γ1, γ2, γ3). This process is equivalent to finding the point (z i ,γ i ) where i = 1, 2, 3; the hyperbolic equation is obtained by expanding the first row of the following matrix using the Lagrange expansion method

[0022]

[0023] The value of each coefficient is equal to the value of each co-factor determinant after the matrix is ​​expanded

[0024]

[0025] Furthermore, in step 6, the vertices of the mesh G' obtained after the Möbius transformation are transformed according to The way of unitization makes every vertex v on G' i ∈[0,1] 2 .

[0026] Furthermore, in real-time rendering, let the vertical direction be the y-axis direction, the horizontal direction be the x-axis direction, and the aspect ratio be r. In this case, y∈[-1,1], x∈[-r,r]. Orthogonal projection is used for rendering. The projection matrix used by orthogonal projection is as follows:

[0027]

[0028] The values ​​of the near plane n and the far plane f only need to ensure that z∈(n,f) of all vertices of the mesh; finally, the image data of the generated sample is obtained by grabbing the frame buffer or rendering to the texture offline and reading it back.

[0029] The beneficial effects of the present invention compared with the prior art are:

[0030] (1) The present invention provides a method for generating training samples for remote sensing image target recognition based on mesh parameterization and Mobius transformation. For a given original image, a grid mesh of corresponding size is constructed in proportion to its size and the mesh is triangulated to obtain an initial triangular mesh M, and the initial triangular mesh M is parameterized to obtain a parameterized mesh G', and after the parameterized mesh G' is normalized, two fixed points γ1 and γ2 are selected on it, and the corresponding Mobius transformation is obtained by reverse deduction through γ1 and γ2, and then the parameterized mesh G' is subjected to Mobius transformation and the result of the transformation is normalized, and finally, according to the rules of the real-time rendering pipeline, the mesh G' obtained after the Mobius transformation is used as the parameterized coordinate, and the original image is used as the texture, and the texture is pasted back to the initial triangular mesh M, and the augmented image can be obtained. In the whole process, only the reverse deduction of the Mobius transformation, the transformation of the mesh G', and the texture pasting need to be repeatedly calculated.

[0031] (2) In the above process, the inverse of the Möbius transform only requires solving one equation, and the amount of calculation is very small; the transformation of the grid G' and the texture back-pasting can be completely parallelized, and the overall computing efficiency is very high and the running speed is fast. It can also achieve a high running speed for tasks that generate a large number of samples.

[0032] (2) The selection of fixed γ1 and γ2 is very free, and the number of permutations and combinations is very large, which ensures the diversity and richness of the generated samples. On the other hand, the entire sample generation process can be completed automatically by the program, which has high robustness and practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of the overall implementation steps of the present invention.

[0034] Figure 2 1 is an example diagram of an initial triangular mesh M obtained after triangulation of a grid mesh used in the implementation process of the present invention, a parameterized mesh G of M, and a mesh G' obtained after G is subjected to a Mobius transformation.

[0035] Figure 3 A sample graph of port samples generated for the present invention, wherein subgraph (1) is the original sample, and subgraphs (2) to (6) are new samples generated using different fixed points γ1 and γ2.

[0036] Figure 4 A sample graph of port and ocean samples is generated for the present invention, wherein subgraph (1) is the original sample, and subgraphs (2) to (6) are new samples generated using different fixed points γ1 and γ2.

[0037] Figure 5 A sample graph of offshore target samples is generated for the present invention, wherein sub-graph (1) is the original sample, and sub-graphs (2) to (6) are new samples generated using different fixed points γ1 and γ2.

[0038] Figure 6 A sample map of land samples generated for the present invention, wherein sub-map (1) is the original sample, and sub-maps (2) to (6) are new samples generated using different fixed points γ1 and γ2. DETAILED DESCRIPTION

[0039] The specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0040] In order to generate training samples for remote sensing image target recognition, the present invention provides a training sample enhancement method based on grid parameterization and Mobius transformation to generate additional training data for deep learning network training. The present invention can mathematically ensure the conformality of the transformation between the augmented sample and the recognition target in the original sample, that is, the shape of the object does not change significantly, and can significantly transform the map geographic image information as the background, thereby improving the richness and diversity of the training samples.

[0041] In geometry and complex analysis, the Möbius transformation on the complex plane can be written as a rational function The form of, where z is a complex variable, a, b, c and d are four complex coefficients, satisfying the condition ad-bc≠0. The Möbius transformation can also be expressed in the form of a matrix Similarly, the inverse transformation of the Möbius transformation can be obtained by using the inverse matrix of the original transformation matrix Although a Möbius transformation can be uniquely determined when the coefficients a, b, c and d are given and the value of the matrix corresponding to the determinant ad-bc≠0, the transformation is not yet injective. This is because multiple transformations can be made to have the same result by multiplying the matrix by a non-zero coefficient. This problem can be solved by normalizing the Möbius transformation, and ad-bc=1 after normalization.

[0042] In intuitive geometric form, the Möbius transformation can be obtained by first performing a stereoscopic projection from the plane to the unit 2D sphere, then rotating and moving the sphere to a new position and orientation in space, and then stereoscopically projecting the sphere back onto the plane from its new position. When c≠0, the above transformation can be written as: in (translation transform), (Do the inverse transformation with the unit circle and then do the flip transformation with the real axis as the axis), (similarity and rotation transformations centered at the origin), (Translation transformation). Through this decomposition we can obtain many basic properties of the Möbius transformation.

[0043] The existence and expression of the inverse transformation of the Möbius transformation can be obtained by combining the inverse functions of the four functions obtained by decomposing the transformation. Since each transformation mentioned in the above decomposition is reversible and its inverse transformation is very easy to obtain, the inverse transformation of the Möbius transformation is also a Möbius transformation. That is, for each f i , calculate its inverse function By combining these inverse functions, we can get the inverse Möbius transformation From the decomposition of the Mobius transform, it can also be seen that the Mobius transform has almost all the non-trivial properties of maintaining the inverse transformation. Since the angle remains unchanged during the translation, inversion, similarity and rotation transformation, the difference in amplitude between two complex numbers can still remain unchanged after the Mobius transform. A generalized circle will still be mapped to a generalized circle after the Mobius transform. The generalized circle is different from the ordinary circle. It refers to the circle on the Riemann sphere. It includes ordinary circles and straight lines with infinite breakpoints, the latter of which can also be regarded as a circle with infinite radius. The result of inversion maintaining the generalized circle makes the Mobius transform not intuitively mapping circles to circles and straight lines to straight lines. The transformation makes it possible for a straight line to become a circle, and a circle to become a straight line. The above characteristics mathematically guarantee the conformality of the transformation between the enhanced sample generated by the present invention and the original sample.

[0044] For each non-unitized Möbius transformation, there exist two fixed points γ1 and γ2, which are coincident and one or both of which can be infinitely far away. The fixed points can be obtained by solving the equation f(γ) = γ. When c≠0, the roots corresponding to the two fixed points can be obtained by solving the quadratic equation cγ 2 -(ad)γ-b=0. When c=0, the above quadratic equation degenerates into a general linear equation, and the original transformation also degenerates into a linear transformation. Geometrically, this is equivalent to one of the fixed points being at an infinite distance. When a≠d, the other fixed point is At this time, the original transformation is equivalent to a combination of translation, rotation and expansion transformation. When c = 0 and a = d are satisfied at the same time, the two fixed points are located at infinite distances.

[0045] The present invention proposes a method for generating training samples for remote sensing image target recognition based on grid parameterization. The overall flow chart can be seen in the attached figure. Figure 1 , the basic implementation steps are as follows:

[0046] Step 1: Construct a grid of the corresponding size in proportion to the size of the original image. The size of the grid (the number of horizontal and vertical grids) should be smaller than the size of the original image, and the ratio is generally between 1:2 and 1:4. In order to facilitate the implementation of subsequent steps, the height of the grid is set to 2, and the ratio of the length and width of the entire grid should be equal to the ratio of the length and width of the original image. After obtaining the grid, connect the upper left corner and the lower right corner of each square to obtain the initial triangular grid M. The constructed triangular grid is shown in the attached figure. Figure 2 shown.

[0047] Step 2: Place each vertex on the initial triangle mesh according to Normalize to [0,1]2 By discarding the coordinate components perpendicular to the mesh plane, the parameterized mesh G of the initial triangular mesh M can be obtained.

[0048] Step 3: Select two fixed points γ1 and γ2 on the parameterized mesh G, adjust the positions of the fixed points to the two fixed points (0,0) and (1,0) on the circumference and keep the topological structure of the entire triangular mesh unchanged.

[0049] Step 4: Use γ1 and γ2 to infer the four coefficients a, b, c and d corresponding to the Möbius transformation. For a given Möbius transformation We can substitute γ1 and γ2 into the equation f(γ) = γ to obtain This equation is equivalent to the standard hyperbolic equation cγz-az+dγ-b=0 on the zγ plane. Note that we only selected two fixed points, γ1 and γ2, and at least three unknowns are required to solve the coefficients a, b, c, and d, so the midpoint of γ1 and γ2 is taken here. As the third fixed point (the method of deriving the third control point can be adjusted according to the situation). At this time, the process of inversely deducing the Möbius transform is equivalent to finding the four coefficients a, b, c and d of the mapping f(z) from the triple (z1, z2, z3) to (γ1, γ2, γ3). This process is equivalent to finding the point (z i ,γ i ) is the four coefficients a, b, c and d of the equation of the hyperboloid, where i = 1, 2, 3. Since the hyperbola equation at this time can be obtained by expanding the first row of the following matrix using the Lagrange expansion method

[0050]

[0051] According to the principle that coefficients of similar items are the same, the values ​​of each coefficient should be equal to the values ​​of the determinants of each cofactor after the matrix is ​​expanded.

[0052]

[0053] Step 5: Perform a Möbius transformation on each vertex of the parameterized mesh G The positions of the transformed vertices are obtained, and then a triangular mesh is constructed according to the connection method of the vertices of the mesh G to obtain the transformed parameterized mesh G'.

[0054] Step 6: The vertices of the mesh G' obtained after the Möbius transformation are The way of unitization makes every vertex v on G' i ∈[0,1] 2 .

[0055] Step 7: Use the initial triangular mesh M as the position coordinates, the vertices of the mesh G' obtained after the Mobius transformation as the texture coordinates, and the original image as the input texture, and submit it to the real-time rendering pipeline for rendering. During the rendering process, place the triangular mesh M at any position perpendicular to the camera.

[0056] Assume that the vertical direction is the y-axis direction, the horizontal direction is the x-axis direction, and the aspect ratio is r. Since the height of the grid is 2, y∈[-1,1], x∈[-r,r]. In order to avoid perspective distortion and make the grid M fill the entire viewport, orthogonal projection is used for rendering. The projection matrix used by orthogonal projection is as follows

[0057]

[0058] The values ​​of the near plane n and the far plane f are relatively arbitrary, as long as z∈(n,f) is guaranteed for all vertices of the mesh. Finally, the image data of the generated sample is obtained by grabbing the frame buffer or rendering offline to a texture and reading it back.

[0059] The above method of selecting fixed points is to manually select the first two fixed points first, and then obtain the third fixed point by calculating the middle point. However, it can be seen from the relevant description of the Mobius transformation in the content of the invention that the method of selecting fixed points is not unique and can be changed according to actual needs. The examples shown in the accompanying drawings prove that the present invention can mathematically guarantee the conformality of the transformation between the recognition target in the enhanced sample and the original sample, that is, the shape of the object does not change significantly, and can make a significant transformation of the map geographic image information as the background, thereby improving the richness and diversity of the training samples. The relevant results generated by the algorithm are shown in the attached figure. Figure 3 , Attachment Figure 4 , Attachment Figure 5 and attached Figure 6 Although the present invention has been described in detail through the above preferred embodiments, it should be understood by those skilled in the art that various changes may be made thereto in form and detail without departing from the scope of the present invention defined in the claims.

[0060] Parts of the present invention that are not described in detail belong to common knowledge among those skilled in the art.

Claims

1. A method for generating training samples for remote sensing image target recognition based on grid parameterization, characterized in that The steps include: Step 1: Construct a grid of corresponding size in proportion to the size of the original image, and triangulate it to obtain an initial triangular grid M; Step 2: Parameterize the initial triangular mesh M to obtain [0,1] 2 The parameterized grid G ​​on Step 3: Select two fixed points γ1 and γ2 on the parameterized grid G, and adjust the positions of the fixed points to the two fixed points (0,0) and (1,0) on the circumference; Step 4: Use γ1 and γ2 to inversely obtain the four coefficients a, b, c and d corresponding to the Möbius transformation; Step 5: Keep the original triangular mesh topology unchanged and perform a Möbius transformation on each vertex of the parameterized mesh G to obtain the mesh G'; Step 6: Normalize the mesh G' obtained after the Möbius transformation so that each vertex v on the mesh G' i ∈[0,1] 2 ; Step 7: Use the grid G' obtained after the Mobius transformation as the parameterized coordinates and the original image as the texture. According to the rules of the real-time rendering pipeline, the texture is pasted onto the initial triangular grid M to obtain the augmented image, thus completing the generation of remote sensing image target recognition training samples based on grid parameterization. The second step is to parameterize the initial triangular mesh M to obtain [0,1] 2 The parameterized grid G ​​on is: Each vertex on the initial triangle mesh M is Normalize to [0,1] 2 And discard the coordinate components perpendicular to the mesh plane to obtain the parameterized mesh G of the initial triangular mesh M; Among them, v x is the x-coordinate value of vertex V, x min is the minimum x-coordinate of all vertices, x max is the maximum value of the x coordinates of all vertices; In step 4, the four coefficients a, b, c and d corresponding to the Mobius transformation are obtained by inverse deduction through γ1 and γ2, which are specifically: For a given Möbius transformation Substituting γ1 and γ2 into the equation f(γ) = γ, we obtain Where γ is a synonym for γ1 and γ2, and the equation is equivalent to the standard hyperbolic equation cγz-az+dγ-b=0 on the zγ plane; Take the midpoint of γ1 and γ2 As the third fixed point, the process of inverse Möbius transformation is equivalent to finding the four coefficients a, b, c and d of the mapping f(z) from the triple (z1, z2, z3) to (γ1, γ2, γ3). This process is equivalent to finding the point (z i ,γ i ) where i = 1, 2, 3; the hyperbolic equation is obtained by expanding the first row of the following matrix using the Lagrange expansion method The value of each coefficient is equal to the value of each co-factor determinant after the matrix is ​​expanded 2. The method for generating training samples for remote sensing image target recognition based on grid parameterization according to claim 1, characterized in that: The step 1 constructs a grid mesh of corresponding size in proportion to the size of the original image, specifically: the size of the grid is smaller than the size of the original image, and the ratio is between 1:2 and 1:4, and the size of the grid is the number of horizontal and vertical grids; after obtaining the grid mesh, the upper left corner and the lower right corner of each square are connected to obtain the initial triangular mesh M.

3. The method for generating training samples for remote sensing image target recognition based on grid parameterization according to claim 1, characterized in that: Step 6. The vertices of the mesh G' obtained after the Möbius transformation are The way of unitization makes every vertex v on G' i ∈[0,1] 2 .

4. The method for generating training samples for remote sensing image target recognition based on grid parameterization according to claim 1, characterized in that: In real-time rendering, let the vertical direction be the y-axis direction, the horizontal direction be the x-axis direction, and the aspect ratio be r. In this case, y∈[-1,1], x∈[-r,r]. Orthogonal projection is used for rendering. The projection matrix used by orthogonal projection is as follows: The values ​​of the near plane n and the far plane f only need to ensure that z∈(n,f) of all vertices of the mesh; finally, the image data of the generated sample is obtained by grabbing the frame buffer or rendering to the texture offline and reading it back.

Images