A method and system for high resolution 3D reconstruction of offshore areas

Abstract

The application discloses a kind of offshore area high-resolution three-dimensional reconstruction method and system, including using perspective size invariant feature to match the multi-view angle image of offshore area, according to the feature similarity guide sampling, obtain sparse feature point cloud;Based on structured local binary pattern, the depth of each pixel is obtained using graph cut method, and the sparse feature point cloud is encrypted to generate dense point cloud;Global error optimization is carried out to the point cloud of all view angles, and according to the depth information of the matching feature points, the three-dimensional coordinates of the feature points in space are generated;Strong and weak connection relationship between plane primitives is constructed, and the surface model is obtained by graph structure and energy function.The application can obtain wide area large range panoramic high-resolution information of sea area, construct three-dimensional model, and can intuitively display the space-time big data of offshore area.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a high-resolution three-dimensional reconstruction method and system for offshore areas. BACKGROUND

[0002] Offshore areas are rich in natural resources, have superior geographical location and pleasant living environment, and are currently the most economically developed and most densely populated areas. Human economic activities have disturbed the normal structure and function of the ecological system of offshore areas, leading to changes and even degradation of the ecological system. A series of environmental problems such as artificialization of natural landscape, reduction of vegetation coverage, destruction of shelter belts, and deterioration of water quality in nearshore areas threaten the ecological security of the region.

[0003] Current geographic spatial application service systems for offshore areas lack data processing and decision-making systems based on large-scale real three-dimensional scenes and environmental intelligent perception. The mainstream systems only statically display images of offshore areas and coastal zones, and are difficult to intelligently analyze events occurring in offshore areas and activities occurring in coastal zones. SUMMARY

[0004] In order to solve the problem in the prior art that the mainstream systems only statically display images of offshore areas and coastal zones, and are difficult to intelligently analyze events occurring in offshore areas and activities occurring in coastal zones, and lack large-scale real three-dimensional scenes, the present application provides a high-resolution three-dimensional reconstruction method and system for offshore areas to solve the above technical problems.

[0005] According to one aspect of the present application, a high-resolution three-dimensional reconstruction method for offshore areas is provided, comprising:

[0006] S1: matching multi-view images of offshore areas using perspective size-invariant features, and obtaining a sparse feature point cloud according to feature similarity guided sampling;

[0007] S2: obtaining the depth of each pixel using graph cut based on structured local binary patterns, and generating a dense point cloud by encrypting the sparse feature point cloud;

[0008] S3: performing global error optimization on point clouds of all views, and generating three-dimensional coordinates of feature points in space according to depth information of the matched feature points; and

[0009] S4: constructing strong and weak connection relationships between plane primitives, and obtaining a surface model through a graph structure and an energy function.

[0010] In some specific embodiments, the feature similarity guided sampling in step S1 comprises:

[0011] calculating N sets of multi-view offshore image feature matching pairs Residual sequence of transform model set wherein represents the jth smallest model of the residual in the ith matching group;

[0012] The relaxation area expansion window scanning range is set, the model inverse sequence is calculated by using the arrangement relationship in the observation window, the similarity function is constrained, and the similarity function wherein, h is the window length, ε is the relaxation area, β(h) is a penalty factor related to the window length h, is the inverse sequence of the overlapping model arrangement in the window, and A and B are the weights of the two models of the variable window and the inverse sequence model respectively. By virtue of the structure, the structure estimation speed can be accelerated and the estimation reliability can be improved.

[0013] In some specific embodiments, the graph cut method in step S2 acquires the pixel depth by using the following optimization function wherein, is a data item fused with the structural local binary pattern pixel template similarity, is a smoothing item for constraining the neighborhood depth information. By virtue of the optimization function, more accurate dense point cloud spatial coordinates can be obtained.

[0014] In some specific embodiments, the generation of the dense point cloud in step S2 includes: solving the depth information of all pixel points, inversely calculating the pixel corresponding relationship by using the geometric principle of computer vision, and generating the dense point cloud.

[0015] In some specific embodiments, step S3 further includes a curvature filtering distortion correction technology, and curvature information is generated by using adjacent data to smooth the point cloud data. The filtered data can avoid excessive smoothing, and has good fidelity to the edge while eliminating the distortion signal.

[0016] In some specific embodiments, step S4 includes:

[0017] S41: detecting to generate an initial primitive based on a maximum tolerance distance of a maximum distance of fitted local points to a plane primitive and a minimum set number of a minimum number of local points fitted into a plane primitive;

[0018] S42: sequentially constructing a strong connection graph and a soft connection graph according to different internal topological relationships between the primitives;

[0019] S43: generating all candidate face sets and acquiring a three-dimensional surface model by using the graph structure on the basis of the soft connection graph, and optimizing and constraining the three-dimensional surface model according to an energy function and a constraint.

[0020] In some specific embodiments, the energy function and the constraint are min x λ f E f +λ c E c +λ m E m , wherein E f is an energy term based on surface coverage, the larger the area of the selected planar convex polygon coverage is, the smaller the energy is; E c is an energy term based on data fitting, the more the triangles in the original three-dimensional model that are covered by the selected planar polygon, the smaller the energy is; E m is an energy term based on model complexity, the fewer the edges in the final model, the smaller the energy is; the constraint indicates that each edge of the result must connect two planes, N is the total number of the candidate polygon plane set, |E| is the total number of the edges of the generated model; the value of the indicator variable x i is 0 when the ith plane is not selected, and is 1 when the ith plane is selected. By virtue of this step, a surface model with more topological logic and manifold watertight properties can be obtained.

[0021] In some specific embodiments, the strong connection graph and the soft connection graph in step S42 include:

[0022] Traverse the elementary local points, perform spherical neighborhood search on each local point, and in response to at least one local point in the element finding other elementary local points through spherical neighborhood search, the two elements have a strong connection relationship, and a strong connection graph is constructed;

[0023] Using a plane shape growing algorithm, the local point set of each element is projected to a fitting plane to obtain a minimum convex polygon of the two-dimensional point set, based on which a multi-scale region of basic scale, stretchable tolerance scale and detection scale and its corresponding confidence are determined, the confidence sum of the intersection points of the element planes is obtained, and in response to the confidence sum of the intersection points being greater than a preset threshold, a soft connection edge is added on the basis of the strong connection graph to construct a soft connection graph.

[0024] According to a second aspect of the present application, a computer readable storage medium is provided, which stores one or more computer programs, and the one or more computer programs are executed by a computer processor to implement the above method.

[0025] According to a third aspect of the present application, a high-resolution three-dimensional reconstruction system for offshore areas is provided, which comprises:

[0026] An image matching unit configured to match multi-view angle images of offshore areas using perspective size invariant features, and to obtain a sparse feature point cloud according to feature similarity guided sampling;

[0027] The dense point cloud generating unit is configured to generate a dense point cloud by using a graph cut method based on a structured local binary pattern and obtaining the depth of each pixel;

[0028] The registration unit is configured to perform global error optimization on the point clouds of all views and generate three-dimensional coordinates of the feature points in space according to the depth information of the matched feature points;

[0029] The three-dimensional reconstruction unit is configured to construct a strong and weak connection relationship between plane primitives and obtain a surface model through a graph structure and an energy function.

[0030] The present application proposes a high-resolution three-dimensional reconstruction method for offshore areas, uses a perspective scale invariant feature to ensure the stability of multi-view image matching, introduces a graph cut method for multi-view stereo matching based on a sparse feature point cloud, and improves the accuracy of dense point cloud generation; the research trend of signal smoothing is introduced, curvature information of adjacent data is fully considered through a curvature filtering strategy, and the data is smoothed based on the curvature information; the filtered data can avoid excessive smoothing, and has good fidelity to the edges while eliminating distorted signals. By constructing a strong and weak connection relationship between plane primitives, the missing point cloud caused by occlusion and other phenomena is solved, and through the design of a closed constraint energy function in the model generation process, a surface model with more topological logic and manifold watertight characteristics is obtained, which can enhance the visualization effect of the three-dimensional model under the condition of reducing the resolution. BRIEF DESCRIPTION OF DRAWINGS

[0031] The accompanying drawings are included to provide a further understanding of embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and serve to explain principles of the application. Other embodiments and many of the intended advantages of the present application will be readily appreciated as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings. Other features, objects, and advantages of the application will become apparent from the detailed description taken in connection with the accompanying drawings:

[0032] Figure 1 is a flow chart of a high-resolution three-dimensional reconstruction method for offshore areas according to an embodiment of the present application;

[0033] Figure 2a -c is a plane shape growing flow chart according to a specific embodiment of the present application;

[0034] Figure 3 is a multi-scale confidence generation schematic diagram according to an embodiment of the present application;

[0035] Figure 4 is a soft connection graph vertex generation schematic diagram according to a specific embodiment of the present application;

[0036] Figure 5a -b is the case of three-plane intersection according to an embodiment of the present application;

[0037] Figure 6 is a framework diagram of a high-resolution three-dimensional reconstruction system for offshore areas according to an embodiment of the present application;

[0038] Figure 7 is a structural schematic diagram of a computer system of an electronic device suitable for implementing embodiments of the present application. DETAILED DESCRIPTION

[0039] The present application will be further described below in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related application, and are not a limitation on the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for ease of description.

[0040] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0041] An image-based eye movement tracking method according to an embodiment of the present application, Figure 1 A flowchart of a high-resolution three-dimensional reconstruction method for offshore areas according to an embodiment of the present application is shown. As shown in Figure 1 The method comprises:

[0042] S101: Match the multi-view angle images of the offshore area using perspective size invariant features, guide sampling according to feature similarity, and obtain sparse feature point clouds. According to the perspective projection principle of computer vision, a multi-angle observation model of images is proposed, the image deformation phenomenon caused by changes in viewing angle is simulated in a perspective transformation sampling manner, and the matching stability of different viewing angles is effectively improved by using perspective size invariant features.

[0043] In specific embodiments, based on the estimation strategy of guided sampling, the structure estimation speed can be accelerated and the estimation reliability can be improved. Calculate N sets of multi-view offshore image feature matching pairs For the residual sequence of the transformation model set Wherein The model represents the jth smallest residual in the ith group of matches. The similarity function between feature pairs is evaluated by the residual sequence to guide the sampling process. According to the characteristics of offshore image matching, an adaptive window length adjustment strategy is adopted: a relaxation area is set to expand the window scanning range, so that the similarity function can more objectively reflect the possibility of different features coming from the same model; the inverse order of the arrangement within the observation window is calculated to constrain the similarity function. The similarity function combining variable window and inverse order model is as follows: where h is the window length, ε is the relaxation area, β(h) is the penalty factor related to the window length h, is the inverse order of the arrangement of overlapping models within the window, and A and B are the weights of the variable window and inverse order models, respectively.

[0044] In specific embodiments, sampling is guided according to feature similarity, and the sampling process is based on the Bayesian principle. The selection of subsequent feature matching points depends on the probability product between the selected points. The point set obtained according to the conditional probability generates a new transformation model. The new model can continuously add to the original model set, and the feature similarity function is also updated accordingly. Based on feature matching and robust estimation, information such as GPS and rotation angle parameters obtained by sensors corresponding to the image can be integrated into the adjustment process to construct a global optimization function for geometric constraints of matching features, thereby improving the estimation accuracy of camera parameters and the positioning accuracy of sparse feature point clouds.

[0045] S102: Based on the structured local binary pattern, the depth of each pixel is obtained using the graph cut method, and the sparse feature point cloud is encrypted to generate a dense point cloud. Sparse feature point clouds are not sufficient as a basis for offshore point cloud fitting, so the point cloud needs to be encrypted as described above.

[0046] In specific embodiments, to improve the accuracy of the dense point cloud, the structured local binary pattern is used according to the texture characteristics of the image, which can effectively describe the subtle changes in texture and has good robustness to noise and illumination. A histogram is constructed based on the structured local binary of each pixel within the window as a local descriptor of the template. Based on the sparse feature point cloud, the graph cut method is used to solve the optimal value of the depth of each pixel, and the following optimization function is used: where, is the data item that combines the similarity of the structured local binary pattern pixel template, is a smooth term for constraining the neighborhood depth information. For the accuracy requirement of offshore area reconstruction, the depth information of the corresponding pixels of the existing matching feature pairs is embedded in the similarity data term, and the neighborhood smooth term is also constrained to some extent. The depth of all pixels is solved. Finally, the corresponding relationship of the pixels is calculated through the geometric principle of computer vision to generate a dense point cloud. By using this method, the problem of low signal-to-noise ratio of the generated point cloud caused by the similarity of the sea surface and beach texture in the offshore area and the unreliability of the traditional template matching of statistical correlation can be solved, and the accuracy of the dense point cloud can be improved through weak texture matching.

[0047] S103: Global error optimization is performed on the point clouds of all views, and three-dimensional coordinates of the feature points in space are generated according to the depth information of the matched feature points.

[0048] In specific embodiments, aerial high-resolution image information can be mapped into the point cloud generated by the depth map according to the calibration parameters. According to the scale and rotation invariance of the local features of the image, the three-dimensional coordinates of the feature points in space are generated according to the depth information of the matched feature points. In camera positioning and point cloud external parameter calculation, the constraints of point cloud feature matching and image feature matching are fully integrated to improve the accuracy of the camera external parameters and the accuracy of the point cloud fusion. This is achieved through the following formula: wherein x i represents the i-th point coordinate in the input point cloud, T(x i ) is the position of x i mapped to the coordinate system of the reference point cloud, y i is the coordinate of the nearest point to T(x i ), f j is the position of the j-th local feature in the point cloud corresponding to the image, f j is the spatial coordinate of the matching point of the image feature f j , and FPH(·) represents the feature descriptor of the point cloud. The local feature matching method can realize automatic registration of the point cloud.

[0049] In specific embodiments, marine virtual display needs to build a three-dimensional scene of the offshore area, and needs to fuse point cloud information of multiple views. If the matching method is used for each view, the transformation error will accumulate. Global error optimization is performed on the point clouds of all views, that is, the coordinate and feature vector error of all homonymous feature points are counted, the actual orientation information of each scanning point is solved, and the transformation parameters are extracted and the multi-view point cloud is fused by optimizing the overall matching error of the multi-view point cloud. The specific formula is as follows: wherein k∈1, 2, …, K is the view variable. T N is the transformation of mapping all point clouds to the same coordinate system.

[0050] In specific embodiments, a signal smoothing method can also be introduced, and the curvature information generated by adjacent data is used to smooth the data. The filtered data can avoid excessive smoothing and has good fidelity to the edges while eliminating distorted signals. Through an efficient image edge feature extraction method, a regularization term is constructed according to the edge information during the point cloud generation process, which is used to constrain the positioning accuracy of the point cloud. On this basis, a K-neighbor graph is constructed using the spatial information of the point cloud, different cut sets are defined, the implicit Gaussian curvature signal is analyzed, and noise points are filtered, thereby effectively improving the surface fine distortion problem of the three-dimensional model.

[0051] S104: Constructing strong and weak connection relationships between plane primitives, obtaining a surface model through a graph structure and an energy function. The source of offshore point cloud data includes air space radar scanning, unmanned aerial vehicle motion structure recovery, or multi-view geometry, etc. The visualization model of the original point cloud data generally has the problems of large storage capacity and occlusion loss. The construction of strong and weak connection relationships between plane primitives can effectively solve the point cloud loss caused by occlusion. By designing a closed constraint energy function, a surface model with topological logic and manifold watertight characteristics can be obtained, which can enhance the visualization effect of the three-dimensional model under the condition of reducing the resolution.

[0052] In specific embodiments, the reconstruction of the surface model includes the following steps: first, detecting the generation of the initial plane primitive, the initial primitive is generated by structure random sampling, and the input model includes plane, sphere, cylinder and cone, etc. In a preferred embodiment, the input of the algorithm of the present application is point cloud data or triangular mesh data with normal vector attributes. The point cloud data uses the RANSAC algorithm for plane primitive detection, and the triangular mesh data selects a region growing algorithm based on planarity. For the detection of the plane primitive, the maximum tolerance distance of the maximum distance of the fitted local points to the plane primitive and the minimum set number of the minimum number of local points fitted into a plane primitive are used.

[0053] Secondly, a strong connection graph is constructed according to the strong connection relationship. The purpose of the generation of the connection graph is to capture the internal topological relationship between the primitives from the original model and encode it in the undirected graph. For point cloud data, the strong connection relationship construction of a single primitive needs to traverse the local points of the primitive and perform a spherical neighborhood search on each local point. If at least one local point of the primitive can be searched to the local point of another primitive by the spherical neighborhood search, it is considered that the two primitives have a strong connection relationship; for triangular mesh data, the strong connection relationship construction of a single primitive is to traverse each triangular face of the primitive, and if at least one triangular face of the primitive is adjacent to the triangular face of another primitive, it is considered that the two primitives have a strong connection relationship. In some preferred embodiments, before constructing the strong connection relationship graph, some primitives with a number of local points less than a threshold value and some primitives with a merging normal vector angle less than a given threshold value and local points satisfying coplanar conditions can be removed by an optimization process to make the solving process more stable and remove certain noise.

[0054] In specific embodiments, on the basis of the strong connection graph, a soft connection graph is used to depict the relationship between the relationship primitives. A plane shape is generated by scale extension, for each set of local points of a primitive, the set is projected to the fitting plane, three-dimensional points are mapped to two-dimensional points, and the minimum convex polygon of the two-dimensional point set on the plane is calculated. For each vertex of the convex polygon, two attributes are assigned, including the straight line L from the barycenter to the vertex, and the circle C passing through the vertex with the barycenter as the center. By controlling the length of the radius of each circle relative to the initial radius to grow in proportion, the effect of growing the convex polygon of the model is achieved. The specific generation process is as shown in Figure 2a .

[0055] In specific embodiments, the stretching of the plane convex polygon is determined as three scales: basic scale, stretching tolerable scale, and detection scale, and different confidence levels are assigned to the three scales. The basic scale is defined as the highest confidence level 3, and is also the smallest convex polygon grown; the detection scale is defined as the lowest confidence level 1, and is also the largest convex polygon grown. The three scales correspond to three growth parameters, and the use of a unified parameter in the implementation process can make the three scales grow in the same proportion (the three scales can be defined as 1.1K, 1.4K, and 2K, and only one additional parameter K needs to be controlled to control the growth scale). In addition, the confidence level of the fitted primitive plane can be defined as 0. The specific multi-scale confidence level generation diagram is as shown in Figure 3 .

[0056] In specific embodiments, after each primitive calculates the multi-scale convex polygon, the intersection points of all primitive planes are first determined in the 3D bounding box of the original model. Each intersection point is generated by the intersection of three plane primitives, and the three planes in space are not parallel to each other. The three planes will intersect at a point (as shown inFigure 4 If all the planes are directly put into the energy function to solve, the solving time will increase exponentially with the number of planes. Therefore, the application uses the early calculation of intersection points to greatly reduce the solving time of the energy function. After obtaining all the intersection points in space, the confidence of each vertex can be calculated. The confidence calculation method is as follows: for an intersection point P and its three planes, project the three-dimensional point P onto the three two-dimensional planes respectively. According to which confidence convex polygon the two-dimensional point P' falls into, add the confidence level of the convex polygon to the confidence of the intersection point (the initial confidence of each intersection point is 0).

[0057] In specific embodiments, Figure 5a Two situations of three-plane intersection are shown in b: when three planes cannot form a strong connection relationship, the relationship of the smallest convex polygon formed by the three plane elements in space Figure 5a If the intersection points of the three planes can all fall into the stretchable tolerance scale convex polygon when projected onto the planes, it can be considered that the three planes are soft connected; for example Figure 5b When two pairs of planes among the three planes have formed a strong connection relationship, and the remaining pair of planes cannot form a strong connection relationship, the relationship of the smallest convex polygon formed by the three plane elements in space is shown in 5b. In this case, when the intersection points of the three planes are projected onto the planes respectively, the intersection points fall into the basic scale convex polygon of plane A, the stretchable tolerance scale convex polygon of plane B, and the detection scale polygon of plane C, respectively, it can be considered that the three planes are soft connected. The confidence of the intersection points obtained by the above two situations is the sum of the confidence of the three planes. It can be known that the confidence of the intersection points of the above two situations is 6, the minimum threshold of the allowed intersection point confidence is 6, and only when the intersection point confidence is greater than or equal to 6, the relationship formed by the intersection point can be added to the soft connection. Other plane space structures will form intersection points with different confidences, and the maximum value of the intersection point confidence is 9 and the minimum value is 0. For all intersection points with a confidence greater than 6, a soft connection graph can be established in the form of soft connection. All strong connection relationships can form soft connections. Therefore, the soft connection graph can be formed by adding soft connection edges to the strong connection graph.

[0058] In specific embodiments, on the basis of constructing a soft connection graph, a candidate surface model is generated using a graph structure. Two vertices and their connection form an edge, and three vertices form a closed loop to form a point, obtaining all candidate surface sets, and then generating a candidate surface model. The model is optimized and constrained, and the solving of the model is based on a binary linear integer coding problem. Set the indicator variable x iThe value is 0 (the i-th plane is not selected) or 1 (the i-th plane is selected). Constraints are added to ensure the generated model has manifold and watertight properties. Specifically, the energy function and constraints are min... x λ f E f +λ c E c +λ m E m , Among them, E f This is an energy term based on surface coverage; the larger the area covered by the chosen planar convex polygon, the smaller the energy. c This is the energy term based on data fitting; the more triangles in the original 3D model are covered by the selected plane, the lower the energy. E m The energy term is based on model complexity; the fewer edges in the final model, the lower the energy. The constraint states that each edge in the result must connect two planes. N is the total number of candidate polygon planes, and |E| is the total number of edges in the generated model. This step yields a surface model with better topological logic and manifold watertightness.

[0059] Continue to refer to Figure 6 , Figure 6 A framework diagram of a high-resolution 3D reconstruction system for nearshore areas according to an embodiment of this application is shown. The system specifically includes an image matching unit 601, a dense point cloud generation unit 602, a registration unit 603, and a 3D reconstruction unit 604. Specifically, the image matching unit 601 is configured to match nearshore multi-view images using perspective-invariant features, and obtain a sparse feature point cloud by sampling guided by feature similarity; the dense point cloud generation unit 602 is configured to obtain the depth of each pixel based on a structured local binary pattern using a graph cut method, and densify the sparse feature point cloud to generate a dense point cloud; the registration unit 603 is configured to perform global error optimization on the point clouds from all viewpoints, and generate the 3D coordinates of the feature points in space based on the depth information of the matched feature points; the 3D reconstruction unit 604 is configured to construct strong and weak connections between planar primitives, and obtain a surface model through graph structure and energy function.

[0060] The following is for reference. Figure 7 It shows a schematic diagram of the structure of a computer system 700 suitable for implementing electronic devices according to embodiments of the present application. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0061] like Figure 7As shown, the computer system 700 includes a central processing unit (CPU) 701 which can perform various appropriate actions and processes in accordance with programs stored in a read only memory (ROM) 702 or loaded into a random access memory (RAM) 703 from a storage section 708. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0062] Connected to the I / O interface 705 are an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a display such as a liquid crystal display (LCD), and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as required. A removable recording medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 710 as required, so that a computer program read therefrom is installed into the storage section 708 as required.

[0063] In particular, the processes described above with reference to the flow charts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable storage medium, the computer program comprising program code for performing the methods illustrated by the flow charts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable media 711. When the computer program is executed by the central processing unit (CPU) 701, the above-described functions defined in the methods of the present application are performed. It should be noted that the computer readable storage medium of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, be - but is not limited to - an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable storage medium that can be used for by or in connection with an instruction execution system, apparatus or device, and that can contain or store a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.

[0064] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0065] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0066] The modules involved in the embodiments of the present application can be implemented in the form of software, or can be implemented in the form of hardware.

[0067] As another aspect, the application further provides a computer readable storage medium, which can be included in the electronic device described in the above embodiments, or can exist independently without being assembled into the electronic device. The computer readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: match the offshore area multi-view field angle image by using the perspective size invariant feature, guide sampling according to the feature similarity, and obtain a sparse feature point cloud; obtain the depth of each pixel by using the graph cut method based on the structured local binary pattern, and encrypt the sparse feature point cloud to generate a dense point cloud; perform global error optimization on the point clouds of all view angles, and generate three-dimensional coordinates of the feature points in space according to the depth information of the matched feature points; and construct strong and weak connection relationships between plane primitives, and obtain a surface model by using a graph structure and an energy function.

[0068] The above description is merely the preferred embodiments of the present application and the explanation of the technical principles applied. It should be understood by those skilled in the art that the inventive scope of the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or equivalent features without departing from the inventive concept. For example, the above features can be replaced by the technical features disclosed in the present application (but not limited to) with similar functions to form the technical solutions.

Claims

1. A method for high-resolution three-dimensional reconstruction in near-shore areas, characterized in that, include: S1: Use perspective dimension-invariant features to match multi-view images of nearshore areas, and use feature similarity to guide sampling to obtain sparse feature point clouds; S2: Based on the structured local binary pattern, the depth of each pixel is obtained using the graph cut method, and the sparse feature point cloud is encrypted to generate a dense point cloud; S3: Perform global error optimization on the point cloud from all viewpoints, and generate the three-dimensional coordinates of the feature points in space based on the depth information of the matched feature points; as well as S4: Construct strong and weak connections between planar primitives, and obtain the surface model through graph structure and energy function; Step S4 includes: S41: Based on the maximum tolerance distance of the maximum distance from the fitted local point to the planar primitive and the minimum set number of local points that can be fitted into a planar primitive, the initial primitive is generated by detection. S42: Construct strong connection graphs and soft connection graphs sequentially based on the different intrinsic topological relationships between primitives; S43: Based on the soft connection graph, generate a set of all candidate surfaces using the graph structure and obtain a three-dimensional surface model. Optimize and constrain the three-dimensional surface model according to the energy function and constraints. The strong connection graph and soft connection graph in step S42 include: Traverse the local points of the primitives and perform a spherical neighborhood search for each local point. In response to the fact that at least one local point in the primitive can find other local points of the primitives through the spherical neighborhood search, the two primitives are strongly connected and a strong connection graph is constructed. Using a planar shape growth algorithm, the set of local points of each primitive is projected onto the fitting plane to obtain the minimum convex polygon of the two-dimensional point set. Based on this, the multi-scale regions of the basic scale, the scaling tolerance scale, and the detection scale and their corresponding confidence scores are determined. The confidence scores of the intersection points of the primitive planes in each plane are obtained. In response to the fact that the confidence scores of the intersection points are greater than a preset threshold, soft connection edges are added on the basis of the strong connection graph to construct a soft connection graph.

2. The method for high-resolution three-dimensional reconstruction in near-shore areas according to claim 1, characterized in that, Step S1, which guides sampling based on feature similarity, includes: Calculate N sets of feature matching pairs from multi-view nearshore images For the residual order of the transformation model set ,in This represents the model with the j-th smallest residual in the i-th matching group; The relaxation region is expanded by an extended window, and the model inversion order is calculated using the arrangement relationship within the observation window. This constrains the similarity function. Where h is the window length and ε is the relaxation region. It is a penalty factor related to the window length h. Let A and B be the reverse order of the overlapping models within the window, and let A and B be the weights of the variable window and reverse order models, respectively.

3. The method for high-resolution three-dimensional reconstruction in near-shore areas according to claim 1, characterized in that, In step S2, the pixel depth is obtained using the graph cut method with the following optimization function. ,in, This is a data item that incorporates the similarity of pixel templates in the local binary pattern of the structure. It is a smoothing term that constrains the depth information of the neighborhood.

4. The method for high-resolution three-dimensional reconstruction in near-shore areas according to claim 3, characterized in that, The generation of dense point cloud in step S2 includes: solving for the depth information of all pixels, using the geometric principles of computer vision to calculate the pixel correspondence, and generating dense point cloud.

5. The method for high-resolution three-dimensional reconstruction in near-shore areas according to claim 1, characterized in that, Step S3 further includes curvature filtering distortion correction technology, which uses adjacent data to generate curvature information to smooth the point cloud data.

6. The method for high-resolution three-dimensional reconstruction in near-shore areas according to claim 1, characterized in that, The energy function and constraints are as follows: ,in, It is an energy term based on surface coverage; the larger the area covered by the selected planar convex polygon, the smaller the energy. It is an energy term based on data fitting. The more triangular faces the selected plane covers in the original 3D model, the smaller the energy. The energy term is based on model complexity; the fewer edges in the final model, the lower the energy. The constraint indicates that each edge in the result must connect two planes. N is the total number of candidate polygon planes, and |E| is the total number of edges in the generated model. Indicator variables... The value is 0 when the i-th plane is not selected, and 1 when the i-th plane is selected.

7. A computer-readable storage medium having one or more computer programs stored thereon, characterized in that, When the one or more computer programs are executed by a computer processor, they perform the method according to any one of claims 1 to 6.

8. A high-resolution three-dimensional reconstruction system for nearshore areas, characterized in that, include Image matching unit: configured to match near-shore multi-view images using perspective-invariant features, and to obtain sparse feature point clouds by guiding sampling based on feature similarity; Dense point cloud generation unit: configured to obtain the depth of each pixel based on a structured local binary pattern using a graph cut method, and to encrypt the sparse feature point cloud to generate a dense point cloud; Registration unit: configured to perform global error optimization on point clouds from all viewpoints, and generate the three-dimensional coordinates of the feature points in space based on the depth information of the matched feature points; 3D Reconstruction Unit: Configured to build strong and weak connections between planar primitives, and obtains the surface model through graph structure and energy function; The three-dimensional reconstruction unit includes: S41: Based on the maximum tolerance distance of the maximum distance from the fitted local point to the planar primitive and the minimum set number of local points that can be fitted into a planar primitive, the initial primitive is generated by detection. S42: Construct strong connection graphs and soft connection graphs sequentially based on the different intrinsic topological relationships between primitives; S43: Based on the soft connection graph, generate a set of all candidate surfaces using the graph structure and obtain a three-dimensional surface model. Optimize and constrain the three-dimensional surface model according to the energy function and constraints. The strong connection graph and soft connection graph in step S42 include: Traverse the local points of the primitives and perform a spherical neighborhood search for each local point. In response to the fact that at least one local point in the primitive can find other local points of the primitives through the spherical neighborhood search, the two primitives are strongly connected and a strong connection graph is constructed. Using a planar shape growth algorithm, the set of local points of each primitive is projected onto the fitting plane to obtain the minimum convex polygon of the two-dimensional point set. Based on this, the multi-scale regions of the basic scale, the scaling tolerance scale, and the detection scale and their corresponding confidence scores are determined. The confidence scores of the intersection points of the primitive planes in each plane are obtained. In response to the fact that the confidence scores of the intersection points are greater than a preset threshold, soft connection edges are added on the basis of the strong connection graph to construct a soft connection graph.

Images