Method and system for realizing three-dimensional visual display of ship based on digitization

Abstract

The invention relates to the field of computer vision, and discloses a method and system for realizing three-dimensional visual display of a ship based on digitization, and the method comprises the steps: collecting panoramic data of a to-be-modeled ship, carrying out the geometric correction of a panoramic image of the ship, obtaining a geometric correction panoramic image, recognizing the image feature points of the geometric correction panoramic image, and carrying out the modeling of the to-be-modeled ship. Matching the image feature points to obtain matched image feature points; pixel depth information of the geometric correction panoramic image is calculated, and an initial three-dimensional model of the ship to be modeled is established; marking ship components of the initial three-dimensional model, identifying component functions of the ship components, and performing virtual function layout on the ship components to obtain function layout components; and optimizing the initial three-dimensional model to obtain a ship panoramic state three-dimensional model, and visualizing the ship panoramic state three-dimensional model to obtain a visualized panoramic state three-dimensional model of the ship to be modeled. According to the invention, the three-dimensional visual display effect of the ship can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method and system for realizing three-dimensional visualization display of ships based on digitization. Background Art

[0002] The 3D visualization of a ship refers to the process of presenting the internal and external structure of a ship in the form of a 3D model using 3D modeling technology and virtual reality technology. The 3D visualization of a ship can not only provide a high degree of realism and interactivity, but also provide strong technical support for ship design, manufacturing, maintenance and teaching.

[0003] At present, the three-dimensional visualization display of ships is mainly carried out by collecting the structural data of the ship for three-dimensional modeling, and then displaying the page through interactive technology. This method can only display the three-dimensional structure of the ship, and cannot realize the control of the ship through the three-dimensional model, which leads to the poor degree of interactivity of the three-dimensional visualization display of the ship. Summary of the invention

[0004] The present invention provides a method and system for realizing three-dimensional visualization display of ships based on digitization, the main purpose of which is to improve the three-dimensional visualization display effect of ships.

[0005] To achieve the above object, the present invention provides a method for realizing three-dimensional visualization display of a ship based on digitization, comprising:

[0006] Identify data collection influencing factors of the ship to be modeled, construct a panoramic data collection environment of the ship to be modeled based on the data collection influencing factors, and collect panoramic data of the ship to be modeled based on the panoramic data collection environment, wherein the panoramic data includes: a panoramic image of the ship, ship navigation data, and ship environment data;

[0007] Performing geometric correction on the panoramic image of the ship to obtain a geometrically corrected panoramic image, identifying image feature points of the geometrically corrected panoramic image, and matching the image feature points to obtain matched image feature points;

[0008] Calculating pixel depth information of the geometrically corrected panoramic image based on the matching image feature points, and establishing an initial three-dimensional model of the ship to be modeled based on the pixel depth information;

[0009] Based on a preset ship design drawing, marking the ship components of the initial three-dimensional model, identifying the component functions of the ship components, and performing a virtual functional layout of the ship components based on the component functions to obtain functional layout components;

[0010] Based on the functional layout components, the ship navigation data and the ship environment data, the initial three-dimensional model is optimized to obtain a three-dimensional model of the ship's panoramic state, and the three-dimensional model of the ship's panoramic state is visualized to obtain a visualized three-dimensional model of the panoramic state of the ship to be modeled.

[0011] Optionally, constructing a panoramic data collection environment of the ship to be modeled based on the data collection influencing factor includes:

[0012] Identifying the influence relationship of the data collection influencing factor on the data collection of the ship to be modeled;

[0013] Collecting impact factor data of the data collection impact factor;

[0014] Based on the impact factor data, analyzing the impact factor status of the data acquisition impact factor;

[0015] Based on the influencing factor status and the data collection influencing relationship, a panoramic data collection environment of the ship to be modeled is constructed.

[0016] Optionally, constructing a panoramic data collection environment of the ship to be modeled based on the influencing factor state and the data collection influencing relationship includes:

[0017] Analyzing the factor weights of the ship to be modeled based on the data collection influence relationship;

[0018] Based on the factor weight and the influencing factor state, the collection influence coefficient of the data collection influencing factor corresponding to the ship to be modeled is calculated using the following formula:

[0019]

[0020] in, A represents the collection influence coefficient of the data collection influence factor corresponding to the ship to be modeled, v Indicates the factor state value of the influencing factor state corresponding to the vth data collection influencing factor of the ship to be modeled, E v represents the factor weight of the vth data collection influencing factor of the ship to be modeled, and θ represents the factor influence coefficient of the vth data collection influencing factor of the ship to be modeled on other data collection influencing factors;

[0021] Based on the acquisition influence coefficient, a panoramic data acquisition environment of the ship to be modeled is constructed.

[0022] Optionally, performing geometric correction on the panoramic image of the ship to obtain the geometrically corrected panoramic image includes:

[0023] Identifying a distortion coefficient of a camera device corresponding to the panoramic image of the ship;

[0024] Based on the distortion coefficient, determining a distortion model of the panoramic image of the ship;

[0025] Based on the distortion model, the panoramic image of the ship is geometrically corrected to obtain a geometrically corrected panoramic image.

[0026] Optionally, the performing geometric correction on the ship panoramic image based on the distortion model to obtain the geometrically corrected panoramic image includes:

[0027] Marking the image center of the panoramic image of the ship;

[0028] Calculate the center distance from the corresponding pixel point of the panoramic image of the ship to the center of the image;

[0029] Based on the center distance and the distortion model, the corrected coordinates of the corresponding pixel points of the panoramic image of the ship are calculated using the following formula:

[0030] x_{corr}=x*(1+S1*r^2+S2*r^4+S3*r^6)+(2*B1*x*y

[0031] +B2*(r^2+2*x^2))

[0032] y_{corr}=y*(1+S1*r^2+S2*r^4+S3*r^6)+(B1*(r^2+2

[0033] *y^2)+2*B2*x*y)

[0034] Wherein, (x_{corr}, y_{corr}) represents the corrected coordinates of the pixel point (x, y) corresponding to the panoramic image of the ship, x represents the value of the pixel point corresponding to the panoramic image of the ship on the x-axis, y represents the value of the pixel point corresponding to the panoramic image of the ship on the y-axis, r represents the center distance of the pixel point (x, y) corresponding to the panoramic image of the ship, S1, S2, S3 represent the radial distortion coefficients corresponding to the distortion model, B1, B2 represent the tangential distortion coefficients corresponding to the distortion model;

[0035] Based on the correction coordinates, the ship panoramic image is geometrically corrected to obtain the geometrically corrected panoramic image.

[0036] Optionally, the identifying image feature points of the geometrically corrected panoramic image includes:

[0037] gray-scale the geometrically corrected panoramic image to obtain a gray-scale panoramic image;

[0038] Calculating pixel gradients of the grayscale panoramic image;

[0039] Calculating pixel response values ​​of the geometrically corrected panoramic image based on the pixel gradients;

[0040] Based on the pixel response values, image feature points of the geometrically corrected panoramic image are determined.

[0041] Optionally, the calculating the pixel response value of the geometrically corrected panoramic image based on the pixel gradient includes:

[0042] Based on the pixel gradients, calculating a gradient covariance matrix of the geometrically corrected panoramic image;

[0043] The eigenvalues ​​of the gradient covariance matrix are calculated using the following formula:

[0044]

[0045] Wherein, ω1, ω2 represent the eigenvalues ​​of the gradient covariance matrix, Ia represents the pixel gradient of the pixel point in the geometrically corrected panoramic image corresponding to the gradient covariance matrix on the x-axis, and Ic represents the pixel gradient of the pixel point in the geometrically corrected panoramic image corresponding to the gradient covariance matrix on the y-axis;

[0046] Based on the characteristic values, pixel response values ​​of the geometrically corrected panoramic image are calculated.

[0047] Optionally, the calculating pixel depth information of the geometrically corrected panoramic image based on the matching image feature points includes:

[0048] constructing a corresponding map of the geometrically corrected panoramic image based on the matching image feature points;

[0049] Based on the correspondence map, evaluating the pixel depth of the geometrically corrected panoramic image;

[0050] Based on the pixel depths, constructing a depth map of the geometrically corrected panoramic image;

[0051] Based on the depth map, pixel depth information of the geometrically corrected panoramic image is determined.

[0052] Optionally, establishing an initial three-dimensional model of the ship to be modeled based on the pixel depth information includes:

[0053] Converting the pixel depth information into a voxel grid;

[0054] identifying voxel attributes of voxels corresponding to the voxel grid;

[0055] Extracting triangular facets of the voxel grid based on the voxel attributes;

[0056] Based on the triangular facets, an initial three-dimensional model of the ship to be modeled is established.

[0057] In order to solve the above problems, the present invention also provides a three-dimensional visualization display system for ships based on digitalization, the system comprising:

[0058] A panoramic data acquisition module is used to identify data acquisition influencing factors of the ship to be modeled, construct a panoramic data acquisition environment of the ship to be modeled based on the data acquisition influencing factors, and acquire panoramic data of the ship to be modeled based on the panoramic data acquisition environment, wherein the panoramic data includes: panoramic images of the ship, ship navigation data, and ship environmental data.

[0059] A feature point matching module is used to perform geometric correction on the panoramic image of the ship to obtain a geometrically corrected panoramic image, identify image feature points of the geometrically corrected panoramic image, and match the image feature points to obtain matched image feature points;

[0060] An initial three-dimensional model building module, used to calculate the pixel depth information of the geometrically corrected panoramic image based on the matching image feature points, and to establish an initial three-dimensional model of the ship to be modeled based on the pixel depth information;

[0061] A virtual function layout module is used to mark the ship components of the initial three-dimensional model based on a preset ship design drawing, identify the component functions of the ship components, and perform virtual function layout on the ship components based on the component functions to obtain function layout components;

[0062] The ship three-dimensional visualization module is used to optimize the initial three-dimensional model based on the functional layout components, the ship navigation data and the ship environment data to obtain a three-dimensional model of the ship's panoramic state, and to visualize the three-dimensional model of the ship's panoramic state to obtain a visualized three-dimensional model of the panoramic state of the ship to be modeled.

[0063] The embodiment of the present invention constructs a panoramic data acquisition environment for the ship to be modeled based on the data acquisition influencing factors, which can achieve the data integrity of the ship to be modeled, thereby improving the completeness of the later ship modeling; the embodiment of the present invention obtains a geometrically corrected panoramic image by geometrically correcting the panoramic image of the ship, which can obtain an image with a more realistic and accurate visual effect; the embodiment of the present invention can identify stable feature points from the geometrically corrected panoramic image by identifying the image feature points of the geometrically corrected panoramic image, and these feature points can be used for subsequent image processing tasks; further, the embodiment of the present invention calculates the pixel depth information of the geometrically corrected panoramic image based on the matching image feature points to provide a basis for three-dimensional reconstruction and visualization; further, the embodiment of the present invention performs a virtual functional layout of the ship components based on the component functions, and the obtained functional layout components can realize the model's control interface for the ship. Among them, the functional layout component refers to the component after the functional interface is connected to the model. Finally, the embodiment of the present invention optimizes the initial three-dimensional model based on the functional layout component, the ship navigation data and the ship environment data to obtain a three-dimensional model of the panoramic state of the ship. A more detailed model can be obtained, which improves the authenticity and controllability of the model. The embodiment of the present invention visualizes the three-dimensional model of the panoramic state of the ship to be modeled, and obtains a visualized three-dimensional model of the panoramic state of the ship to be modeled, which can realize the visualization display of the ship to be modeled. Therefore, the method and system for realizing the three-dimensional visualization display of ships based on digitization proposed by the present invention can improve the three-dimensional visualization display effect of ships. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic diagram of a process for realizing a three-dimensional visual display method of a ship based on digitization provided in one embodiment of the present invention;

[0065] Figure 2 A functional module diagram of a three-dimensional visual display system for ships based on digitalization provided by an embodiment of the present invention;

[0066] Figure 3 A schematic diagram of the structure of an electronic device for realizing a three-dimensional visual display system of a ship based on digitization provided by an embodiment of the present invention;

[0067] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0068] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0069] The embodiment of the present application provides a three-dimensional visualization display method for ships based on digitalization. The execution subject of the three-dimensional visualization display method for ships based on digitalization includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the three-dimensional visualization display method for ships based on digitalization can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0070] Reference Figure 1 FIG. 1 is a flow chart of a method for realizing three-dimensional visualization display of a ship based on digitization according to an embodiment of the present invention. In this embodiment, the method for realizing three-dimensional visualization display of a ship based on digitization includes:

[0071] S1. Identify data collection influencing factors of the ship to be modeled, and based on the data collection influencing factors, construct a panoramic data collection environment of the ship to be modeled. Based on the panoramic data collection environment, collect panoramic data of the ship to be modeled, wherein the panoramic data includes: panoramic images of the ship, ship navigation data, and ship environmental data.

[0072] In the embodiment of the present invention, the data collection influencing factor refers to a factor that affects the data collection of the ship to be modeled, such as the hidden structure and environment of the ship to be modeled.

[0073] Optionally, the embodiment of the present invention constructs a panoramic data acquisition environment of the ship to be modeled based on the data acquisition influencing factors to achieve data integrity of the ship to be modeled, thereby improving the completeness of later ship modeling. The panoramic data acquisition environment refers to an environment in which ship data can be fully collected, such as a camera environment, a sensor environment, and the like. For example, the camera environment can be integrated with LiDAR and CMOS sensor cameras, with an image quality of 16K and 134 million pixels.

[0074] As an embodiment of the present invention, constructing a panoramic data acquisition environment for the ship to be modeled based on the data acquisition influencing factors includes: identifying the data acquisition influence relationship of the data acquisition influencing factors on the ship to be modeled; collecting influence factor data of the data acquisition influencing factors; based on the influence factor data, analyzing the influence factor status of the data acquisition influencing factors; and constructing a panoramic data acquisition environment for the ship to be modeled based on the influence factor status and the data acquisition influence relationship.

[0075] Among them, the data collection influencing relationship refers to the relationship between the data collection influencing factor and the data collection of the ship to be modeled. For example, in a ship rainstorm scene, the heavier the rain, the more difficult it is to collect ship data. The influencing factor data refers to data describing the data collection influencing factor, such as ship corner structure, ship surrounding environment and other data. The influencing factor state refers to the current state of the data collection influencing factor, such as heavy rain, ship structure obstruction and other states.

[0076] Further, in an optional embodiment of the present invention, the panoramic data collection environment of the ship to be modeled is constructed based on the influencing factor state and the data collection influence relationship, including: analyzing the factor weight of the ship to be modeled based on the data collection influence relationship; and calculating the collection influence coefficient of the data collection influence factor corresponding to the ship to be modeled based on the factor weight and the influencing factor state using the following formula:

[0077]

[0078] in, A represents the collection influence coefficient of the data collection influence factor corresponding to the ship to be modeled, v Indicates the factor state value of the influencing factor state corresponding to the vth data collection influencing factor of the ship to be modeled, E v represents the factor weight of the vth data collection influencing factor of the ship to be modeled, and θ represents the factor influence coefficient of the vth data collection influencing factor of the ship to be modeled on other data collection influencing factors;

[0079] Based on the acquisition influence coefficient, a panoramic data acquisition environment of the ship to be modeled is constructed.

[0080] Among them, the factor state value refers to the quantitative value of the influencing factor state, such as the amount of heavy rain, the area blocked by the ship structure, etc. The factor influence coefficient refers to the degree of mutual influence between the data collection influencing factors, and the collection influence coefficient refers to the degree of influence of the data collection influencing factors on the ship data collection.

[0081] In the embodiment of the present invention, the panoramic data refers to a data set that describes the internal and external structure, environment, etc. of the ship to be modeled, obtained by all-round photography, for example, multi-dimensional images of the ship, internal and external structure data of the ship, ship environment data, and other data.

[0082] S2. geometrically correct the panoramic image of the ship to obtain a geometrically corrected panoramic image, identify image feature points of the geometrically corrected panoramic image, and match the image feature points to obtain matched image feature points.

[0083] The embodiment of the present invention performs geometric correction on the panoramic image of the ship to obtain a geometrically corrected panoramic image, which can obtain an image with a more realistic and accurate visual effect. The geometrically corrected panoramic image refers to the panoramic image of the ship after correcting the geometric deformation of the image caused by factors such as camera lens distortion or shooting angle.

[0084] As an embodiment of the present invention, the geometric correction of the panoramic image of the ship to obtain the geometrically corrected panoramic image includes: identifying the distortion coefficient of the camera equipment corresponding to the panoramic image of the ship; determining the distortion model of the panoramic image of the ship based on the distortion coefficient; and geometrically correcting the panoramic image of the ship based on the distortion model to obtain the geometrically corrected panoramic image.

[0085] The distortion coefficient refers to a parameter describing the optical distortion of a camera lens. The distortion coefficient generally includes a radial distortion coefficient and a tangential distortion coefficient. The distortion model includes: a radial distortion model and a tangential distortion model.

[0086] Further, in an optional embodiment of the present invention, the geometric correction of the panoramic image of the ship is performed based on the distortion model to obtain a geometrically corrected panoramic image, including: marking the image center of the panoramic image of the ship; calculating the center distance from the corresponding pixel point of the panoramic image of the ship to the image center; based on the center distance and the distortion model, calculating the corrected coordinates of the corresponding pixel point of the panoramic image of the ship using the following formula:

[0087] x_{corr}=x*(1+S1*r^2+S2*r^4+S3*r^6)+(2*B1*x*y

[0088] +B2*(r^2+2*x^2))

[0089] y_{corr}=y*(1+S1*r^2+S2*r^4+S3*r^6)+(B1*(r^2+2

[0090] *y^2)+2*B2*x*y)

[0091] Among them, (x_{corr}, y_{corr}) represents the corrected coordinates of the pixel point (x, y) corresponding to the panoramic image of the ship, x represents the value of the pixel point corresponding to the panoramic image of the ship on the x-axis, y represents the value of the pixel point corresponding to the panoramic image of the ship on the y-axis, r represents the center distance of the pixel point (x, y) corresponding to the panoramic image of the ship, S1, S2, S3 represent the radial distortion coefficients corresponding to the distortion model, B1, B2 represent the tangential distortion coefficients corresponding to the distortion model; based on the corrected coordinates, the panoramic image of the ship is geometrically corrected to obtain the geometrically corrected panoramic image.

[0092] Optionally, the embodiment of the present invention can identify stable feature points from the geometrically corrected panoramic image by identifying image feature points of the geometrically corrected panoramic image, and these feature points can be used for subsequent image processing tasks. The image feature points refer to representative points in the image.

[0093] Optionally, as an embodiment of the present invention, the identifying of image feature points of the geometrically corrected panoramic image includes: performing grayscale processing on the geometrically corrected panoramic image to obtain a grayscale panoramic image; calculating pixel gradients of the grayscale panoramic image; based on the pixel gradients, calculating pixel response values ​​of the geometrically corrected panoramic image; based on the pixel response values, determining image feature points of the geometrically corrected panoramic image.

[0094] Among them, the grayscale panoramic image refers to the image obtained by converting the geometrically corrected panoramic image into a grayscale image, the pixel gradient refers to a vector, which contains the direction in which the image brightness changes the most and the rate of change in this direction, and the pixel response value refers to the potential value of each pixel point in the image as a corner point.

[0095] Further, in an optional embodiment of the present invention, the pixel response value of the geometrically corrected panoramic image is calculated based on the pixel gradient, including: calculating the gradient covariance matrix of the geometrically corrected panoramic image based on the pixel gradient; and calculating the eigenvalue of the gradient covariance matrix using the following formula:

[0096] ω1=(ΣIa^2+ΣIa^2+√((ΣIa^2-ΣIc^2)^2+4ΣIaIc^2)) / 2

[0097] ω2=(ΣIa^2+ΣIc^2-√((ΣIa^2-ΣIc^2)^2+4ΣIaIc^2)) / 2

[0098] Wherein, ω1, ω2 represent the eigenvalues ​​of the gradient covariance matrix, Ia represents the pixel gradient of the pixel point in the geometrically corrected panoramic image corresponding to the gradient covariance matrix on the x-axis, and Ic represents the pixel gradient of the pixel point in the geometrically corrected panoramic image corresponding to the gradient covariance matrix on the y-axis;

[0099] Based on the characteristic values, pixel response values ​​of the geometrically corrected panoramic image are calculated.

[0100] The embodiment of the present invention can achieve image matching by matching the image feature points to obtain matching image feature points, thereby providing a data basis for subsequent image registration. The matching image feature points refer to the feature points that are matched. In detail, the matching image feature points can be achieved through matching algorithms such as Brute-Force, FLANN and other algorithms. Based on the matching image feature points, the automatic completion of image stitching, image white balance processing and HDR processing of geometrically corrected panoramic images can be achieved to prepare for subsequent modeling.

[0101] S3. Calculate pixel depth information of the geometrically corrected panoramic image based on the matching image feature points, and establish an initial three-dimensional model of the ship to be modeled based on the pixel depth information.

[0102] The embodiment of the present invention calculates the pixel depth information of the geometrically corrected panoramic image based on the matching image feature points to provide a basis for three-dimensional reconstruction and visualization, wherein the pixel depth information refers to the depth value in the three-dimensional space corresponding to each pixel point in the two-dimensional image.

[0103] As an embodiment of the present invention, the calculating the pixel depth information of the geometrically corrected panoramic image based on the matching image feature points includes: constructing a correspondence map of the geometrically corrected panoramic image based on the matching image feature points; evaluating the pixel depth of the geometrically corrected panoramic image based on the correspondence map; constructing a depth map of the geometrically corrected panoramic image based on the pixel depth; and determining the pixel depth information of the geometrically corrected panoramic image based on the depth map.

[0104] Among them, the correspondence map refers to a correspondence relationship map between feature points established between images of multiple perspectives, the pixel depth refers to the position of the pixel point in the three-dimensional space, and the depth map refers to an image representing the pixel depth.

[0105] Optionally, based on the pixel depth information, the initial three-dimensional model of the ship to be modeled can be established by implementing the establishment of an initial model of the ship, wherein the initial three-dimensional model refers to a three-dimensional basic model of the ship to be modeled.

[0106] Optionally, as an embodiment of the present invention, the initial three-dimensional model of the ship to be modeled is established based on the pixel depth information, including: converting the pixel depth information into voxel grids; identifying voxel attributes of voxels corresponding to the voxel grids; based on the voxel attributes, extracting triangular facets of the voxel grids; and establishing the initial three-dimensional model of the ship to be modeled based on the triangular facets.

[0107] The voxel grid refers to a three-dimensional image data structure used to represent a point in a three-dimensional space, the voxel attribute refers to its position and direction in the three-dimensional space, and the triangle patch refers to a graphic data structure used to represent a three-dimensional shape. In detail, the process of establishing the initial three-dimensional model uses graphics technology to achieve automatic model splicing, model mapping and other tasks.

[0108] S4. Based on a preset ship design drawing, mark the ship components of the initial three-dimensional model, identify the component functions of the ship components, and perform virtual functional layout of the ship components based on the component functions to obtain functional layout components.

[0109] In the embodiment of the present invention, the ship components refer to various components of the ship such as the hull, deck, engine room, engine and other structures. The component functions refer to functional indicators of the ship components, such as the engine providing power, the engine room providing accommodation and other indicators.

[0110] Optionally, the embodiment of the present invention performs a virtual functional layout of the ship components based on the component functions, and the obtained functional layout components can realize the control interface of the model to the ship. Among them, the functional layout component refers to the component after the functional interface is connected to the model. In detail, the virtual functional layout of the ship components is achieved by setting the functional control interface of the components corresponding to the initial three-dimensional model according to the component functions. Exemplarily, customized multimedia detail display: supports video and virtual screen embedding, supports customized navigation functions, and more flexibly and conveniently displays the details of various parts of the ship, and realizes the digital marking of ship equipment, marks and displays the details of the equipment of the entire ship, and more conveniently completes the construction of the digital database of the entire ship.

[0111] S5. Based on the functional layout components, the ship navigation data and the ship environment data, the initial three-dimensional model is optimized to obtain a three-dimensional model of a panoramic state of the ship, and the three-dimensional model of the panoramic state of the ship is visualized to obtain a visualized three-dimensional model of the panoramic state of the ship to be modeled.

[0112] The embodiment of the present invention optimizes the initial three-dimensional model based on the functional layout component, the ship navigation data and the ship environmental data, and obtains a three-dimensional model of the panoramic state of the ship, which can obtain a more detailed model and improve the authenticity and controllability of the model. Among them, the three-dimensional model of the panoramic state of the ship refers to a three-dimensional model that can display the current driving state, current environmental state and structural display of the ship. In detail, the three-dimensional model of the panoramic state of the ship analyzes the ship driving state and the ship driving environment of the ship to be modeled through the ship navigation data and the ship environmental data, and uses three-dimensional software to integrate the ship driving state, the ship driving environment and the functional interface of the functional layout component into the initial three-dimensional model for construction.

[0113] Finally, the embodiment of the present invention visualizes the panoramic state three-dimensional model of the ship, and obtains the visualized panoramic state three-dimensional model of the ship to be modeled, which can realize the visualized display of the ship to be modeled. In detail, the visualization of the panoramic state three-dimensional model of the ship can render the three-dimensional model in real time on the web page through WebGL, and the user can view and interact through the browser. For example, through a rich VR editor, various display requirements are met, such as multimedia tags, spatial mosaics, etc., and the panoramic state of the ship is fully displayed on both mobile phones and computers. It is equipped with tile loading to solve the browsing jam problem of the display. It should be noted that the whole ship model is displayed in 1:1, the hull structure and various pipelines are clearly visible, and the appearance of the whole ship can be viewed as a whole, and the interior of the hull can also be viewed in layers.

[0114] The embodiment of the present invention constructs a panoramic data acquisition environment for the ship to be modeled based on the data acquisition influencing factors, which can achieve the data integrity of the ship to be modeled, thereby improving the completeness of the later ship modeling; the embodiment of the present invention obtains a geometrically corrected panoramic image by geometrically correcting the panoramic image of the ship, which can obtain an image with a more realistic and accurate visual effect; the embodiment of the present invention can identify stable feature points from the geometrically corrected panoramic image by identifying the image feature points of the geometrically corrected panoramic image, and these feature points can be used for subsequent image processing tasks; further, the embodiment of the present invention calculates the pixel depth information of the geometrically corrected panoramic image based on the matching image feature points to provide a basis for three-dimensional reconstruction and visualization; further, the embodiment of the present invention performs a virtual functional layout of the ship components based on the component functions, and the obtained functional layout components can realize the model's control interface for the ship. Among them, the functional layout component refers to the component after the functional interface is connected to the model. Finally, the embodiment of the present invention optimizes the initial three-dimensional model based on the functional layout component, the ship navigation data and the ship environment data to obtain a three-dimensional model of the panoramic state of the ship. A more detailed model can be obtained, which improves the authenticity and controllability of the model. The embodiment of the present invention visualizes the three-dimensional model of the panoramic state of the ship to be modeled, and obtains a visualized three-dimensional model of the panoramic state of the ship to be modeled, which can realize the visualization display of the ship to be modeled. Therefore, the three-dimensional visualization display method of the ship based on digitization proposed by the present invention can improve the three-dimensional visualization display effect of the ship.

[0115] like Figure 2 1 is a functional module diagram of a system for realizing three-dimensional visualization of ships based on digitalization provided by an embodiment of the present invention.

[0116] The three-dimensional visualization display system 200 for realizing a ship based on digitalization according to the present invention can be installed in an electronic device. According to the functions realized, the three-dimensional visualization display system 200 for realizing a ship based on digitalization can include a panoramic data acquisition module 201, a feature point matching module 202, an initial three-dimensional model construction module 203, a virtual function layout module 204 and a ship three-dimensional visualization module 205. The module according to the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0117] In this embodiment, the functions of each module / unit are as follows:

[0118] The panoramic data acquisition module 201 is used to identify the data acquisition influencing factors of the ship to be modeled, construct a panoramic data acquisition environment of the ship to be modeled based on the data acquisition influencing factors, and acquire panoramic data of the ship to be modeled based on the panoramic data acquisition environment, wherein the panoramic data includes: panoramic images of the ship, ship navigation data, and ship environmental data.

[0119] The feature point matching module 202 is used to perform geometric correction on the panoramic image of the ship to obtain a geometrically corrected panoramic image, identify image feature points of the geometrically corrected panoramic image, and match the image feature points to obtain matched image feature points;

[0120] The initial three-dimensional model building module 203 is used to calculate the pixel depth information of the geometrically corrected panoramic image based on the matching image feature points, and to build the initial three-dimensional model of the ship to be modeled based on the pixel depth information;

[0121] The virtual function layout module 204 is used to mark the ship components of the initial three-dimensional model based on the preset ship design drawings, identify the component functions of the ship components, and perform virtual function layout on the ship components based on the component functions to obtain function layout components;

[0122] The ship three-dimensional visualization module 205 is used to optimize the initial three-dimensional model based on the functional layout components, the ship navigation data and the ship environment data to obtain a ship panoramic state three-dimensional model, and visualize the ship panoramic state three-dimensional model to obtain a visualized panoramic state three-dimensional model of the ship to be modeled.

[0123] In detail, the modules described in the three-dimensional visualization display system 200 for realizing a ship based on digitalization in the embodiment of the present invention adopt the same technical means as the three-dimensional visualization display method for realizing a ship based on digitalization described in the accompanying drawings when used, and can produce the same technical effects, which will not be repeated here.

[0124] An embodiment of the present invention provides an electronic device for implementing a method for realizing a three-dimensional visualization display of a ship based on digitization.

[0125] See also Figure 3 As shown, the electronic device may include a processor 30, a memory 31, a communication bus 32 and a communication interface 33, and may also include a computer program stored in the memory 31 and executable on the processor 30, such as a program for a three-dimensional visualization display method of a ship based on digitalization.

[0126] In some embodiments, the processor may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory (for example, executing a three-dimensional visualization display program for ships based on digitalization, etc.), and calls data stored in the memory to execute various functions of the electronic device and process data.

[0127] The memory includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory may also include both an internal storage unit of the electronic device and an external storage device. The memory may be used not only to store application software and various types of data installed in the electronic device, such as the code of a three-dimensional visualization display program based on digitalization of ships, but also to temporarily store data that has been output or is to be output.

[0128] The communication bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory and at least one processor, etc.

[0129] The communication interface is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0130] For example, although not shown, the electronic device may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor through a power management system, so that the power management system can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0131] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0132] The program for realizing the three-dimensional visualization display of a ship based on digitization stored in the memory of the electronic device is a combination of multiple instructions. When running in the processor, the following can be achieved:

[0133] Identify data collection influencing factors of the ship to be modeled, construct a panoramic data collection environment of the ship to be modeled based on the data collection influencing factors, and collect panoramic data of the ship to be modeled based on the panoramic data collection environment, wherein the panoramic data includes: a panoramic image of the ship, ship navigation data, and ship environment data;

[0134] Performing geometric correction on the panoramic image of the ship to obtain a geometrically corrected panoramic image, identifying image feature points of the geometrically corrected panoramic image, and matching the image feature points to obtain matched image feature points;

[0135] Calculating pixel depth information of the geometrically corrected panoramic image based on the matching image feature points, and establishing an initial three-dimensional model of the ship to be modeled based on the pixel depth information;

[0136] Based on a preset ship design drawing, marking the ship components of the initial three-dimensional model, identifying the component functions of the ship components, and performing a virtual functional layout of the ship components based on the component functions to obtain functional layout components;

[0137] Based on the functional layout components, the ship navigation data and the ship environment data, the initial three-dimensional model is optimized to obtain a three-dimensional model of the ship's panoramic state, and the three-dimensional model of the ship's panoramic state is visualized to obtain a visualized three-dimensional model of the panoramic state of the ship to be modeled.

[0138] Specifically, the specific implementation method of the processor for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0139] Furthermore, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or system capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0140] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0141] Identify data collection influencing factors of the ship to be modeled, construct a panoramic data collection environment of the ship to be modeled based on the data collection influencing factors, and collect panoramic data of the ship to be modeled based on the panoramic data collection environment, wherein the panoramic data includes: a panoramic image of the ship, ship navigation data, and ship environment data;

[0142] Performing geometric correction on the panoramic image of the ship to obtain a geometrically corrected panoramic image, identifying image feature points of the geometrically corrected panoramic image, and matching the image feature points to obtain matched image feature points;

[0143] Calculating pixel depth information of the geometrically corrected panoramic image based on the matching image feature points, and establishing an initial three-dimensional model of the ship to be modeled based on the pixel depth information;

[0144] Based on a preset ship design drawing, marking the ship components of the initial three-dimensional model, identifying the component functions of the ship components, and performing a virtual functional layout of the ship components based on the component functions to obtain functional layout components;

[0145] Based on the functional layout components, the ship navigation data and the ship environment data, the initial three-dimensional model is optimized to obtain a three-dimensional model of the ship's panoramic state, and the three-dimensional model of the ship's panoramic state is visualized to obtain a visualized three-dimensional model of the panoramic state of the ship to be modeled.

[0146] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0147] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0148] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0149] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0150] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0151] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0152] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or systems stated in a system claim can also be implemented by one unit or system through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for realizing three-dimensional visualization display of ships based on digitalization, characterized in that: The method comprises: Identify data collection influencing factors of the ship to be modeled, construct a panoramic data collection environment of the ship to be modeled based on the data collection influencing factors, and collect panoramic data of the ship to be modeled based on the panoramic data collection environment, wherein the panoramic data includes: a panoramic image of the ship, ship navigation data, and ship environment data; Performing geometric correction on the panoramic image of the ship to obtain a geometrically corrected panoramic image, identifying image feature points of the geometrically corrected panoramic image, and matching the image feature points to obtain matched image feature points; Calculating pixel depth information of the geometrically corrected panoramic image based on the matching image feature points, and establishing an initial three-dimensional model of the ship to be modeled based on the pixel depth information; Based on a preset ship design drawing, marking the ship components of the initial three-dimensional model, identifying the component functions of the ship components, and performing a virtual functional layout of the ship components based on the component functions to obtain functional layout components; Based on the functional layout components, the ship navigation data and the ship environment data, the initial three-dimensional model is optimized to obtain a three-dimensional model of the ship's panoramic state, and the three-dimensional model of the ship's panoramic state is visualized to obtain a visualized three-dimensional model of the panoramic state of the ship to be modeled.

2. The method for realizing three-dimensional visualization display of a ship based on digitization according to claim 1, characterized in that: The step of constructing a panoramic data collection environment of the ship to be modeled based on the data collection influencing factors includes: Identifying the influence relationship of the data collection influencing factor on the data collection of the ship to be modeled; Collecting impact factor data of the data collection impact factor; Based on the impact factor data, analyzing the impact factor status of the data acquisition impact factor; Based on the influencing factor status and the data collection influencing relationship, a panoramic data collection environment of the ship to be modeled is constructed.

3. The method for realizing three-dimensional visualization display of a ship based on digitization according to claim 2, characterized in that: The step of constructing a panoramic data collection environment of the ship to be modeled based on the influencing factor state and the data collection influencing relationship includes: Analyzing the factor weights of the ship to be modeled based on the data collection influence relationship; Based on the factor weight and the influencing factor state, the collection influence coefficient of the data collection influencing factor corresponding to the ship to be modeled is calculated using the following formula: in, A represents the collection influence coefficient of the data collection influence factor corresponding to the ship to be modeled, v Indicates the factor state value of the influencing factor state corresponding to the vth data collection influencing factor of the ship to be modeled, E v represents the factor weight of the vth data collection influencing factor of the ship to be modeled, and θ represents the factor influence coefficient of the vth data collection influencing factor of the ship to be modeled on other data collection influencing factors; Based on the acquisition influence coefficient, a panoramic data acquisition environment of the ship to be modeled is constructed.

4. The method for realizing three-dimensional visualization display of a ship based on digitization according to claim 1, characterized in that: The step of geometrically correcting the panoramic image of the ship to obtain a geometrically corrected panoramic image includes: Identifying a distortion coefficient of a camera device corresponding to the panoramic image of the ship; Based on the distortion coefficient, determining a distortion model of the panoramic image of the ship; Based on the distortion model, the panoramic image of the ship is geometrically corrected to obtain a geometrically corrected panoramic image.

5. The method for realizing three-dimensional visualization display of a ship based on digitization according to claim 1, characterized in that: The step of geometrically correcting the panoramic image of the ship based on the distortion model to obtain a geometrically corrected panoramic image includes: Marking the image center of the panoramic image of the ship; Calculate the center distance from the corresponding pixel point of the panoramic image of the ship to the center of the image; Based on the center distance and the distortion model, the corrected coordinates of the corresponding pixel points of the panoramic image of the ship are calculated using the following formula: x_{corr}=x*(1+S1*r^2+S2*r^4+S3*r^6)+(2*B1*x*y+B2*(r^2+2*x^2)) y_{corr}=y*(1+S1*r^2+S2*r^4+S3*r^6)+(B1*(r^2+2*y^2)+2*B2*x*y) Wherein, (x_{corr}, y_{corr}) represents the corrected coordinates of the pixel point (x, y) corresponding to the panoramic image of the ship, x represents the value of the pixel point corresponding to the panoramic image of the ship on the x-axis, y represents the value of the pixel point corresponding to the panoramic image of the ship on the y-axis, r represents the center distance of the pixel point (x, y) corresponding to the panoramic image of the ship, S1, S2, S3 represent the radial distortion coefficients corresponding to the distortion model, B1, B2 represent the tangential distortion coefficients corresponding to the distortion model; Based on the correction coordinates, the ship panoramic image is geometrically corrected to obtain the geometrically corrected panoramic image.

6. The method for realizing three-dimensional visualization display of a ship based on digitization according to claim 1, characterized in that: The identifying of image feature points of the geometrically corrected panoramic image comprises: gray-scale the geometrically corrected panoramic image to obtain a gray-scale panoramic image; Calculating pixel gradients of the grayscale panoramic image; Calculating pixel response values ​​of the geometrically corrected panoramic image based on the pixel gradients; Based on the pixel response values, image feature points of the geometrically corrected panoramic image are determined.

7. The method for realizing three-dimensional visualization display of a ship based on digitization according to claim 6, characterized in that: The step of calculating the pixel response value of the geometrically corrected panoramic image based on the pixel gradient comprises: Based on the pixel gradients, calculating a gradient covariance matrix of the geometrically corrected panoramic image; The eigenvalues ​​of the gradient covariance matrix are calculated using the following formula: ω1=(ΣIa^2+ΣIa^2+√((ΣIa^2-ΣIc^2)^2+4ΣIaIc^2)) / 2 ω2=(ΣIa^2+ΣIc^2-√((ΣIa^2-ΣIc^2)^2+4ΣIaIc^2)) / 2 Wherein, ω1, ω2 represent the eigenvalues ​​of the gradient covariance matrix, Ia represents the pixel gradient of the pixel point in the geometrically corrected panoramic image corresponding to the gradient covariance matrix on the x-axis, and Ic represents the pixel gradient of the pixel point in the geometrically corrected panoramic image corresponding to the gradient covariance matrix on the y-axis; Based on the characteristic values, pixel response values ​​of the geometrically corrected panoramic image are calculated.

8. The method for realizing three-dimensional visualization display of a ship based on digitization according to claim 1, characterized in that: The step of calculating pixel depth information of the geometrically corrected panoramic image based on the matched image feature points includes: constructing a corresponding map of the geometrically corrected panoramic image based on the matching image feature points; Based on the correspondence map, evaluating the pixel depth of the geometrically corrected panoramic image; Based on the pixel depths, constructing a depth map of the geometrically corrected panoramic image; Based on the depth map, pixel depth information of the geometrically corrected panoramic image is determined.

9. The method for realizing three-dimensional visualization display of a ship based on digitization according to claim 1, characterized in that: The step of establishing an initial three-dimensional model of the ship to be modeled based on the pixel depth information includes: Converting the pixel depth information into a voxel grid; identifying voxel attributes of voxels corresponding to the voxel grid; Extracting triangular facets of the voxel grid based on the voxel attributes; Based on the triangular facets, an initial three-dimensional model of the ship to be modeled is established.

10. A three-dimensional visualization display system for ships based on digitalization, characterized in that: The system is used to execute the method for realizing three-dimensional visualization display of a ship based on digitization according to any one of claims 1 to 9, the system comprising: A panoramic data acquisition module is used to identify data acquisition influencing factors of the ship to be modeled, construct a panoramic data acquisition environment of the ship to be modeled based on the data acquisition influencing factors, and acquire panoramic data of the ship to be modeled based on the panoramic data acquisition environment, wherein the panoramic data includes: panoramic images of the ship, ship navigation data, and ship environmental data. A feature point matching module is used to perform geometric correction on the panoramic image of the ship to obtain a geometrically corrected panoramic image, identify image feature points of the geometrically corrected panoramic image, and match the image feature points to obtain matched image feature points; An initial three-dimensional model building module, used to calculate the pixel depth information of the geometrically corrected panoramic image based on the matching image feature points, and to establish an initial three-dimensional model of the ship to be modeled based on the pixel depth information; A virtual function layout module is used to mark the ship components of the initial three-dimensional model based on a preset ship design drawing, identify the component functions of the ship components, and perform virtual function layout on the ship components based on the component functions to obtain function layout components; The ship three-dimensional visualization module is used to optimize the initial three-dimensional model based on the functional layout components, the ship navigation data and the ship environment data to obtain a three-dimensional model of the ship's panoramic state, and to visualize the three-dimensional model of the ship's panoramic state to obtain a visualized three-dimensional model of the panoramic state of the ship to be modeled.

Images