Method for mixed reality registration of a ship model, terminal and medium

Abstract

The application provides a mixed reality registration method of a ship model, a terminal and a medium. The method comprises the following steps: determining a positioning member in a ship body real object in a work area; selecting a real object identification point on the positioning member and acquiring spatial distribution information of the real object identification point; selecting a model identification point corresponding to the real object identification point on a part model corresponding to the positioning member and acquiring spatial distribution information of the model identification point; acquiring a spatial conversion mode between the real object identification point and the model identification point based on the spatial distribution information of the real object identification point and the model identification point; and performing spatial conversion on the ship model based on the spatial conversion mode, so that a new ship model after spatial conversion is registered on the ship body real object, thereby realizing the mixed registration of the ship model quickly and efficiently without the need of additionally setting a positioning identification.

Description

Technical Field

[0001] This invention relates to the field of mixed reality technology, and in particular to a mixed reality registration method, terminal, and computer storage medium for a ship model. Background Technology

[0002] With the continuous development of mixed reality technology, it is being gradually applied in ship production and inspection processes to improve production and inspection efficiency. Current mixed reality applications often require spatial registration of each model to ensure accurate alignment with the physical scene within the work area, enabling a mixed-reality display between virtual models and physical objects. In existing mixed reality registration methods, location markers are typically set at specific locations within the work area, and these markers are associated with location information to facilitate model registration.

[0003] However, since ship products are usually large in size and the lighting environment varies in different areas of the ship, when registering ship models using mixed reality devices, the main method used is to collect positioning marker information using optical acquisition devices such as cameras. However, in areas with insufficient ambient light, or when the markers are blocked by other workpieces, the positioning marker information cannot be collected, thus making it impossible to register ship models using mixed reality.

[0004] Furthermore, since the positioning markers for ship models are usually fixed to the actual production environment of the ship by drawing or pasting the marker images, when the positioning markers become blurred due to aging, or become loose or lost due to reduced adhesion, staff often need to climb or walk back to the original fixing point of the positioning markers to redraw or paste new positioning markers. This makes it impossible to register anytime and anywhere, resulting in an inconvenient and inefficient mixed reality registration process. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a mixed reality registration method, terminal and computer storage medium for ship models, which solves the problems of the existing mixed reality registration process requiring additional positioning markers in the work area, and the inconvenience, low efficiency or even inability to achieve mixed reality registration when the positioning markers are blocked, lost or the ambient light is too weak.

[0006] To achieve the above and other related objectives, the present invention provides a mixed reality registration method for a ship model, applicable to a mixed reality device, comprising: determining a positioning component among physical ship parts within a work area; selecting physical marker points on the positioning component and acquiring spatial distribution information of the physical marker points; the positioning component being a physical ship part used for positioning the ship model; displaying the ship model in the mixed reality device; determining a part model corresponding to the positioning component in the ship model; selecting a model marker point corresponding to the physical marker point on the part model and acquiring spatial distribution information of the model marker point; acquiring a spatial conversion method between the physical marker point and the model marker point based on the spatial distribution information of the physical marker point and the model marker point; and performing a spatial conversion on the ship model based on the spatial conversion method, so that the spatially converted new ship model is registered to the physical ship part.

[0007] In one embodiment of the present invention, the spatial distribution information includes the coordinate information and normal direction of the marker point; the method for obtaining the spatial transformation mode includes: obtaining a coordinate transformation vector between the physical marker point and the model marker point based on the coordinate information of the physical marker point and the coordinate information of the model marker point; obtaining an attitude transformation vector between the physical marker point and the model marker point based on the normal direction of the physical marker point and the normal direction of the model marker point; obtaining a total spatial transformation vector based on the coordinate transformation vector and the attitude transformation vector; and using the total spatial transformation vector as the spatial transformation mode.

[0008] In one embodiment of the present invention, obtaining the total spatial transformation vector based on the coordinate transformation vector and the attitude transformation vector includes: superimposing the coordinate transformation vector and the attitude transformation vector, and using the vector obtained after superposition as the total spatial transformation vector.

[0009] In one embodiment of the present invention, the mixed reality registration method for the ship model further includes: collecting the total matching difference between the new ship model and the actual ship hull, and fine-tuning the spatial distribution of the new ship model by comparing the total matching difference with a preset matching difference threshold.

[0010] In one embodiment of the present invention, the step of collecting the total matching difference between the new ship model and the actual ship hull includes: determining each model sampling point in the ship model, and finding the actual sampling point corresponding to each model sampling point in the actual ship hull; collecting the distribution distance between the model sampling point and the corresponding actual sampling point as the matching difference of the model sampling point; and obtaining the total matching difference between the ship model and the actual ship hull based on the matching difference of each model sampling point.

[0011] In one embodiment of the present invention, the step of fine-tuning the spatial distribution of the new ship model by comparing the total matching difference with a preset matching difference threshold includes: detecting whether the total matching difference is less than the preset matching difference threshold; if so, updating the spatial information of the new ship model according to the total matching difference, updating the display of the ship model based on the updated ship model spatial information, and re-collecting the total matching difference based on the updated ship model until the total matching difference is less than the matching difference threshold.

[0012] In a second aspect, the present invention provides a terminal comprising: a processor and a memory, wherein the memory and the processor are communicatively connected; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the terminal to perform the mixed reality registration method for a ship model as described above.

[0013] In a third aspect, the present invention also provides a computer storage medium storing a computer program, the computer program being executed by a processor of the mixed reality registration method for a ship model as described above.

[0014] As described above, the mixed reality registration method, terminal, and computer storage medium for ship models provided by the present invention, by setting physical identification points and corresponding model identification points, obtains the spatial conversion method between the two based on the spatial distribution information of the physical identification points and the model identification points; based on the spatial conversion method, the ship model is converted into a new ship model so that when the new ship model is displayed in the display unit of the MR device, the ship model can be registered to the ship hull object, thus eliminating the need to set additional positioning identification information in the work area, and achieving mixed registration of ship models quickly and efficiently based solely on the existing ship object. Attached Figure Description

[0015] Figure 1 The diagram shown is a structural schematic of the MR device provided in an embodiment of the present invention.

[0016] Figure 2 The diagram shows a flowchart of an embodiment of the mixed reality registration method for the ship model provided by the present invention.

[0017] Figure 3 The diagram shown is a flowchart of step S400 in one embodiment of the method provided by the present invention.

[0018] Figure 4 The diagram shown is a structural schematic of the terminal provided in an embodiment of the present invention.

[0019] Component designation explanation

[0020] 800MR equipment

[0021] 801SLAM scanning unit

[0022] 802 display unit

[0023] 803 Acquisition Unit Detailed Implementation

[0024] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0025] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention, and are therefore not intended to be considered as drawings.

[0026] Existing mixed reality registration methods require additional positioning markers within the work area, leading to inconvenience and low efficiency. To improve the convenience and efficiency of mixed reality registration methods for ship models, this invention provides a mixed reality registration method for ship models that does not require additional positioning markers. This method is applicable to mixed reality devices (hereinafter referred to as MR devices) and enables spatial registration of ship models within the work area using MR devices.

[0027] Among them, the MR device is as follows Figure 1 As shown, it includes: a SLAM scanning unit 801, a display unit 802, and a data acquisition unit 803; the SLAM scanning unit 801 is used to scan the physical ship hull within the work area to obtain the three-dimensional model information of the physical ship hull within the work area; the display unit 802 is used to display each ship model in three dimensions to generate a mixed reality scene; the data acquisition unit 803 is used to acquire feature information in the mixed reality scene, the feature information including at least length.

[0028] Please see Figure 2 The diagram shows a flowchart of a mixed reality registration method for a ship model provided in an embodiment of the present invention.

[0029] like Figure 2 As shown, the mixed reality registration method for the ship model includes:

[0030] S100, Select a positioning component among the hull components in the work area, select a physical marker point on the positioning component, and obtain the coordinate information and normal direction of the physical marker point;

[0031] The positioning component is a physical hull part used for positioning the ship model; for example, the positioning component includes an elbow plate.

[0032] It should be noted that, in order to improve the accuracy of the ship model's positioning, the actual ship hull part corresponding to the sub-model located in the middle area of ​​the ship model is used as the positioning part.

[0033] Specifically, after initiating the mixed reality registration of the ship model, the MR device acquires the ship model corresponding to the work area, and the ship model includes model information corresponding to the physical ship hull in the work area.

[0034] Staff members wear MR equipment and use the SLAM scanning unit in the MR equipment to scan the work area to obtain three-dimensional model information of the ship hull objects in the work area, and then display the three-dimensional model information in the display unit of the MR equipment.

[0035] Among the various physical components of the ship hull, one of the positioning components is selected, and a physical marker point is selected on the surface of the positioning component through a mixed reality interaction method; in the display unit of the MR device, the coordinate information and normal direction of the physical marker point in the model coordinate system are read.

[0036] Wherein, the normal direction of the physical marker point is the normal direction of the tangent surface of the positioning part at the physical marker point in the model coordinate system.

[0037] In one specific implementation, after the worker selects an elbow plate as the positioning element, they select a physical marker point on the plane through gesture interaction ray or eye tracking ray. Then the MR device reads and records the current position and normal direction of the physical marker point in the model coordinate system.

[0038] S200: Display a ship model in the MR device; determine the part model corresponding to the positioning component in the ship model; select a model marker point corresponding to the physical marker point on the part model; and obtain the coordinate information and normal direction of the model marker point.

[0039] Specifically, staff members use a hybrid display interaction method to select a model identifier point on the part model that corresponds to the physical identifier point, and obtain the coordinate information and normal direction of the model identifier point in the model space coordinate system.

[0040] In one specific implementation, the operator loads and displays the ship model information on the display unit of the MR device via voice commands or gesture interaction. The operator then searches for the part model corresponding to the positioning component among the various part models of the ship model via voice commands or gesture interaction, and highlights the part model on the display unit of the MR device to facilitate the operator's quick location of the part model. The operator then selects the model positioning marker point on the surface of the part model that corresponds to the physical marker point via gesture interaction. The MR device then reads and records the coordinate information and normal direction of the current model marker point in the model coordinate system.

[0041] S300: Based on the coordinate information and normal direction of the physical marker point, and the coordinate information and normal direction of the model marker point, obtain the spatial transformation method between the physical marker point and the model marker point; based on the spatial transformation method, perform spatial transformation on the ship model so that the spatially transformed new ship model is registered to the physical ship hull.

[0042] Specifically, based on the coordinate information of the physical marker point and the coordinate information of the model marker point, a coordinate transformation vector between the physical marker point and the model marker point is obtained, and an attitude transformation vector between the physical marker point and the model marker point is obtained based on the normal direction of the physical marker point and the normal direction of the model marker point. The coordinate transformation vector and the attitude transformation vector are superimposed, and the superimposed total vector is used as the spatial transformation vector between the physical marker point and the model marker point. Based on this spatial transformation vector, the ship model is subjected to model coordinate transformation and model attitude transformation, that is, spatial transformation of the ship model, to obtain new ship model information after spatial transformation. The new ship model information is reloaded and displayed in the display unit of the MR device, so that the new ship model is superimposed on the ship hull, thereby realizing the mixed reality registration of the ship model.

[0043] The ship model mixed reality registration method provided in this embodiment is based on the spatial distribution information of physical marker points on the ship's physical parts and the spatial distribution information of model marker points on the part model. By obtaining the spatial transformation between the two, the ship model is converted into a new ship model. When the new ship model is displayed in the MR device display unit, the ship model can be registered to the ship's physical parts. Therefore, there is no need to set additional positioning marker information in the work area. Based only on the existing ship physical parts, the mixed registration of ship models can be quickly realized, which improves the convenience and flexibility of mixed registration of ship models.

[0044] In another embodiment, the mixed reality registration method for the ship model, after performing step S300, further includes:

[0045] S400, collect the total matching difference between the new ship model and the actual ship hull, and fine-tune the spatial distribution of the new ship model by comparing the total matching difference with a preset matching difference threshold, so that the total matching difference is less than the matching difference threshold.

[0046] The physical sampling point is the corresponding point of the model sampling point on the actual ship component.

[0047] Specifically, when step S400 is executed, such as... Figure 3 As shown, it includes the following sub-steps:

[0048] S401, determine each model sampling point in the ship model, and find the physical sampling point corresponding to each model sampling point in the ship hull physical parts;

[0049] Specifically, after determining each model sampling point in the ship model, based on the positional distribution of the model sampling points relative to the ship model, physical sampling points corresponding to each model sampling point are found in the physical ship hull.

[0050] S402, collect the distribution distance between the model sampling points and the corresponding physical sampling points, as the matching difference of the model sampling points; based on the matching difference of each model sampling point, obtain the total matching between the ship model and the physical ship hull;

[0051] S403, detect whether the total matching difference is less than a preset matching difference threshold. If so, update the spatial information of the new ship model according to the total matching difference, update the display of the ship model in the display unit based on the updated ship model spatial information, and re-execute the above steps based on the updated ship model until the total matching difference is less than the preset matching difference threshold, thereby realizing the registration between the ship model and the ship entity.

[0052] To address the technical problems existing in the prior art, the present invention also provides a terminal, please refer to [link / reference]. Figure 3 The diagram illustrates the structure of a terminal provided in an embodiment of the present invention; as shown below. Figure 3 As shown, the terminal 5 includes a memory 51 and a processor 52 connected to each other; the memory 51 is used to store computer programs, and the processor 52 is used to execute the computer programs stored in the memory so that the terminal can implement the steps in the mixed reality registration method for ship models when it is executed.

[0053] Optionally, the number of memories can be one or more, and the number of processors can be one or more.

[0054] Optionally, the processor in the terminal loads one or more instructions corresponding to application processes into the memory according to the steps in the mixed reality registration method for ship models described above, and the processor runs the application stored in the memory to realize the functions in the mixed reality registration method for ship models described above, which will not be elaborated here.

[0055] It should be noted that memory includes, but is not limited to, random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device. Similarly, processors can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0056] Furthermore, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when invoked by a processor, implements a mixed reality registration method for the ship model.

[0057] A computer-readable storage medium can be a tangible device capable of holding and storing instructions used by an instruction execution device. Computer-readable storage media can be, for example, (but not limited to) electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, and mechanical encoding devices.

[0058] The computer-readable program described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device. A network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network and forwards these instructions to the computer-readable storage medium in the respective computing / processing device.

[0059] In summary, the ship model mixed reality registration method, terminal, and computer storage medium provided by this invention, by setting physical identification points on the physical ship parts and model identification points on the corresponding part models, obtains the spatial conversion method between the two based on the spatial distribution information of the physical identification points and the model identification points; based on the spatial conversion method, the ship model is converted into a new ship model so that when the new ship model is displayed in the MR device display unit, the ship model can be registered to the physical ship parts, thus eliminating the need to set additional positioning identification information in the work area, and achieving rapid mixed registration of ship models based solely on the existing physical ship parts; furthermore, by collecting the total matching difference between the new ship model and the physical ship parts, and by comparing the magnitude of the total matching difference with a preset matching difference threshold, the registration accuracy between the ship model and the physical ship parts can be improved.

[0060] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A mixed reality registration method for ship models, characterized in that, The method, which utilizes a mixed reality device suitable for ship production and inspection scenarios, includes: Identify a positioning component among the physical ship hull parts within the work area; select physical marker points on the positioning component and obtain the spatial distribution information of the physical marker points; the positioning component is a physical ship hull part used for positioning the ship model; A ship model is displayed in the mixed reality device; a part model corresponding to the positioning component is determined in the ship model; a model marker point corresponding to the physical marker point is selected on the part model, and the spatial distribution information of the model marker point is obtained; Based on the spatial distribution information of the physical markers and the model markers, the spatial transformation method between the physical markers and the model markers is obtained; The spatial distribution information includes the coordinates and normal directions of the marker points; the method for obtaining the spatial transformation method includes: Based on the coordinate information of the physical marker point and the coordinate information of the model marker point, obtain the coordinate transformation vector between the physical marker point and the model marker point, and obtain the attitude transformation vector between the physical marker point and the model marker point based on the normal direction of the physical marker point and the normal method of the model marker point; obtain the total spatial transformation vector based on the coordinate transformation vector and the attitude transformation vector; use the total spatial transformation vector as the spatial transformation method; Based on the aforementioned spatial transformation method, the ship model is spatially transformed so that the new ship model after spatial transformation is registered onto the actual ship hull.

2. The mixed reality registration method for ship models according to claim 1, characterized in that, The step of obtaining the total spatial transformation vector based on the coordinate transformation vector and the attitude transformation vector includes: The coordinate transformation vector is superimposed with the attitude transformation vector, and the resulting vector is used as the total spatial transformation vector.

3. The mixed reality registration method for ship models according to claim 1, characterized in that, Also includes: The total matching difference between the new ship model and the actual ship is collected, and the spatial distribution of the new ship model is fine-tuned by comparing the total matching difference with a preset matching difference threshold.

4. The mixed reality registration method for ship models according to claim 3, characterized in that, The collection of the total matching difference between the new ship model and the actual ship hull includes: Each model sampling point is determined in the ship model, and the physical sampling point corresponding to each model sampling point is found in the actual ship hull component; The distribution distance between the model sampling points and the corresponding physical sampling points is collected and used as the matching difference of the model sampling points; Based on the matching difference of each of the model sampling points, the total matching difference between the ship model and the actual ship hull is obtained.

5. The mixed reality registration method for ship models according to claim 3, characterized in that, The step of fine-tuning the spatial distribution of the new ship model by comparing the total matching difference with a preset matching difference threshold includes: If the total matching difference is less than a preset matching difference threshold, the spatial information of the new ship model is updated based on the total matching difference. The display of the ship model is then updated based on the updated spatial information of the ship model. The total matching difference is then re-collected based on the updated ship model until the total matching difference is less than the matching difference threshold.

6. A terminal, characterized in that, include: A processor and a memory, wherein the memory and the processor are communicatively connected; The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to cause the terminal to perform the mixed reality registration method for the ship model as described in any one of claims 1 to 5.

7. A computer storage medium storing a computer program, characterized in that, The computer program is executed by a processor using the mixed reality registration method for the ship model as described in any one of claims 1 to 5.

Images