Point position calibration method and device, server and computer readable storage medium

Abstract

The invention is suitable for the technical field of image processing, and provides a point location calibration method and device, a server and a computer readable storage medium, and the method comprises the steps: obtaining a three-dimensional model, determining an initial point location path between virtual point locations in the three-dimensional model, calibrating the initial point location path, and obtaining a target point location path. Therefore, the point location path between the virtual point locations can be calibrated, so that the point location path between the virtual point locations is reasonable in layout.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular, relates to a point calibration method, device, server, and computer-readable storage medium. Background Art

[0002] 3D reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of 3D objects. It is the basis for processing, operating and analyzing the properties of 3D objects in a computer environment, and is also a key technology for establishing virtual reality in a computer to express the objective world. In the prior art, by generating virtual points in a 3D model, users can navigate the scene through the transition between virtual points. However, the point paths between virtual points are not laid out rationally, and there is a lack of a point calibration method. Summary of the invention

[0003] The embodiments of the present application provide a point calibration method, device, server and storage medium, which can solve the technical problem of unreasonable layout of point paths between virtual points in the prior art.

[0004] In a first aspect, an embodiment of the present application provides a point calibration method, comprising: Obtaining a three-dimensional model; Determining an initial point path between virtual points in the three-dimensional model; The initial point path is calibrated to obtain the target point path.

[0005] In a possible implementation manner of the first aspect, obtaining a three-dimensional model includes: Acquire a panoramic image to be processed, where the panoramic image to be processed is a panoramic image captured by a panoramic camera at different shooting points in a real scene; generating a point cloud according to the panoramic image to be processed; Reconstruct a local three-dimensional model according to the type of the point cloud; The local three-dimensional models are combined to obtain a three-dimensional model, wherein each virtual point in the three-dimensional model corresponds to a shooting point in the real scene.

[0006] In a possible implementation manner of the first aspect, determining an initial point path between virtual points in the three-dimensional model includes: Align the coordinate system between the real scene and the 3D model; Generate virtual coordinates of virtual points in the three-dimensional model according to the real coordinates of the shooting points in the real scene; According to the preset timestamp, an initial point path is planned according to the virtual coordinates corresponding to the virtual points, and the initial point path includes the virtual points and the connection relationship between the virtual points.

[0007] In a possible implementation manner of the first aspect, calibrating the initial point path to obtain the target point path includes: Traverse each virtual point according to the initial point path; The virtual points and the connection relationship between the virtual points are adjusted according to a preset calibration strategy to generate a target point path.

[0008] In a possible implementation manner of the first aspect, calibrating the initial point path to obtain the target point path includes: Display virtual points according to the initial point path, and the connection relationship between virtual points to the user; A calibration instruction from a user is received, and a calibration action corresponding to the calibration instruction is executed to adjust the virtual points and the connection relationship between the virtual points to obtain a target point path.

[0009] In a second aspect, an embodiment of the present application provides a point calibration device, comprising: An acquisition module, used for acquiring a three-dimensional model; A determination module, used to determine an initial point path between virtual points in the three-dimensional model; The calibration module is used to calibrate the initial point path to obtain the target point path.

[0010] In a possible implementation manner of the second aspect, the acquiring module includes: An acquisition submodule is used to acquire a panoramic image to be processed, where the panoramic image to be processed is a panoramic image captured by a panoramic camera at different shooting points in a real scene; A generating submodule, used for generating a point cloud according to the panoramic image to be processed; A reconstruction submodule, used to reconstruct a local three-dimensional model according to the type of the point cloud; The merging submodule is used to merge the local three-dimensional models to obtain a three-dimensional model; wherein each virtual point in the three-dimensional model corresponds to a shooting point in the real scene.

[0011] In a possible implementation manner of the second aspect, the determining module includes: An alignment submodule, used to align the coordinate system between the real scene and the 3D model; A generation submodule, used to generate virtual coordinates of virtual points in the three-dimensional model according to the real coordinates of the shooting points in the real scene; The planning submodule is used to plan an initial point path according to the virtual coordinates corresponding to the virtual points according to a preset timestamp, wherein the initial point path includes the virtual points and the connection relationship between the virtual points.

[0012] In a possible implementation manner of the second aspect, the calibration module includes: The traversal submodule is used to traverse each virtual point according to the initial point path; The first adjustment submodule is used to adjust the virtual points and the connection relationship between the virtual points according to a preset calibration strategy to generate a target point path.

[0013] In a possible implementation manner of the second aspect, the calibration module includes: A display submodule is used to display virtual points according to the initial point path, as well as the connection relationship between the virtual points to the user; The second adjustment submodule receives a calibration instruction from a user, executes a calibration action corresponding to the calibration instruction to adjust the virtual points and the connection relationship between the virtual points, and obtains a target point path.

[0014] In a third aspect, an embodiment of the present application provides a server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present application provides a readable storage medium, and when the computer program is executed by a processor, the method described in the first aspect above is implemented.

[0016] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: In the embodiment of the present application, a three-dimensional model is obtained, an initial point path between virtual points in the three-dimensional model is determined, the initial point path is calibrated, and a target point path is obtained. It can be seen that the embodiment of the present application can calibrate the point path between virtual points so that the point path layout between virtual points is reasonable. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 is a flow chart of a point calibration method provided in an embodiment of the present application; Figure 2 is a structural block diagram of a point calibration device provided in an embodiment of the present application; Figure 3 It is a schematic diagram of the structure of the server provided in the embodiment of the present application. Implementation

[0019] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0020] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0021] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0022] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0023] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0025] The technical solution provided by the embodiments of the present application will be introduced below through specific examples.

[0026] See also Figure 1 , is a flow chart of a point calibration method provided in an embodiment of the present application. As an example but not a limitation, the method can be applied to a server. The method may include the following steps: Step S101, obtaining a three-dimensional model.

[0027] In specific applications, obtaining a three-dimensional model includes: Step S201, obtaining a panoramic image to be processed.

[0028] The panoramic images to be processed are panoramic images captured by a panoramic camera at different shooting points in a real scene.

[0029] It should be noted that the panoramic camera can be an octagonal camera or a laser camera, wherein the octagonal camera consists of two groups of upper and lower groups, each group of four fisheye lenses, and the four lenses respectively collect four groups of lens images and stitch them into a 360° panoramic image; the laser camera consists of three laser radars and one spherical screen camera, and the relative positions between the three laser radars and the spherical screen camera are fixed. The laser radars include a first laser radar (i.e., UpLidar laser radar), a second laser radar (i.e., MidLidar laser radar), and a third laser radar (i.e., DownLidar laser radar) with relatively fixed positions. Exemplarily, the UpLidar laser radar, the MidLidar laser radar, and the DownLidar laser radar are arranged at intervals in the order of top, middle, and bottom on the same vertical line, and the laser camera rotates 6 times at a rotation angle of 60 degrees at the same point to finally obtain a 360-degree panoramic image.

[0030] Step S202: generating a point cloud according to the panoramic image to be processed.

[0031] In a specific application, depth estimation is performed on the panoramic image to be processed to obtain depth information; and a point cloud is obtained according to the depth information.

[0032] Step S203: reconstructing a local three-dimensional model according to the type of the point cloud.

[0033] In a specific application, the three-dimensional coordinates of the point cloud are input into a pre-trained point cloud recognition model to obtain the type of the point cloud; and a local three-dimensional model is reconstructed according to the type of the point cloud.

[0034] Step S204: merge the local three-dimensional models to obtain a three-dimensional model.

[0035] Each virtual point in the three-dimensional model corresponds to a shooting point in the real scene.

[0036] In specific applications, a first registration is performed on the point clouds in the local three-dimensional model according to a preset ICP algorithm to obtain the local three-dimensional model after the first registration; the edge point cloud between the local three-dimensional models after the first registration is determined; a second registration is performed on the edge point cloud according to a pre-trained registration neural network model; and a global three-dimensional model is formed based on the local three-dimensional model after the first registration and the edge point cloud after the second registration.

[0037] Step S102, determining an initial point path between virtual points in the three-dimensional model.

[0038] Exemplarily, determining an initial point path between virtual points in a three-dimensional model includes: Step S201, aligning the coordinates between the real scene and the three-dimensional model.

[0039] Step S202, generating virtual coordinates of the virtual point in the three-dimensional model according to the real coordinates of the shooting point in the real scene.

[0040] Step S203, according to the preset timestamp, an initial point path is planned according to the virtual coordinates corresponding to the virtual points, and the initial point path includes the virtual points and the connection relationship between the virtual points.

[0041] The preset timestamp may be the time sequence in which the panoramic camera captures the panoramic images at different shooting points in the real scene.

[0042] Step S103, calibrating the initial point path to obtain the target point path.

[0043] Exemplarily, calibrating the initial point path to obtain the target point path includes: Step S301, traverse each virtual point according to the initial point path.

[0044] Step S302: adjusting the virtual points and the connection relationship between the virtual points according to a preset calibration strategy to generate a target point path.

[0045] The preset calibration strategy includes that the distance between virtual points cannot be less than a preset distance, such as 5 meters.

[0046] Optionally, calibrate the initial point path to obtain the target point path, including: Step S401, displaying virtual points and connection relationships between virtual points to the user according to the initial point path.

[0047] Step S402, receiving a calibration instruction from a user, executing a calibration action corresponding to the calibration instruction to adjust the virtual points and the connection relationship between the virtual points, and obtaining a target point path.

[0048] The calibration instructions include adding or deleting virtual points, and adding or deleting lines between virtual points.

[0049] It can be understood that the embodiments of the present application are displayed to the user through a top view, a side view or a 3D view, so that the user can add or delete virtual points, and add or delete lines between virtual points.

[0050] In the embodiment of the present application, a three-dimensional model is obtained, an initial point path between virtual points in the three-dimensional model is determined, the initial point path is calibrated, and a target point path is obtained. It can be seen that the embodiment of the present application can calibrate the point path between virtual points so that the point path layout between virtual points is reasonable.

[0051] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0052] Corresponding to the method described in the above embodiment, Figure 2 A structural block diagram of a point calibration device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0053] Reference Figure 2 , the device comprises: An acquisition module 21, used for acquiring a three-dimensional model; A determination module 22, used to determine an initial point path between virtual points in the three-dimensional model; The calibration module 23 is used to calibrate the initial point path to obtain the target point path.

[0054] In a possible implementation, obtaining a module includes: An acquisition submodule is used to acquire a panoramic image to be processed, where the panoramic image to be processed is a panoramic image captured by a panoramic camera at different shooting points in a real scene; A generating submodule, used for generating a point cloud according to the panoramic image to be processed; A reconstruction submodule, used to reconstruct a local three-dimensional model according to the type of the point cloud; The merging submodule is used to merge the local three-dimensional models to obtain a three-dimensional model; wherein each virtual point in the three-dimensional model corresponds to a shooting point in the real scene.

[0055] In a possible implementation, the determining module includes: An alignment submodule, used to align the coordinate system between the real scene and the 3D model; A generation submodule, used to generate virtual coordinates of virtual points in the three-dimensional model according to the real coordinates of the shooting points in the real scene; The planning submodule is used to plan an initial point path according to the virtual coordinates corresponding to the virtual points according to a preset timestamp, wherein the initial point path includes the virtual points and the connection relationship between the virtual points.

[0056] In a possible implementation, the calibration module includes: The traversal submodule is used to traverse each virtual point according to the initial point path; The first adjustment submodule is used to adjust the virtual points and the connection relationship between the virtual points according to a preset calibration strategy to generate a target point path.

[0057] In a possible implementation, the calibration module includes: A display submodule is used to display virtual points according to the initial point path, as well as the connection relationship between the virtual points to the user; The second adjustment submodule receives a calibration instruction from a user, executes a calibration action corresponding to the calibration instruction to adjust the virtual points and the connection relationship between the virtual points, and obtains a target point path.

[0058] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0059] Figure 3 This is a schematic diagram of the structure of the server provided in the embodiment of the present application. Figure 3 As shown, the server 3 of this embodiment includes: at least one processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, and when the processor 30 executes the computer program 32, the steps in any of the above-mentioned method embodiments are implemented.

[0060] The server 3 may be a computing device such as a cloud server. The server may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3 It is only an example of server 3 and does not constitute a limitation on server 3. Server 3 may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0061] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0062] In some embodiments, the memory 31 may be an internal storage unit of the server 3, such as a hard disk or memory of the server 3. In other embodiments, the memory 31 may also be an external storage device of the server 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the server 3. Further, the memory 31 may also include both an internal storage unit of the server 3 and an external storage device. The memory 31 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0063] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0064] An embodiment of the present application further provides a computer-readable storage medium, which is preferably a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0065] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the server, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0066] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0067] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0068] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0069] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0070] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A point calibration method, characterized in that: include: Obtaining a three-dimensional model; Determining an initial point path between virtual points in the three-dimensional model; The initial point path is calibrated to obtain the target point path.

2. The point calibration method according to claim 1, characterized in that: Get 3D models, including: Acquire a panoramic image to be processed, wherein the panoramic image to be processed is a panoramic image captured by a panoramic camera at different shooting points in a real scene; Generate a point cloud according to the panoramic image to be processed; Reconstruct a local three-dimensional model according to the type of the point cloud; The local three-dimensional models are combined to obtain a three-dimensional model, wherein each virtual point in the three-dimensional model corresponds to a shooting point in the real scene.

3. The point calibration method according to claim 2, characterized in that: Determining an initial point path between virtual points in the three-dimensional model includes: Align the coordinate system between the real scene and the 3D model; Generate virtual coordinates of virtual points in the three-dimensional model according to the real coordinates of the shooting points in the real scene; According to the preset timestamp, an initial point path is planned according to the virtual coordinates corresponding to the virtual points; wherein the initial point path includes the virtual points and the connection relationship between the virtual points.

4. The point calibration method according to any one of claims 1 to 3, characterized in that: Calibrating the initial point path to obtain the target point path includes: Traverse each virtual point according to the initial point path; The virtual points and the connection relationship between the virtual points are adjusted according to a preset calibration strategy to generate a target point path.

5. The point calibration method according to any one of claims 1 to 3, characterized in that: Calibrating the initial point path to obtain the target point path includes: Display virtual points according to the initial point path, and the connection relationship between virtual points to the user; A calibration instruction from a user is received, and a calibration action corresponding to the calibration instruction is executed to adjust the virtual points and the connection relationship between the virtual points to obtain a target point path.

6. A point calibration device, characterized in that: include: An acquisition module, used for acquiring a three-dimensional model; A determination module, used to determine an initial point path between virtual points in the three-dimensional model; The calibration module is used to calibrate the initial point path to obtain the target point path.

7. The point calibration device according to claim 6, characterized in that: Get modules, including: An acquisition submodule is used to acquire a panoramic image to be processed, where the panoramic image to be processed is a panoramic image captured by a panoramic camera at different shooting points in a real scene; A generating submodule, used for generating a point cloud according to the panoramic image to be processed; A reconstruction submodule, used to reconstruct a local three-dimensional model according to the type of the point cloud; The merging submodule is used to merge the local three-dimensional models to obtain a three-dimensional model; wherein each virtual point in the three-dimensional model corresponds to a shooting point in the real scene.

8. The point calibration device according to claim 7, characterized in that: The determining module comprises: An alignment submodule, used to align the coordinate system between the real scene and the 3D model; A generation submodule, used to generate virtual coordinates of virtual points in the three-dimensional model according to the real coordinates of the shooting points in the real scene; The planning submodule is used to plan an initial point path according to the virtual coordinates corresponding to the virtual points according to a preset timestamp, wherein the initial point path includes the virtual points and the connection relationship between the virtual points.

9. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images