Panoramic roaming scene construction method, device, computer equipment, and storage medium

Abstract

The present disclosure relates to a method, device, computer equipment, and storage medium for constructing a panoramic roaming scene, and relates to the field of artificial intelligence virtual reality technology. The method includes: calculating the coordinate deviation rate between a roaming panorama and a three-dimensional panorama, wherein the roaming panorama is obtained by inputting the three-dimensional panorama into a panoramic roaming production platform; adjusting the roaming panorama according to the coordinate deviation rate to obtain a calibrated panorama; constructing a project model according to the calibrated panorama and the panoramic roaming production platform; adjusting the point spatial data of the project model according to the coordinate deviation rate to obtain a calibrated model; integrating and connecting the calibrated panorama and the calibrated model to obtain a panoramic roaming scene. The use of this method can ensure the accuracy of spatial coordinates during panoramic roaming, and smoothly browse the scene.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence virtual reality technology, and in particular to a method, device, computer equipment, and storage medium for constructing a panoramic roaming scene. Background Art

[0002] With the continuous improvement of virtual reality technology, computer-aided software is being used to build virtual three-dimensional spaces, simulating real-world display environments and providing visually immersive browsing and intuitive, vivid interactive experiences. Using panoramic images to build virtual 3D scenes helps computers save significant operating costs, allowing for faster loading of scene content and timely display of spatial effects.

[0003] Panoramic images are currently primarily used in panoramic tour creation tools. These images are imported into a panoramic software production platform and then displayed on the web. However, the finished product currently suffers from numerous visual issues that impact the user experience. Currently, after generating a corresponding project model using a panoramic tour platform, when performing a panoramic tour through the project model, point coordinates can shift significantly when switching between panoramas. This lacks accuracy in spatial coordinates and can cause noticeable image skew, impacting visual browsing. Summary of the Invention

[0004] Based on this, it is necessary to provide a panoramic roaming scene construction method, device, computer equipment, and storage medium that can ensure the accuracy of spatial coordinates and smoothly browse scenes during panoramic roaming to address the above technical problems.

[0005] In a first aspect, the present disclosure provides a method for constructing a panoramic roaming scene, the method comprising:

[0006] Calculating a coordinate deviation rate between a roaming panorama and a three-dimensional panorama, wherein the roaming panorama is obtained by inputting the three-dimensional panorama into a panoramic roaming production platform;

[0007] adjusting the roaming panoramic image according to the coordinate deviation rate to obtain a calibrated panoramic image;

[0008] Building a project model based on the calibrated panorama and the panoramic tour production platform;

[0009] Adjusting the point spatial data of the project model according to the coordinate deviation rate to obtain a calibration model;

[0010] The calibrated panorama and the calibrated model are integrated and connected to obtain a panoramic roaming scene.

[0011] In one embodiment, calculating the coordinate deviation rate between the roaming panoramic image and the three-dimensional panoramic image includes:

[0012] Obtaining a first position difference value of a preset position in the roaming panoramic image and a second position difference value of a preset position in the three-dimensional panoramic image, wherein there are at least two preset positions, and each of the preset positions has the same vertical coordinate position;

[0013] Calculating a coordinate difference ratio according to the first position difference and the second position difference, wherein the coordinate difference ratio includes a first coordinate axis difference ratio and a second coordinate axis difference ratio;

[0014] The coordinate deviation rate between the roaming panoramic image and the three-dimensional panoramic image is calculated according to the coordinate difference ratio, and the coordinate deviation rate includes a first deviation rate of the first coordinate axis and a second deviation rate of the second coordinate axis.

[0015] In one embodiment, adjusting the roaming panoramic image according to the coordinate deviation rate to obtain a calibrated panoramic image includes:

[0016] Adjust each coordinate point in the roaming panoramic image according to the first deviation rate and the second deviation rate to obtain a calibrated coordinate value;

[0017] A calibrated panorama is determined according to the calibrated coordinate values.

[0018] In one embodiment, adjusting the point spatial data of the project model according to the coordinate deviation rate to obtain a calibration model includes:

[0019] The project model is simplified, and the simplification includes: reducing the number of faces of the project model without affecting the structure of the project model.

[0020] In one embodiment, adjusting the point spatial data of the project model according to the coordinate deviation rate to obtain a calibration model includes:

[0021] Adjusting the point spatial data in the project model according to the first deviation rate and the second deviation rate to obtain calibrated spatial point data;

[0022] A calibration model is obtained according to the calibration space point data.

[0023] In one embodiment, integrating and connecting the calibrated panoramic image and the calibrated model to obtain a panoramic roaming scene includes:

[0024] The calibrated panorama and the calibrated model are integrated and connected by image stitching technology to obtain a panoramic roaming scene. The image stitching technology includes: image registration and image fusion. The image registration algorithm adopts a region-based method.

[0025] In one embodiment, the roaming panorama, the three-dimensional panorama, and the calibrated panorama are all spherical panoramas.

[0026] In a second aspect, the present disclosure further provides a device for constructing a panoramic roaming scene, the device comprising:

[0027] a calculation module for calculating a coordinate deviation rate between a roaming panorama and a three-dimensional panorama, wherein the roaming scene is obtained by inputting the three-dimensional scene into a panoramic roaming production platform;

[0028] A panoramic image calibration module, configured to adjust the roaming panoramic image according to the coordinate deviation rate to obtain a calibrated panoramic image;

[0029] A model building module, configured to build a project model based on the calibrated panorama and the panoramic roaming production platform;

[0030] A model calibration module, configured to adjust the point spatial data of the project model according to the coordinate deviation rate to obtain a calibration model;

[0031] The integration module is used to integrate and connect the calibrated panoramic image and the calibrated model to obtain a panoramic roaming scene.

[0032] In one embodiment of the device, the calculation module includes: a coordinate difference acquisition module, a difference ratio calculation module, and a deviation rate calculation module;

[0033] The coordinate difference acquisition module is configured to acquire a first position difference of a preset position in the roaming panoramic image and a second position difference of a preset position in the three-dimensional panoramic image, wherein there are at least two preset positions, and each of the preset positions has the same vertical coordinate position;

[0034] The difference ratio calculation module is configured to calculate a coordinate difference ratio based on the first position difference and the second position difference, wherein the coordinate difference ratio includes a first coordinate axis difference ratio and a second coordinate axis difference ratio;

[0035] The deviation rate calculation module is used to calculate the coordinate deviation rate between the roaming panoramic image and the three-dimensional panoramic image according to the coordinate difference ratio, and the coordinate deviation rate includes a first deviation rate of the first coordinate axis and a second deviation rate of the second coordinate axis.

[0036] In one embodiment of the device, the panoramic image calibration module includes: a deviation rate adjustment module, a coordinate value determination module;

[0037] The deviation rate adjustment module is configured to adjust each coordinate point in the roaming panoramic image according to the first deviation rate and the second deviation rate to obtain a calibrated coordinate value;

[0038] The coordinate value determination module is used to determine the calibration panorama according to the calibration coordinate values.

[0039] In one embodiment of the device, the device further includes: a simplification processing module for simplifying the project model, and the simplification processing includes: reducing the number of faces of the project model without affecting the structure of the project model.

[0040] In one embodiment of the device, the model calibration module includes: a data adjustment module, a model determination module;

[0041] The data adjustment module is configured to adjust the point spatial data in the project model according to the first deviation rate and the second deviation rate to obtain calibrated spatial point data;

[0042] The model determination module is used to obtain a calibration model according to the calibration space point data.

[0043] In one embodiment of the device, the integration module is further used to integrate and connect the calibrated panorama and the calibration model through image stitching technology to obtain a panoramic roaming scene. The image stitching technology includes: image registration and image fusion, and the image registration algorithm adopts a region-based method.

[0044] In a third aspect, the present disclosure further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0045] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.

[0046] In a fifth aspect, the present disclosure further provides a computer program product, comprising a computer program that implements the steps of the above method when executed by a processor.

[0047] In each of the above embodiments, the coordinate deviation rate between the roaming panorama and the three-dimensional panorama is calculated, and the roaming panorama is adjusted according to the coordinate deviation rate to obtain a calibrated panorama, ensuring that the calibrated panorama is closer to the actual scene, so that the coordinates at the 2D level will not deviate. In addition, by adjusting the point spatial data of the project model through the coordinate deviation rate, the project model can be integrated at the 3D level, and the accuracy of the point spatial data can be achieved at the 3D level, ensuring that the coordinates at the 3D level will not deviate. Then, by combining the 2D-level calibrated panorama and the 3D-level calibrated model, the panoramic roaming scene points obtained will not be offset. The panoramic roaming scene finally obtained can be browsed smoothly without obstacles and offset, achieving the experience of free walking in the virtual space. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 A schematic diagram of an application environment of a method for constructing a panoramic roaming scene in one embodiment;

[0050] Figure 2 A schematic diagram of a flow chart of a method for constructing a panoramic roaming scene in one embodiment;

[0051] Figure 3 Schematic diagram of the process of step S20 in one embodiment;

[0052] Figure 4 Schematic diagram of the process of step S50 and steps before step S50 in one embodiment;

[0053] Figure 5 A flowchart of a method for constructing a panoramic roaming scene in another embodiment;

[0054] Figure 6 A schematic block diagram of the structure of a panoramic roaming scene construction device in one embodiment;

[0055] Figure 7 Schematic diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not intended to limit the present disclosure.

[0057] It should be noted that the terms "first," "second," and the like in the specification and claims herein and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0058] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" could mean: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the related objects.

[0059] In this disclosure, a panorama, typically a wide-angle image, can exist in the form of a painting, photograph, video, or 3D model. Modern panoramas use wide-angle techniques and forms such as paintings, photographs, videos, and 3D models to depict as much of the surrounding environment as possible. A 360-degree panorama is a process of capturing the entire scene using a professional camera or rendering an image using modeling software, stitching the images together using software, and playing them with a dedicated player. This process converts a two-dimensional photo or computer-generated image into a 360-degree panoramic view for virtual reality viewing, simulating a two-dimensional image into a real three-dimensional space and presenting it to the viewer.

[0060] As mentioned in the background, with the development of virtual reality technology, VR is breaking the boundaries of time and space, a technologically innovative application of the technology. Simultaneously, panoramic image generation technologies are becoming increasingly diverse and widespread. Currently, virtual spaces presented via the web or HTML5 can be broadly categorized into two types. The first is the spatial presentation of a real scene. The other is the spatial presentation of a fictional scene. While both methods share the same presentation format, the content is produced by different companies. For virtual spaces, the production of corresponding panoramic images is more complex and takes longer. However, this facilitates later scene modifications.

[0061] Typically, the required panoramas can be generated using computer graphics software, such as 3D modeling software. Within the 3D modeling software, a three-dimensional spatial model is constructed, rendering requirements and panorama parameters are set, and a rendering calculation is performed to generate a panorama of the corresponding space. This method uses a computer to create a panorama corresponding to a virtual scene. The panorama corresponding to the real scene is captured from multiple angles using a camera. The data is collected, transmitted, and processed using panoramic camera technology, and ultimately displayed on a terminal device.

[0062] Panoramic virtual display spaces combine two-dimensional panoramas (IBR) created through image rendering with three-dimensional models based on these panoramas. Using image stitching technology, these interrelated images are integrated and connected, enabling 360-degree, surround-view viewing of the scene. Since the normal human eye's field of view typically ranges from 90° horizontally and 70° vertically, a comprehensive view of the scene requires a 360° horizontal and 180° vertical perspective. This creates a sense of being immersed in the virtual display.

[0063] Panoramic images are currently primarily used in panoramic tour creation tools. Current panoramic tour creation tools include Panno2VR, Krpano, and everpano. Krpano's open-source code provides developers with an easily extensible interface. Many panoramic service software applications have been developed based on the Krpano core, making the interface visual and user-friendly, reducing the learning curve. However, these tools also have limitations. Panoramic creation tools with visual interfaces eliminate the need for remodeling the panorama and allow direct link publishing. However, these tools suffer from abrupt transitions between panoramas, resulting in a poor immersive experience and choppy browsing. These panoramas are still essentially two-dimensional and lack three-dimensional spatial information, making them unrealistically 3D scenes, let alone the ability to freely navigate within them. Furthermore, these visually accessible panoramic tour systems can only be controlled via touch, mouse, or keyboard, and the switching between scenes is somewhat different from the actual free-flowing experience.

[0064] Therefore, in order to solve the above problems, the present disclosure provides a virtual space display method, which can be applied to Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store data that server 104 needs to process. The data storage system can be integrated with server 104, or located in the cloud or on another network server. 3D modeling software can be installed on server 104 or terminal 102, and used to output a 3D panoramic image. The 3D panoramic image is input into a panoramic roaming creation platform on terminal 102 or server 104, and then processed by the roaming creation platform to generate a roaming panoramic image. Terminal 102 can calculate the coordinate deviation rate between the roaming panoramic image and the 3D panoramic image. The roaming panoramic image is then adjusted based on the coordinate deviation rate to obtain a calibrated panoramic image. Terminal 102 adjusts the spatial data of the project model's points based on the coordinate deviation rate to obtain a calibrated model. Terminal 102 integrates the calibrated model and the calibrated panoramic image to obtain a panoramic roaming scene. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers. It should be noted that this solution may also be used for the terminal 102 or the server 104 alone.

[0065] In one embodiment, Figure 2 As shown, a method for constructing a panoramic roaming scene is provided, which is applied to Figure 1 Taking the terminal 102 in FIG. 1 as an example, the method includes the following steps:

[0066] S20, calculating a coordinate deviation rate between the roaming panorama and the three-dimensional panorama, wherein the roaming panorama is obtained by inputting the three-dimensional panorama into a panoramic roaming production platform.

[0067] 3D panoramas are typically generated using 3D modeling software, which is easier to modify than panoramas captured with panoramic cameras. Roaming panoramas are typically created by processing 3D panoramas using a panoramic roaming production platform, typically Everpano 3D. Everpano 3D is typically developed to build 3D models for Krpano roaming scenes. Its post-model roaming functionality is highly integrated with Krpano 1.20's Depthmap technology. This allows reverse modeling and mapping of Krpano panoramic scene spaces without the need for stereo scanner data, transforming Krpano panoramic web projects into 3D models while achieving highly accurate and smooth forward roaming. Everpano's implementation principle is to manually outline the elevations of interior spaces and objects by adding lines and geometric shapes. Based on the size and relative position of these outlines, the virtual space model is constructed, direction is determined, and roaming is enabled. The coordinate deviation rate is usually the deviation between the 3D modeling software and the panoramic roaming production platform, and can usually be regarded as the difference ratio between the 3D modeling software and the panoramic roaming production platform. In this embodiment, the Everpano 3D platform and the Everpano platform are the same platform.

[0068] Specifically, because the 3D modeling software and the panoramic roaming production platform operate on different software platforms, data derived from the two platforms often deviate. Therefore, it is necessary to determine the deviation between the 3D modeling software and the panoramic roaming production platform. To do this, a 3D panorama created based on the actual scene using the 3D modeling software is input into the panoramic roaming production platform. The roaming panorama is then input from the roaming panorama production platform. The coordinate deviation rate between the roaming panorama and the 3D panorama is then calculated.

[0069] S30: Adjust the roaming panoramic image according to the coordinate deviation rate to obtain a calibrated panoramic image.

[0070] Specifically, the coordinate values ​​of the roaming panorama in the roaming panorama production platform are redefined according to the coordinate deviation rate to obtain a calibrated panorama, which is usually closer to the actual scene.

[0071] S40: Build a project model based on the calibrated panoramic image and the panoramic tour production platform.

[0072] Among them, the project model can usually be a model of the display space, usually a three-dimensional scene.

[0073] Specifically, coordinates can be collected based on the obtained calibration panorama to obtain the specific coordinate points of the calibration panorama in space, and multi-point paths can be drawn continuously. Then, a panoramic roaming production platform can be used to manually outline the three-dimensional model based on the above information by pulling lines. Based on the two-dimensional plane (calibration panorama), the three-dimensional space modeling is reversely constructed to produce the project model.

[0074] It should be noted that the above is only carried out in a manual wire-drawing modeling manner. Those skilled in the art can choose other methods to produce the project model according to the actual situation based on the calibrated panoramic map and the panoramic roaming production platform. The specific modeling method is not limited in this embodiment.

[0075] S50: adjusting the point spatial data of the project model according to the coordinate deviation rate to obtain a calibration model.

[0076] Specifically, although the project model constructed using the calibration panorama may not match the actual model, and the output model data may be inaccurate and imprecise. Therefore, it is necessary to adjust the project model using the coordinate deviation rate to make it more accurate. The adjusted project model can be used as the calibration model.

[0077] S60: Integrate and connect the calibrated panoramic image and the calibrated model to obtain a panoramic roaming scene.

[0078] Specifically, the calibrated panorama and the calibrated model can be integrated and connected to obtain a panoramic roaming scene, which can maximize the immersive roaming experience.

[0079] In the above-mentioned panoramic roaming scene construction method, the coordinate deviation rate between the roaming panorama and the three-dimensional panorama is calculated, the roaming panorama is re-optimized at the 2D level, and the roaming panorama is adjusted according to the coordinate deviation rate to obtain a calibrated panorama, ensuring that the calibrated panorama is closer to the actual scene, so that the coordinates at the 2D level will not deviate. In addition, by adjusting the point spatial data of the project model through the coordinate deviation rate, the project model can be integrated at the 3D level, and the accuracy of the point spatial data can be achieved at the 3D level, ensuring that the coordinates at the 3D level will not deviate. Then, by combining the 2D level calibrated panorama and the 3D level calibrated model, there will be no offset between the points of the panoramic roaming scene. It can be browsed smoothly without obstacles and offsets. The experience of free walking in the virtual space is achieved.

[0080] In one embodiment, Figure 3 As shown, the step S20 of calculating the coordinate deviation rate between the roaming panoramic image and the three-dimensional panoramic image includes:

[0081] S22: Obtain a first position difference value of a preset position in the roaming panoramic image and a second position difference value of a preset position in the three-dimensional panoramic image, wherein there are at least two preset positions, and each of the preset positions has the same vertical coordinate position.

[0082] The preset position may generally be a coordinate position determined by those skilled in the art in the art in the roaming panorama and the three-dimensional panorama according to actual conditions, and there should be at least two preset positions, so as to facilitate calculation of the position difference.

[0083] Specifically, a first position difference between at least two preset positions in the roaming panoramic image and a second position difference between at least two preset positions in the three-dimensional panoramic image are obtained. It should be noted that the preset positions in the roaming panoramic image should correspond to the preset positions in the three-dimensional panoramic image so that the deviation rate can be ultimately calculated. Panoramic images are typically 360-degree horizontal panoramas, so in this embodiment, the vertical coordinate positions of the preset positions can be set to be the same, and the position difference between the horizontal positions can be calculated.

[0084] Typically, the preset position can be represented by three-dimensional coordinates. The three-dimensional coordinates can typically be represented by the horizontal x-axis, the vertical y-axis, and the vertical z-axis. The three-dimensional coordinate corresponding to the z-axis can typically be considered the vertical coordinate position. The first position difference and the second position difference typically include the difference between the positions corresponding to the horizontal x-axis and the vertical y-axis.

[0085] In some exemplary embodiments, if there are three preset positions, the corresponding coordinates in the three-dimensional panoramic image are P1 (x1, y1, z1), P2 (x2, y2, z2), and P3 (x3, y3, z3). Here, z1 = z2 = z3. The differences between the horizontal x-axis and the vertical y-axis of the three positions P1, P2, and P3 are calculated as follows: x1-x2, x1-x3, x2-x3, y1-y2, y1-y3, and y2-y3. These differences in the three-dimensional panoramic image may be collectively referred to as second position differences. The corresponding coordinates of the three preset positions in the roaming panoramic map are P1"(x1", y1", z1"), P2"(x2", y2", z2"), and P3(x3", y3", z3"). The differences between the horizontal x-axis and the vertical y-axis of the three positions P1", P2", and P3" are calculated as x1"-x2", x1"-x3", x2"-x3", y1"-y2", y1"-y3", and y2"-y3". The above-mentioned differences in the roaming panoramic map can be collectively referred to as the first position differences.

[0086] It should be noted that only three preset positions are used as an example here. In actual practice, those skilled in the art may select two or more preset positions for calculation, as long as the position difference of the corresponding panoramic image can be calculated.

[0087] S24 , calculating a coordinate difference according to the first position difference and the second position difference, wherein the coordinate difference ratio includes a first coordinate axis difference ratio and a second coordinate axis difference ratio.

[0088] The coordinate difference ratio may generally be a difference ratio between the three-dimensional panoramic image and the roaming panoramic image relative to the coordinate axis.

[0089] Specifically, the first position difference can be compared with the second position difference to obtain a coordinate difference. Furthermore, because the first position difference and the second position difference typically include the difference between positions corresponding to the horizontal x-axis and the vertical y-axis, the difference between the positions corresponding to the horizontal x-axis in the first position difference and the second position difference can be compared to obtain a first coordinate axis difference ratio. The difference between the positions corresponding to the vertical y-axis in the first position difference and the second position difference can be compared to obtain a second coordinate axis difference ratio.

[0090] In some exemplary embodiments, as described above, the second position difference is: x1-x2, x1-x3, x2-x3, y1-y2, y1-y3, y2-y3. The first position difference is: x1"-x2", x1"-x3", x2"-x3", y1"-y2", y1"-y3", y2"-y3". The corresponding first coordinate axis difference ratios may include: (x1"-x2") / (x1-x2)=a0, (x1"-x3") / (x1-x3)=a1, (x2"-x3") / (x2-x3)=a3; or, (x1-x2) / (x1"-x2")=a0, (x1-x3) / (x1"-x3")=a1, (x2-x3) / (x2"-x3")=a2. The second coordinate axis difference ratio may include: (y1"-y2") / (y1-y2)=b0, (y1"-y3") / (y1-y3)=b1, (y2"-y3") / (y2-y3)=b2; or, (y1-y2) / (y1"-y2")=b0, (y1-y3) / (y1"-y3")=b1, (y2-y3) / (y2"-y3")=b2.

[0091] S26 , calculating a coordinate deviation rate between the roaming panoramic image and the three-dimensional panoramic image according to the coordinate difference ratio, wherein the coordinate deviation rate includes a first deviation rate of a first coordinate axis and a second deviation rate of a second coordinate axis.

[0092] Specifically, if the first coordinate axis difference ratio is a single term, the first deviation rate of the first coordinate axis between the roaming panoramic image and the three-dimensional panoramic image can be calculated based on the first coordinate axis difference ratio. If the first coordinate axis difference ratio is multiple terms, the first deviation rate of the first coordinate axis between the roaming panoramic image and the three-dimensional panoramic image can be calculated based on each first coordinate axis difference ratio and the corresponding number of terms.

[0093] If the second coordinate axis difference ratio is a single term, the second deviation rate of the second coordinate axis between the roaming panoramic image and the three-dimensional panoramic image can be calculated based on the second coordinate axis difference ratio. If the second coordinate axis difference ratio is multiple terms, the second deviation rate of the second coordinate axis between the roaming panoramic image and the three-dimensional panoramic image can be calculated based on each second coordinate axis difference ratio and the corresponding number of terms.

[0094] In some exemplary embodiments, if the difference ratio of the first coordinate axis is a0, a1, a2, the first deviation ratio of the first coordinate axis may be: (a0+a1+a2) / 3, and the second deviation ratio of the second coordinate axis may be: (b0+b1+b2) / 3.

[0095] In this embodiment, if a single preset position is used, it is impossible to calculate the position difference between the roaming panorama and the 3D panorama on the horizontal x-axis and the vertical y-axis, and thus it is impossible to calculate the deviation rate. However, using the position difference of at least two preset positions during calculation can more accurately determine the deviation rate between the roaming panorama and the 3D panorama.

[0096] In one embodiment, adjusting the roaming panoramic image according to the coordinate deviation rate to obtain a calibrated panoramic image includes:

[0097] Adjust each coordinate point in the roaming panoramic image according to the first deviation rate and the second deviation rate to obtain a calibrated coordinate value;

[0098] A calibrated panorama is determined according to the calibrated coordinate values.

[0099] Specifically, after calculating the first deviation rate and the second deviation rate, each coordinate point in the roaming panoramic image is recalculated using the first deviation rate and the second deviation rate, and each coordinate point is adjusted. Each adjusted coordinate point can be a calibration coordinate value. Then, all adjusted calibration coordinate values ​​are combined to obtain a calibrated panoramic image.

[0100] In this embodiment, each coordinate point is adjusted by the first deviation rate and the second deviation rate so that the adjusted coordinates are closer to the camera point coordinates for producing the panoramic image in the original scene, and the connection between the points is more accurate.

[0101] In one embodiment, Figure 4As shown, S50, adjusting the point spatial data of the project model according to the coordinate deviation rate to obtain a calibration model, previously includes:

[0102] S52: embed the code of the project model into the web page.

[0103] Specifically, the code of the project model is embedded into the web page for package development.

[0104] In some exemplary embodiments, the project model can be imported into a panoramic tour production platform, such as the Everpano platform, and the corresponding options can be checked, such as Render Settings - √depth, √convert 2cube, √Krpano projrct, and the project model code can be embedded into the web page for packaging and development for external deployment. Since the project model code is directly embedded in the web page, it does not need to be readjusted and read, so it has a faster loading speed after deployment.

[0105] S54: Simplify the project model. The simplification includes: reducing the number of faces of the project model without affecting the structure of the project model.

[0106] Specifically, simplification can usually be done when the project model on the 3dmax software platform has a large number of faces and volume, and the number of faces of the model needs to be reduced without affecting the overall structure. Therefore, the project model needs to be simplified.

[0107] In this embodiment, simplifying the project model can obtain a lighter project model file, and in subsequent processing, if the simplified project model is used, the subsequent data loading will be lighter and more friendly, and it can run faster in the panoramic roaming production platform.

[0108] S50, adjusting the point spatial data of the project model according to the coordinate deviation rate to obtain a calibration model, including:

[0109] S56 , adjusting the point spatial data in the project model according to the first deviation rate and the second deviation rate to obtain calibrated spatial point data.

[0110] The point spatial data may generally be the spatial coordinates in the project model.

[0111] Specifically, the project model is usually configured and output in a panoramic roaming production platform, so the point spatial data in the project model can be enlarged or reduced according to the first deviation rate and the second deviation rate to be converted into accurate point spatial data.

[0112] In some exemplary embodiments, by selecting the corresponding options in the panoramic tour production platform, such as RenderSettings - √stl, √convert 2cube, √Krpano projrct, a new file corresponding to the panoramic tour production platform is generated. The spatial point data in the project model can be processed one by one according to the first deviation rate and the second deviation rate, and the spatial point data can be enlarged or reduced to convert it into accurate point spatial data. It should be noted that the project model mentioned in this embodiment can be a project model that has been simplified through the above steps, or a project model that has not been simplified.

[0113] S58: Obtain a calibration model according to the calibration space point data.

[0114] Specifically, the new calibration spatial point data is replaced with the original spatial point data, and the obtained new model can be the calibration model.

[0115] In an exemplary embodiment, as shown in Table 1,

[0116] Table 1 Replacement data sheet

[0117] Panos Documents Replace stl file xx.xml code file 0.tiles 0.stl "VR001" field 1 = "panos\0.tiles\xx.jpg"> 1. Tiles 1.stl "VR002" field 1 = "panos\1.tiles\xx.jpg"> 2. Tiles 2.stl "VR003" field 1 = "panos\2.tiles\xx.jpg"> 3.Tiles 3.stl "VR004" field 1 = "panos\3.tiles\xx.jpg"> 4. Tiles 4.stl "VR005" field 1 = "panos\4.tiles\xx.jpg"> N.tiles N.stl "VR005" field 1 = "panos\N.tiles\xx.jpg">

[0118] The "Panos" file is generally understood to be the file corresponding to spatial point data. The "stl" file is generally understood to be the file corresponding to calibration spatial point data. The "xx.xml" code file is generally understood to be the code file that replaces the original spatial point data with the new calibration spatial point data. "Field 1" is generally understood to be the field containing the code to be replaced. The "xx.jpg" file is generally understood to be the image corresponding to the calibration model or project model.

[0119] In this embodiment, the project model is adjusted through the coordinate deviation rate, and the spatial point data therein is replaced, so that the calibration model obtained after processing is more accurate, which helps to accurately calculate the point spatial coordinate values ​​of the overall virtual generated space, and has good spatial fluidity and accurate spatial transition feeling in the subsequent virtual space experience, without causing offset of the point spatial coordinate position.

[0120] In one embodiment, integrating and connecting the calibrated panoramic image and the calibrated model to obtain a panoramic roaming scene includes:

[0121] The calibrated panorama and the calibrated model are integrated and connected by image stitching technology to obtain a panoramic roaming scene. The image stitching technology includes: image registration and image fusion. The image registration algorithm adopts a region-based method.

[0122] Among them, the integration connection can usually be a method of combining a 2D panorama and a 3D calibration model. Image stitching technology is a technology that stitches together several images with overlapping parts (which may be obtained at different times, different perspectives or different sensors) into a seamless panorama or high-resolution image. Image registration can usually be achieved by using a certain matching strategy to find the corresponding position of the template or feature points in the image to be stitched in the reference image, and then determine the transformation relationship between the two images. Image fusion refers to the process of subjecting image data of the same target collected by multiple source channels to image processing and computer technology, etc., to maximize the extraction of beneficial information in each channel, and finally to synthesize it into a high-quality image to improve the utilization of image information.

[0123] Specifically, the calibrated panorama and the calibration model can be integrated and connected through image stitching technology. Image stitching technology can include image registration and image fusion. Image registration technology typically uses a region-based method. This method uses a block in the overlapping area of ​​one image as a template and searches for the matching block that most closely resembles this template in the other image. The panoramic roaming scene obtained using this algorithm has high accuracy. It should be noted that image registration technology can also use phase correlation methods or feature-based methods.

[0124] In this embodiment, the panoramic roaming scene generated through image stitching technology can make the panoramic roaming scene connected in both 2D and 3D. The region-based method used in image registration can generate a highly accurate panoramic roaming scene, which can greatly ensure the accuracy and smoothness of the subsequent roaming experience.

[0125] In one embodiment, the roaming panorama, the three-dimensional panorama, and the calibrated panorama are all spherical panoramas.

[0126] In this embodiment, the panorama can generally include: spherical panorama, cubic panorama, cylindrical panorama, and the spherical panorama is more in line with the observation habits of the human eye and will also provide the best immersive experience when roaming in the virtual display space, so the panorama used in this application is a spherical panorama.

[0127] In another embodiment, Figure 5 As shown, the present disclosure also provides a method for constructing a panoramic roaming scene, including:

[0128] S502: Create a three-dimensional panorama, and input the three-dimensional panorama into a panoramic roaming production scene to obtain a roaming panorama.

[0129] S504: Obtain a first position difference value of a preset position in the roaming panoramic image and a second position difference value of a preset position in the three-dimensional panoramic image, wherein there are at least two preset positions, and each of the preset positions has the same vertical coordinate position.

[0130] S506: Calculate a coordinate difference ratio based on the first position difference and the second position difference, where the coordinate difference ratio includes a first coordinate axis difference ratio and a second coordinate axis difference ratio.

[0131] S508 : Calculate a coordinate deviation rate between the roaming panoramic image and the three-dimensional panoramic image according to the coordinate difference ratio, where the coordinate deviation rate includes a first deviation rate of a first coordinate axis and a second deviation rate of a second coordinate axis.

[0132] S510 : Adjust each coordinate point in the roaming panoramic image according to the first deviation rate and the second deviation rate to obtain a calibrated coordinate value.

[0133] S512: Determine a calibrated panoramic image according to the calibrated coordinate values.

[0134] S514: Build a project model based on the calibrated panoramic image and the panoramic tour production platform.

[0135] S516: Simplify the project model. The simplification includes: reducing the number of faces of the project model without affecting the structure of the project model.

[0136] S518: Adjust the point spatial data in the project model according to the first deviation rate and the second deviation rate to obtain calibrated spatial point data.

[0137] S520: Obtain a calibration model according to the calibration space point data.

[0138] S522 , integrating and connecting the calibrated panorama and the calibrated model through image stitching technology to obtain a panoramic roaming scene. The image stitching technology includes: image registration and image fusion. The image registration algorithm adopts a region-based method.

[0139] It should be noted that the specific implementation methods and limitations in this embodiment can be found in the above embodiments and will not be repeated in this embodiment.

[0140] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0141] Based on the same inventive concept, embodiments of the present disclosure also provide a panoramic roaming scene construction device for implementing the aforementioned panoramic roaming scene construction method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more of the following embodiments of the panoramic roaming scene construction device can be found in the aforementioned limitations of the panoramic roaming scene construction method and will not be further elaborated here.

[0142] In one embodiment, Figure 6 As shown, a panoramic roaming scene construction device 600 is provided, including: a calculation module 602, a panoramic image calibration module 604, a model construction module 606, a model calibration module 608, and an integration module 610, wherein:

[0143] A calculation module 602 is used to calculate a coordinate deviation rate between a roaming panorama and a three-dimensional panorama, wherein the roaming scene is obtained by inputting the three-dimensional scene into a panoramic roaming production platform;

[0144] A panoramic image calibration module 604 is configured to adjust the roaming panoramic image according to the coordinate deviation rate to obtain a calibrated panoramic image;

[0145] A model building module 606 is configured to build a project model based on the calibrated panorama and the panorama tour production platform;

[0146] A model calibration module 608 is configured to adjust the point spatial data of the project model according to the coordinate deviation rate to obtain a calibration model;

[0147] The integration module 610 is configured to integrate and connect the calibrated panoramic image and the calibrated model to obtain a panoramic roaming scene.

[0148] In one embodiment of the device, the calculation module 602 includes: a coordinate difference acquisition module, a difference ratio calculation module, and a deviation rate calculation module;

[0149] The coordinate difference acquisition module is configured to acquire a first position difference of a preset position in the roaming panoramic image and a second position difference of a preset position in the three-dimensional panoramic image, wherein there are at least two preset positions, and each of the preset positions has the same vertical coordinate position;

[0150] The difference ratio calculation module is configured to calculate a coordinate difference ratio based on the first position difference and the second position difference, wherein the coordinate difference ratio includes a first coordinate axis difference ratio and a second coordinate axis difference ratio;

[0151] The deviation rate calculation module is used to calculate the coordinate deviation rate between the roaming panoramic image and the three-dimensional panoramic image according to the coordinate difference ratio, and the coordinate deviation rate includes a first deviation rate of the first coordinate axis and a second deviation rate of the second coordinate axis.

[0152] In one embodiment of the apparatus, the panoramic image calibration module 604 includes: a deviation rate adjustment module, a coordinate value determination module;

[0153] The deviation rate adjustment module is configured to adjust each coordinate point in the roaming panoramic image according to the first deviation rate and the second deviation rate to obtain a calibrated coordinate value;

[0154] The coordinate value determination module is used to determine the calibration panorama according to the calibration coordinate values.

[0155] In one embodiment of the device, the device further includes: a simplification processing module for simplifying the project model, and the simplification processing includes: reducing the number of faces of the project model without affecting the structure of the project model.

[0156] In one embodiment of the apparatus, the model calibration module 608 includes: a data adjustment module, a model determination module;

[0157] The data adjustment module is configured to adjust the point spatial data in the project model according to the first deviation rate and the second deviation rate to obtain calibrated spatial point data;

[0158] The model determination module is used to obtain a calibration model according to the calibration space point data.

[0159] In one embodiment of the device, the integration module 610 is further used to integrate and connect the calibrated panorama and the calibration model through image stitching technology to obtain a panoramic roaming scene. The image stitching technology includes: image registration and image fusion, and the image registration algorithm adopts a region-based method.

[0160] In one embodiment of the device, the roaming panorama, the three-dimensional panorama, and the calibrated panorama are all spherical panoramas.

[0161] Each module in the aforementioned panoramic roaming scene construction device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0162] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for constructing a panoramic roaming scene is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0163] Those skilled in the art will understand that Figure 7 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present disclosure, and does not constitute a limitation on the computer device to which the solution of the present disclosure is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0164] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0165] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0166] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0167] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in the present disclosure may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this disclosure may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in each embodiment provided in this disclosure may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc.

[0168] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The above-described embodiments merely represent several implementation methods of the present disclosure. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present disclosure. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present disclosure, all of which fall within the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be determined by the appended claims.

Claims

1. A method for constructing a panoramic roaming scene, characterized in that: The method comprises: Calculating a coordinate deviation rate between a roaming panorama and a three-dimensional panorama, wherein the roaming panorama is obtained by inputting the three-dimensional panorama into a panoramic roaming production platform; calculating the coordinate deviation rate between the roaming panorama and the three-dimensional panorama, comprising: obtaining a first position difference value of a preset position in the roaming panorama, and a second position difference value of a preset position in the three-dimensional panorama, wherein the preset positions are at least two, and each of the preset positions has the same vertical coordinate position; calculating a coordinate difference ratio based on the first position difference value and the second position difference value, wherein the coordinate difference ratio includes a first coordinate axis difference ratio and a second coordinate axis difference ratio; calculating a coordinate deviation rate between the roaming panorama and the three-dimensional panorama based on the coordinate difference ratio, wherein the coordinate deviation rate includes a first deviation rate of the first coordinate axis and a second deviation rate of the second coordinate axis; adjusting the roaming panoramic image according to the coordinate deviation rate to obtain a calibrated panoramic image; Building a project model based on the calibrated panorama and the panoramic tour production platform; Adjusting the point spatial data of the project model according to the coordinate deviation rate to obtain a calibration model; The calibrated panorama and the calibrated model are integrated and connected to obtain a panoramic roaming scene.

2. The method according to claim 1, characterized in that The step of adjusting the roaming panoramic image according to the coordinate deviation rate to obtain a calibrated panoramic image includes: Adjust each coordinate point in the roaming panoramic image according to the first deviation rate and the second deviation rate to obtain a calibrated coordinate value; A calibrated panorama is determined according to the calibrated coordinate values.

3. The method according to any one of claims 1-2, characterized in that The step of adjusting the point spatial data of the project model according to the coordinate deviation rate to obtain a calibration model includes: The project model is simplified, and the simplification includes: reducing the number of faces of the project model without affecting the structure of the project model.

4. The method according to claim 1, wherein The step of adjusting the point spatial data of the project model according to the coordinate deviation rate to obtain a calibration model includes: Adjusting the point spatial data in the project model according to the first deviation rate and the second deviation rate to obtain calibrated spatial point data; A calibration model is obtained according to the calibration space point data.

5. The method according to claim 1, wherein The step of integrating and connecting the calibrated panorama image and the calibrated model to obtain a panoramic roaming scene includes: The calibrated panorama and the calibrated model are integrated and connected by image stitching technology to obtain a panoramic roaming scene. The image stitching technology includes: image registration and image fusion. The image registration algorithm adopts a region-based method.

6. The method according to claim 1, characterized in that The roaming panorama, three-dimensional panorama, and calibrated panorama are all spherical panoramas.

7. A panoramic roaming scene construction device, characterized in that: The device comprises: A calculation module is configured to calculate a coordinate deviation rate between a roaming panorama and a three-dimensional panorama, wherein the roaming scene is obtained by inputting the three-dimensional panorama into a panoramic roaming production platform; the calculation module comprises: a coordinate difference acquisition module, a difference ratio calculation module, and a deviation rate calculation module; the coordinate difference acquisition module is configured to acquire a first position difference of a preset position in the roaming panorama and a second position difference of a preset position in the three-dimensional panorama, wherein the preset positions are at least two and each of the preset positions has the same vertical coordinate position; the difference ratio calculation module is configured to calculate a coordinate difference ratio based on the first position difference and the second position difference, wherein the coordinate difference ratio includes a first coordinate axis difference ratio and a second coordinate axis difference ratio; the deviation rate calculation module is configured to calculate a coordinate deviation rate between the roaming panorama and the three-dimensional panorama based on the coordinate difference ratio, wherein the coordinate deviation rate includes a first deviation rate of the first coordinate axis and a second deviation rate of the second coordinate axis; A panoramic image calibration module, configured to adjust the roaming panoramic image according to the coordinate deviation rate to obtain a calibrated panoramic image; A model building module, configured to build a project model based on the calibrated panorama and the panoramic roaming production platform; A model calibration module, configured to adjust the point spatial data of the project model according to the coordinate deviation rate to obtain a calibration model; The integration module is used to integrate and connect the calibrated panoramic image and the calibrated model to obtain a panoramic roaming scene.

8. The device according to claim 7, characterized in that The panoramic image calibration module includes: a deviation rate adjustment module and a coordinate value determination module; The deviation rate adjustment module is configured to adjust each coordinate point in the roaming panoramic image according to the first deviation rate and the second deviation rate to obtain a calibrated coordinate value; The coordinate value determination module is used to determine the calibration panorama according to the calibration coordinate values.

9. The device according to any one of claims 7-8, characterized in that The device further includes: a simplification processing module, which is used to perform simplification processing on the project model. The simplification processing includes: reducing the number of faces of the project model without affecting the structure of the project model.

10. The device according to claim 7, characterized in that The model calibration module includes: a data adjustment module and a model determination module; The data adjustment module is configured to adjust the point spatial data in the project model according to the first deviation rate and the second deviation rate to obtain calibrated spatial point data; The model determination module is used to obtain a calibration model according to the calibration space point data.

11. The device according to claim 7, characterized in that The integration module is further used to integrate and connect the calibrated panorama and the calibrated model through image stitching technology to obtain a panoramic roaming scene. The image stitching technology includes: image registration and image fusion. The image registration algorithm adopts a region-based method.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images