Methods, apparatus, devices, and media for switching viewpoints in roaming production applications.

Abstract

Methods, apparatus, devices, and media for switching viewpoints in a walkthrough production application are provided. In one method, a set of panoramic images, each associated with a set of spatial locations within a spatial range, are acquired. The panoramic images represent the surrounding environment at each spatial location within the set of spatial locations. A two-dimensional view of the spatial range is presented in a two-dimensional window of the walkthrough production application. The two-dimensional view includes a planar map of the spatial range and a set of two-dimensional markers in the planar map, each associated with a set of spatial locations. A three-dimensional view generated based on the set of panoramic images is presented in a three-dimensional window of the walkthrough production application. In response to receiving an interaction with a two-dimensional marker within the set of two-dimensional markers, the viewpoint of the three-dimensional view is switched to the spatial location associated with the two-dimensional marker, so that a panoramic image associated with the spatial location is presented in the three-dimensional window. This facilitates the switching of viewpoints in the three-dimensional view for production personnel, thereby improving production efficiency.

Description

Technical Field

[0001] Exemplary implementations of this disclosure generally relate to viewpoint switching principles, and more particularly to methods, apparatus, devices, and computer-readable storage media for switching viewpoints in roaming production applications. Background Technology

[0002] With the development of digital technology, various panoramic tour applications have been developed. Creators can acquire images at multiple spatial locations within a real-world area and generate multiple panoramic images associated with each location, thus creating panoramic tour data. These panoramic images can be loaded into the panoramic tour application, allowing users to roam in virtual space and experience an immersive 3D visual effect. During the production process, creators need to roam in virtual space to change viewpoint positions and orientations to check if the 3D views presented at each location achieve the desired effect. Therefore, how to switch viewpoints more conveniently and quickly and rapidly to view the 3D visual effects at each location becomes a pressing issue. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for switching viewpoints in a roaming creation application is provided. In this method, a set of panoramic images, each associated with a set of spatial locations within a spatial range, are acquired, the panoramic images representing the surrounding environment at each spatial location within the set of spatial locations. A two-dimensional view of the spatial range is presented in a two-dimensional window of the roaming creation application, the two-dimensional view including a planar map of the spatial range and a set of two-dimensional markers in the planar map, each associated with a set of spatial locations. A three-dimensional view generated based on the set of panoramic images is presented in a three-dimensional window of the roaming creation application. In response to receiving an interaction with a two-dimensional marker in the set of two-dimensional markers, the viewpoint of the three-dimensional view is switched to the spatial location associated with the two-dimensional marker, so that a panoramic image associated with the spatial location is presented in the three-dimensional window.

[0004] In a second aspect of this disclosure, an apparatus for switching viewpoints in a roaming creation application is provided. The apparatus includes: an acquisition module configured to acquire a set of panoramic images, each associated with a set of spatial locations within a spatial range, the panoramic images representing the surrounding environment at each spatial location within the set of spatial locations; a two-dimensional rendering module configured to render a two-dimensional view of the spatial range in a two-dimensional window of the roaming creation application, the two-dimensional view including a planar map of the spatial range and a set of two-dimensional markers in the planar map each associated with a set of spatial locations; a three-dimensional rendering module configured to render a three-dimensional view generated based on the set of panoramic images in a three-dimensional window of the roaming creation application; and a switching module configured to, in response to receiving an interaction with a two-dimensional marker in the set of two-dimensional markers, switch the viewpoint of the three-dimensional view to a spatial location associated with the two-dimensional marker, so as to render a panoramic image associated with the spatial location in the three-dimensional window.

[0005] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the device to perform a method according to a first aspect of this disclosure when executed by the at least one processing unit.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to implement the method according to a first aspect of this disclosure.

[0007] It should be understood that the content described in this summary section is not intended to limit the key or essential features of the implementation of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0008] The above and other features, advantages, and aspects of various implementations of this disclosure will become more apparent in the following detailed description, taken in conjunction with the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0009] Figure 1 A block diagram of an example environment in which the implementation of this disclosure can be implemented is shown;

[0010] Figure 2 A block diagram is shown illustrating a process for switching viewpoints in a roaming production application according to some implementations of this disclosure;

[0011] Figure 3 A block diagram is shown illustrating a process for presenting two-dimensional markers in a two-dimensional window according to some implementations of this disclosure.

[0012] Figure 4 A block diagram is shown illustrating a process for switching viewpoints in a three-dimensional window according to some implementations of this disclosure;

[0013] Figure 5 A block diagram is shown illustrating a process for presenting material information in a roaming production application according to some implementations of this disclosure;

[0014] Figure 6 A block diagram is shown illustrating a process for presenting a panoramic image including sensitive information according to some implementations of this disclosure;

[0015] Figure 7 A block diagram is shown illustrating a process for presenting a panoramic image with sensitive information removed, according to some implementations of this disclosure;

[0016] Figure 8 A block diagram is shown illustrating a process for switching viewpoints in a roaming production application according to some implementations of this disclosure;

[0017] Figure 9 A block diagram is shown illustrating a process for navigating in a three-dimensional window according to some implementations of this disclosure;

[0018] Figure 10 A flowchart is shown illustrating a method for switching viewpoints in a roaming production application according to some implementations of this disclosure;

[0019] Figure 11 A block diagram of a device for switching viewpoints in a roaming production application, according to some implementations of this disclosure, is shown; and

[0020] Figure 12 A block diagram of a device capable of implementing various implementations of the present disclosure is shown. Detailed Implementation

[0021] Implementations of this disclosure will now be described in more detail with reference to the accompanying drawings. While some implementations of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the implementations set forth herein. Rather, these implementations are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and implementations of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0022] In the description of the implementation methods disclosed herein, the term "comprising" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one implementation" or "the implementation" should be understood as "at least one implementation". The term "some implementations" should be understood as "at least some implementations". Other explicit and implicit definitions may also be included below. As used herein, the term "model" can represent the relationships between various data. For example, the aforementioned relationships can be obtained based on various currently known and / or future-developed technical solutions.

[0023] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0025] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0026] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, for example, via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0027] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0028] Example Environment

[0029] Several panoramic tour applications have already been developed. Developers can collect images at multiple spatial locations within a real-world area and generate multiple panoramic images associated with those locations, thus creating panoramic tour data. (See also...) Figure 1 Describe the overview of the panoramic roaming application. Figure 1A block diagram of an example environment 100 in which the implementation of this disclosure can be carried out is shown. For example... Figure 1 As shown, panoramic images of multiple spatial locations within a given spatial area (e.g., a residence) can be presented in the panoramic roaming application 110. Specifically, a 2D map 120 can be presented, along with a 3D view 130 viewed from viewpoint 122. Users can roam within the panoramic roaming application 110 to view panoramic images of various spatial locations within the residence.

[0030] Various panorama creation applications have been developed for producing panoramic images. Creators can use these applications to organize view images captured at multiple spatial locations within a given area, thereby generating multiple panoramic images associated with each location. Before publishing the panoramic images, creators need to navigate through a 3D view to ensure a satisfactory panoramic view is available at each location within the spatial area.

[0031] At this point, the production team needs to manually manipulate the 3D view to reach various locations within the spatial area and check whether the corresponding panoramic view meets the standards. Because the spatial area often has a complex structure (e.g., including obstacles such as walls), this requires complex operations for the production team to reach various locations within the spatial area. Therefore, how to switch viewpoints more conveniently and quickly to view the 3D visual effects at various locations becomes a pressing problem to be solved.

[0032] Overview of viewpoint switching process

[0033] To address the shortcomings of the aforementioned technical solutions, a method for switching viewpoints in a roaming production application is proposed based on an exemplary implementation of this disclosure. See below for details. Figure 2 Describe an overview of the viewpoint switching. Figure 2 A block diagram 200 illustrates a process for switching viewpoints in a roaming production application, according to some implementations of this disclosure. For example... Figure 2 As shown, the spatial range 210 may include multiple spatial locations 212, 214, ..., and 216, each of which may have an associated panoramic image. For example, spatial location 212 may have an associated panoramic image 218, and the panoramic image 218 may represent the surrounding environment captured at spatial location 212.

[0034] It will be understood that the panoramic image 218 here can be generated based on multiple view images (e.g., front view, rear view, left view, right view, top view, and bottom view) acquired at spatial location 212. Panoramic view 218 can be acquired using a panoramic view acquisition device. Creators can use a walkthrough application to combine panoramic images associated with different spatial locations to generate panoramic walkthrough data that can be loaded by the walkthrough application.

[0035] According to an exemplary implementation of this disclosure, a set of panoramic images associated with a set of spatial locations 212, 214, ..., 216 within a spatial range 210 can be acquired. In this case, each panoramic image represents the surrounding environment at a spatial location. For example... Figure 2 As shown, the roaming creation application may include a two-dimensional window 220 and a three-dimensional window 230. Specifically, a two-dimensional view of the spatial range can be presented in the two-dimensional window 220, and a two-dimensional view 222 of the spatial range can be presented in the three-dimensional window 230.

[0036] The two-dimensional view may include: a planar map 228 of spatial extent 210; and a set of two-dimensional markers 222, 224, ..., 226 in the planar map 228, each associated with a set of spatial locations 212, 214, ..., 216. A two-dimensional window 220 and a three-dimensional window 230 may be presented side-by-side, and a three-dimensional view 232 generated based on a set of panoramic images may be presented in the three-dimensional window 230. Here, the creator can navigate within the three-dimensional window 230, reaching various locations within the spatial extent and checking whether the corresponding three-dimensional view at each location meets the desired conditions.

[0037] To facilitate rapid viewpoint switching in roaming production applications, the viewpoint of the 3D view 222 can be switched based on the creator's interaction in the 2D window 220. Specifically, if an interaction is received between the creator and a 2D marker (e.g., 2D marker 222), the viewpoint of the 3D view 232 can be switched to the spatial location 212 corresponding to the 2D marker 222. Thus, a panoramic image 218 associated with the spatial location 212 can be presented in the 3D window 230.

[0038] In the context of this disclosure, the user of the roaming creation application is the creator, not the end user of the panoramic roaming application 110, and the spatial locations 212, 214, ..., 216 are not arbitrary locations within the spatial range 210, but rather acquisition points used to acquire and synthesize panoramic images. According to an exemplary implementation of this disclosure, a correlation can be established between the spatial locations 212, 214, ..., 216 within the spatial range 210, the two-dimensional markers 222, 224, ..., 226 on the planar map 228, and the corresponding viewpoints in the three-dimensional window 230. In this way, the creator can more easily and quickly switch the viewpoint of the three-dimensional view 232, thereby more effectively checking whether the panoramic images displayed in the three-dimensional window 230 associated with each spatial location meet expectations.

[0039] Furthermore, by displaying the two-dimensional window 220 and the three-dimensional window 230 side-by-side in the panorama creation application, the creators can simultaneously view and compare the content displayed in the two windows, thereby adjusting any issues in the panoramic image accordingly. This improves the creators' work efficiency, ensuring that panoramic panorama data is generated in a more accurate and effective manner.

[0040] Detailed process of viewpoint switching

[0041] See already Figure 2 This section provides an overview of viewpoint switching. Further details on switching viewpoints in roaming creation applications will be described below. Figure 3 A block diagram 300 illustrates a process for presenting two-dimensional markers in a two-dimensional window according to some implementations of this disclosure. For example... Figure 3 As shown, a planar map 228 of the spatial range 210 can be presented in a two-dimensional window 220. Furthermore, it can be determined where in the planar map 228 the two-dimensional marker 222 corresponding to the spatial location 212 will be displayed.

[0042] According to an exemplary implementation of this disclosure, a reference coordinate system can be established in various ways. For example, the vertex shown in the lower left corner of the spatial range 210 can be used as the origin 310 of the reference coordinate system 312, and then... Figure 3The arrows indicate the x, y, and z coordinate axes, respectively. The position of spatial location 212 within spatial range 210 can then be represented as (X1, Y1, Z1). Furthermore, a coordinate system 322 corresponding to coordinate system 312 can be established within the two-dimensional window 220. Assuming the lower left vertex of the planar map 228 corresponds to the origin 310 within spatial range 210, this vertex can be used as the origin 320, and a coordinate system 322 for the planar map 228 can be established. In this case, coordinate system 322 only includes the x and y axes, and the coordinates of the two-dimensional marker 222 corresponding to spatial location 212 can be represented as (X1, Y1) (i.e., the coordinates of the height axis z are not considered).

[0043] Furthermore, a two-dimensional marker 222 can be displayed at location (X1, Y1) on the planar map 228. The two-dimensional marker can be displayed based on various styles; for example, it can utilize... Figure 3 The circles shown represent two-dimensional markers 222. Using the exemplary implementation of this disclosure, the correspondence between spatial location 212 and the coordinates of the two-dimensional markers can be determined based on simple geometric calculations, thereby presenting the two-dimensional markers corresponding to each spatial location in the two-dimensional window 220 of the roaming creation application. In this way, it is convenient for creators to understand the two-dimensional distribution of spatial locations, and thus convenient to switch the viewpoint of the three-dimensional view 232 to the corresponding spatial location by clicking on each two-dimensional marker (e.g., creators can use a mouse and / or finger to click or double-click, etc.).

[0044] It will be understood that the above description of the process of presenting two-dimensional marker 222 is only an example of spatial position 212 in spatial range 210. According to an exemplary implementation of this disclosure, other spatial positions 214, ..., and 216 in spatial range 210 can be processed in a similar manner, thereby presenting two-dimensional markers 222, ..., and 226 corresponding to each spatial position 214, ..., and 216 in two-dimensional window 220.

[0045] In the following text, we will return Figure 2 Further details regarding viewpoint switching are described below. During the initial process, the viewpoint of the 3D view 232 can be set to a predetermined initial position (e.g., the entrance or center of spatial range 210, etc.). The creator can navigate within the 3D window 230 to browse the surrounding environment of the desired location. Furthermore, the creator can click on any 2D marker in the 2D window 220 to switch viewpoints.

[0046] like Figure 2 As shown, if an interaction between the creator and the two-dimensional marker 222 is received, the viewpoint of the three-dimensional view 232 can be switched to the position corresponding to the two-dimensional marker 222. For example... Figure 2As shown on the right, the viewpoint of 3D view 232 can be located in the center of the living room. In this way, the production team can quickly move the viewpoint to the desired location to view the surrounding environment.

[0047] According to one exemplary implementation of this disclosure, the orientation of the viewpoint can be determined in various ways. For example, the orientation of the current viewpoint of the 3D view 232 can be used as the orientation of the viewpoint after switching. Specifically, assuming the viewpoint orientation before switching is forward, the viewpoint can be switched to the spatial position 212 corresponding to the 2D marker 222 while maintaining the viewpoint orientation unchanged. Another example is that the viewpoint orientation can be switched to a predetermined orientation associated with the spatial position. According to one exemplary implementation of this disclosure, a default orientation can be specified for each spatial position; for example, the direction with the best visual effect visible from that spatial position can be set as the default orientation. Alternatively and / or additionally, the direction with the farthest visible distance from that spatial position can be set as the default orientation, and so on.

[0048] Using the exemplary implementation of this disclosure, the orientation of the viewpoint can be flexibly configured, facilitating the inspection of panoramic images at various locations by the production staff. Furthermore, the production staff can control the viewpoint using a mouse and / or keyboard. For example, specific keyboard shortcuts can be used to perform rotation operations to view a 360° panoramic view around the location. Alternatively, the viewpoint can be moved in various directions by dragging or other methods. In this way, the roaming speed of the production staff within the spatial range 210 can be increased, thereby improving the efficiency of panoramic image production.

[0049] According to one exemplary implementation of this disclosure, it is possible to allow the creator to click on other two-dimensional markers in the planar map 228. Figure 4 A block diagram 400 illustrates a process for switching viewpoints in a three-dimensional window 230 according to some implementations of this disclosure. For example... Figure 4 As shown, if an interaction between the production staff and the two-dimensional marker 224 is received, the three-dimensional view 232 can be switched to spatial position 214 within the spatial range 210. At this time, the three-dimensional view 232 will display the surrounding environment of the passageway.

[0050] According to one exemplary implementation of this disclosure, information about the production materials of panoramic images can be provided in a roaming production application. See also Figure 5 The description presents more details about the material information. Figure 5 A block diagram 500 illustrates a process for presenting material information in a roaming creation application, according to some implementations of this disclosure. For example... Figure 5 As shown, a material window 510 can be presented to display information about various manufacturing materials used to generate a panoramic image associated with the two-dimensional marker 222.

[0051] Here, the production materials may include, for example, multiple view images 512 for generating the panoramic image. The production personnel can click on view images 512 to view and / or edit the original images from various angles used to generate the panoramic image. As another example, the production materials may further include descriptive information 514 about the spatial location. Here, the descriptive information may include the area where the spatial location is located (e.g., "living room"), and may further provide more information about the living room. The production personnel can click on descriptive information 514 to view and / or edit various textual descriptions used to describe the panoramic image.

[0052] Will understand, although Figure 5 Material window 510 is shown as a pop-up window overlaid on 3D window 230. According to an exemplary implementation of this disclosure, material window 510 may be shown at other locations in the roaming production application. Alternatively and / or additionally, the production staff may be allowed to move material window 510. Using the exemplary implementation of this disclosure, during the production of roaming data, the production staff can easily edit the production materials associated with various spatial locations at any time, and use the edited production materials to generate a panoramic image for presentation within 3D window 230.

[0053] According to an exemplary implementation of this disclosure, two-dimensional markers 222 with different styles can be used to represent material information in different states. For example, the view image 512 may include sensitive information such as human figures or text. This sensitive information should be removed when creating panoramic walkthrough data. See below. Figure 6 Describe more details regarding the handling of sensitive information. Figure 6 A block diagram 600 illustrates a process for presenting a panoramic image including sensitive information, according to some implementations of this disclosure. For example... Figure 6 As shown, the 3D view 232 includes sensitive information 610 (i.e., human image). If panoramic roaming data including sensitive information 610 is published directly, it may lead to the risk of leakage of personal privacy.

[0054] According to an exemplary implementation of this disclosure, potentially sensitive information 610 can be automatically detected and marked based on image recognition technology. Alternatively and / or additionally, sensitive information 610 manually annotated by a creator can be received. In this case, if sensitive information 610 is found in the three-dimensional view 232 and / or in the view image, the two-dimensional marker 222 can be presented in a different style. Figure 6 As shown, a star shape can be used to represent a two-dimensional marker 222 containing sensitive information 610.

[0055] Using the exemplary implementation of this disclosure, the different styles of the two-dimensional markers in the two-dimensional window 220 can provide the production staff with the production status of the panoramic image related to each spatial location. In this way, the production staff can directly determine which spatial locations still need to be processed based on the style of the two-dimensional markers.

[0056] See below. Figure 7 Describe further operations related to handling sensitive information. Figure 7 A block diagram 700 illustrates a process for presenting a panoramic image with sensitive information removed, according to some implementations of this disclosure. For example... Figure 7 As shown, according to an exemplary implementation of this disclosure, sensitive information 610 can be blurred to form a corresponding mask image 710. The mask image 710 can be used to update the corresponding view image, thereby generating a three-dimensional view 232 including the mask image 710.

[0057] At this point, having determined that the sensitive information in the view image has been processed, the two-dimensional marker 222 can be presented using, for example, another style of a rhombus. Furthermore, a new panoramic image associated with spatial location, generated using the processed view image, can be presented in the three-dimensional window 230. Here, the three-dimensional view 232 no longer includes sensitive information, but instead includes a mask image 710 blurred using a mosaic effect.

[0058] Using the exemplary implementation of this disclosure, the style of the two-dimensional markers can indicate to the creator the relevant status of the panoramic image associated with each spatial location. In this case, if the creator finds that all two-dimensional markers represented by stars have been updated to represent diamonds, it can be determined that all sensitive information in the panoramic roaming data has been processed. In this way, the creator can easily determine whether the panoramic roaming data includes sensitive information directly based on the display style in the two-dimensional window 220.

[0059] According to one exemplary implementation of this disclosure, during viewpoint switching, more information can be presented in the two-dimensional window 220 and the three-dimensional window 230 of the roaming creation application. Figure 8 A block diagram 800 illustrates a process for switching viewpoints in a roaming production application, according to some implementations of this disclosure. For example... Figure 8 As shown, a two-dimensional view frustum 810 corresponding to the viewpoint of the three-dimensional view 232 can be presented in the two-dimensional window 220. The two-dimensional view frustum 810 can represent the specific location of the two-dimensional marker 222 and the orientation of the viewpoint in the planar map 228. Figure 8In this model, the vertex of the two-dimensional view frustum 810 can represent the position of the two-dimensional marker 222, and the orientation of the sector can represent the orientation of the viewpoint. In this way, during the production of panoramic roaming data, the production staff can easily understand the relationship between the viewpoint orientation of the three-dimensional view 232 and the planar map 228, thereby facilitating the production staff to adjust the production process at any time and improve production efficiency.

[0060] According to an exemplary implementation of this disclosure, three-dimensional markers corresponding to various spatial locations can be presented in the three-dimensional window 230. For example... Figure 8 As shown, a three-dimensional marker 820 corresponding to a two-dimensional marker 224 can be displayed in the three-dimensional view 232. In this way, the production staff can easily determine the spatial position in the two-dimensional window 220 and the three-dimensional window 230, and then adjust the production process accordingly.

[0061] According to one exemplary implementation of this disclosure, an artist is allowed to interact with the 3D marker 820 within the 3D window 230. For example, an artist can click on the 3D marker 820 to switch the 3D view 232 to spatial location 214 corresponding to the 2D marker 224. At this time, a panoramic image associated with spatial location 224 can be presented in the 3D window 230. In this way, artists can switch viewpoints via the 3D view 232 in a more convenient and efficient manner.

[0062] According to one exemplary implementation of this disclosure, an operator is allowed to perform a roaming motion within the 3D window 230. Specifically, if a roaming motion is detected within the 3D window 230, the position of the viewpoint of the 3D view 232 can be updated. Alternatively and / or additionally, the orientation of the viewpoint of the 3D view 232 can be further updated. Figure 9 A block diagram 900 illustrates a process of navigating within a three-dimensional window 230 according to some implementations of this disclosure. For example... Figure 9 As shown, the roaming production application can receive the roaming actions of the production staff in the 3D window 230 and display the 3D view 232. Correspondingly, in the 2D window 220, the corresponding 2D view frustum 910 can be updated based on the updated viewpoint of the 3D view 232. For example, the 2D view frustum 910 moves to the vicinity of the washroom and faces the vanity mirror in the washroom.

[0063] Using the exemplary implementation of this disclosure, the two-dimensional window 220 and the three-dimensional window 230 can share the position and orientation of the viewpoint of the three-dimensional view 232. In this way, the position and orientation of the two-dimensional view frustum 910 can be updated in real time in the two-dimensional window 220, thereby clearly displaying the position and orientation of the viewpoint of the three-dimensional view 232 in the planar map 228. This allows production personnel to browse relevant panoramic images of various spatial locations more conveniently and quickly, thereby improving production efficiency.

[0064] It will be understood that although the technical solution of an exemplary implementation according to this disclosure is described above using an indoor apartment environment as an example, alternatively and / or additionally, the above technical solution can be implemented in other application environments. For example, in a museum tour environment, multiple view images can be taken at multiple spatial locations within the museum's interior environment, and multiple panoramic images associated with the multiple spatial locations can be generated respectively. The methods described above can be executed in a tour creation application to generate and examine panoramic tour data to be invoked by the museum tour application. As another example, the methods described above can be applied to an outdoor environment. Multiple view images can be taken at multiple spatial locations at outdoor attractions, and the methods described above can be invoked in a tour creation application to assist in the creation of panoramic tour data.

[0065] Example process

[0066] Figure 10 A flowchart of a method 1000 for switching viewpoints in a roaming creation application, according to some implementations of this disclosure, is shown. Specifically, at block 1010, a set of panoramic images, each associated with a set of spatial locations within a spatial range, are acquired, the panoramic images representing the surrounding environment at each spatial location within the set of spatial locations. At block 1020, a two-dimensional view of the spatial range is presented in a two-dimensional window of the roaming creation application, the two-dimensional view including a planar map of the spatial range and a set of two-dimensional markers in the planar map each associated with a set of spatial locations. At block 1030, a three-dimensional view generated based on the set of panoramic images is presented in a three-dimensional window of the roaming creation application. At block 1040, in response to receiving an interaction with a two-dimensional marker in the set of two-dimensional markers, the viewpoint of the three-dimensional view is switched to the spatial location associated with the two-dimensional marker, so that a panoramic image associated with the spatial location is presented in the three-dimensional window.

[0067] According to an exemplary implementation of this disclosure, presenting a two-dimensional view includes: presenting a planar map of a spatial extent in a two-dimensional window; and presenting a set of two-dimensional markers based on a first style at a set of planar locations in the planar map that correspond to a set of spatial locations respectively.

[0068] According to an exemplary implementation of this disclosure, it further includes: in response to receiving an interaction with a two-dimensional marker, determining the production material of the panoramic image, the production material including at least one of the following: multiple view images for generating the panoramic image, descriptive information of the spatial location; and relevant information for presenting the production material.

[0069] According to an exemplary implementation of this disclosure, presenting a two-dimensional view further includes: in response to determining that a plurality of view images include sensitive information, presenting a two-dimensional markup based on a second style that is different from the first style.

[0070] According to an exemplary implementation of this disclosure, it further includes: in response to determining that sensitive information in a plurality of view images has been processed, presenting a two-dimensional marker based on a third style different from the first and second styles; and presenting a new panoramic image associated with a spatial location in a three-dimensional window, the new panoramic image being generated using the processed plurality of view images.

[0071] According to an exemplary implementation of this disclosure, presenting a panoramic image in a three-dimensional window includes: determining the orientation of the viewpoint of the three-dimensional view based on at least one of the following: the orientation of the viewpoint of the three-dimensional view before the viewpoint switch, a predetermined orientation associated with the spatial location; and presenting the panoramic image based on the orientation of the viewpoint.

[0072] According to an exemplary implementation of this disclosure, presenting a two-dimensional view in a two-dimensional window further includes: presenting a two-dimensional view frustum corresponding to the viewpoint of the three-dimensional view in the two-dimensional window; and presenting a three-dimensional view in a three-dimensional window further includes: presenting a set of three-dimensional markers corresponding to a set of spatial locations in the three-dimensional window.

[0073] According to an exemplary implementation of this disclosure, it further includes: in response to receiving an interaction with a 3D marker in a set of 3D markers, switching the viewpoint of the 3D view to a spatial location corresponding to the 3D marker, so as to present a panoramic image associated with the spatial location in the 3D window.

[0074] According to an exemplary implementation of this disclosure, it further includes: updating at least one of the position and orientation of the viewpoint of the three-dimensional view in response to detecting a roaming action in the three-dimensional window; and updating the two-dimensional view frustum in the two-dimensional window based on the updated viewpoint of the three-dimensional view.

[0075] Example devices and equipment

[0076] Figure 11A block diagram of an apparatus 1100 for switching viewpoints in a roaming creation application, according to some implementations of the present disclosure, is shown. The apparatus 1100 includes: an acquisition module 1110 configured to acquire a set of panoramic images associated with a set of spatial locations within a spatial range, the panoramic images representing the surrounding environment at each spatial location within the set of spatial locations; a two-dimensional rendering module 1120 configured to render a two-dimensional view of the spatial range in a two-dimensional window of the roaming creation application, the two-dimensional view including a planar map of the spatial range and a set of two-dimensional markers in the planar map associated with each set of spatial locations; a three-dimensional rendering module 1130 configured to render a three-dimensional view generated based on the set of panoramic images in a three-dimensional window of the roaming creation application; and a switching module 1140 configured to switch the viewpoint of the three-dimensional view to a spatial location associated with a two-dimensional marker in response to receiving an interaction with a two-dimensional marker in the set of two-dimensional markers, so as to render a panoramic image associated with the spatial location in the three-dimensional window.

[0077] According to an exemplary implementation of this disclosure, the two-dimensional presentation module 1120 includes: a map presentation module configured to present a planar map of spatial extent in a two-dimensional window; and a marker presentation module configured to present a set of two-dimensional markers based on a first style at a set of planar locations in the planar map that correspond to a set of spatial locations respectively.

[0078] According to an exemplary implementation of this disclosure, the apparatus 1100 further includes: a determination module configured to determine, in response to receiving an interaction with a two-dimensional marker, the production material of the panoramic image, the production material including at least one of the following: multiple view images for generating the panoramic image, and descriptive information of the spatial location; and an information presentation module configured to present relevant information about the production material.

[0079] According to an exemplary implementation of this disclosure, the two-dimensional rendering module 1120 further includes: a sensitive style rendering module configured to render two-dimensional markers based on a second style different from the first style in response to determining that a plurality of view images include sensitive information.

[0080] According to an exemplary implementation of this disclosure, the apparatus 1100 further includes: a mask style rendering module configured to render two-dimensional markers based on a third style different from the first and second styles in response to determining that sensitive information in a plurality of view images has been processed; and an updated image rendering module configured to render a new panoramic image associated with a spatial location in a three-dimensional window, the new panoramic image being generated using the processed plurality of view images.

[0081] According to an exemplary implementation of this disclosure, the 3D rendering module 1130 includes: an orientation determination module configured to determine the orientation of a viewpoint of a 3D view based on at least one of the following: the orientation of the viewpoint of the 3D view before a viewpoint switch, a predetermined orientation associated with a spatial location; and an orientation-based rendering module configured to render a panoramic image based on the orientation of the viewpoint.

[0082] According to an exemplary implementation of this disclosure, the two-dimensional rendering module 1120 further includes: a view frustum rendering module, configured to render a two-dimensional view frustum corresponding to the viewpoint of the three-dimensional view in a two-dimensional window; and wherein the three-dimensional rendering module further includes: a three-dimensional marker rendering module, configured to render a set of three-dimensional markers corresponding to a set of spatial locations in a three-dimensional window.

[0083] According to an exemplary implementation of this disclosure, the device 1100 further includes: a 3D switching module configured to switch the viewpoint of the 3D view to a spatial position corresponding to the 3D marker in response to receiving an interaction with a 3D marker in a set of 3D markers, so as to present a panoramic image associated with the spatial position in a 3D window.

[0084] According to an exemplary implementation of this disclosure, the apparatus 1100 further includes: a viewpoint update module configured to update at least one of the position and orientation of the viewpoint of the three-dimensional view in response to detecting a roaming action in the three-dimensional window; and a view frustum update module configured to update a two-dimensional view frustum in the two-dimensional window based on the updated viewpoint of the three-dimensional view.

[0085] Figure 12 A block diagram of a device 1200 capable of implementing various implementations of the present disclosure is shown. It should be understood that... Figure 12 The computing device 1200 shown is merely exemplary and should not be construed as limiting the functionality and scope of the implementation described herein. Figure 12 The computing device 1200 shown can be used to implement the method described above.

[0086] like Figure 12 As shown, computing device 1200 is in the form of a general-purpose computing device. Components of computing device 1200 may include, but are not limited to, one or more processors or processing units 1210, memory 1220, storage devices 1230, one or more communication units 1240, one or more input devices 1250, and one or more output devices 1260. Processing unit 1210 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 1220. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of computing device 1200.

[0087] Computing device 1200 typically includes multiple computer storage media. Such media can be any available media accessible to computing device 1200, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 1220 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 1230 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within computing device 1200.

[0088] The computing device 1200 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 12 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 1220 may include computer program product 1225 having one or more program modules configured to perform various methods or actions of various implementations of this disclosure.

[0089] The communication unit 1240 enables communication with other computing devices via a communication medium. Additionally, the functionality of the components of the computing device 1200 can be implemented as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device 1200 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.

[0090] Input device 1250 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 1260 can be one or more output devices, such as a monitor, speaker, printer, etc. Computing device 1200 can also communicate as needed with one or more external devices (not shown) via communication unit 1240. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with computing device 1200, or with any device that enables computing device 1200 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interfaces (not shown).

[0091] According to exemplary implementations of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to exemplary implementations of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above. According to exemplary implementations of this disclosure, a computer program product is provided that stores a computer program thereon, which, when executed by a processor, implements the methods described above.

[0092] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0093] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0094] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0096] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for switching viewpoints in a roaming creation application, comprising: Acquire a set of panoramic images that are associated with a set of spatial locations within a spatial range, wherein the panoramic images in the set of panoramic images represent the surrounding environment at the spatial locations in the set of spatial locations; A two-dimensional view of the spatial range is presented in the two-dimensional window of the roaming creation application. The two-dimensional view includes a planar map of the spatial range and a set of two-dimensional markers in the planar map that are respectively associated with the set of spatial locations. The material window of the roaming creation application presents material information for generating the set of panoramic images associated with the set of two-dimensional markers, wherein the style of the set of two-dimensional markers indicates the state of the material information of the creation material, and the creation material includes multiple view images for generating the panoramic images; A 3D view generated based on the set of panoramic images is presented in the 3D window of the roaming creation application; as well as In response to receiving an interaction with a two-dimensional marker in the set of two-dimensional markers: The viewpoint of the three-dimensional view is switched to a spatial location associated with the two-dimensional marker, so that a panoramic image associated with the spatial location is presented in the three-dimensional window; as well as The multiple view images used to generate the panoramic image are presented in the material window so that the user of the roaming creation application can edit the multiple view images.

2. The method of claim 1, wherein presenting the two-dimensional view comprises: The planar map of the spatial range is presented in the two-dimensional window; as well as At a set of planar locations on the planar map that correspond to the set of spatial locations, the set of two-dimensional markers are presented based on a first style.

3. The method according to claim 2, further comprising: In response to receiving an interaction with the two-dimensional marker, The material used to create the panoramic image is determined, and the material further includes descriptive information about the spatial location; as well as Present relevant information about the materials used in the production process.

4. The method of claim 3, wherein presenting the two-dimensional view further comprises: In response to determining that the plurality of view images contain sensitive information, the two-dimensional markup is rendered based on a second style that is different from the first style.

5. The method according to claim 4, further comprising: In response to determining that the sensitive information in the plurality of view images has been processed, the two-dimensional marker is presented based on a third style that is different from the first style and the second style; as well as A new panoramic image associated with the spatial location is presented in the three-dimensional window, the new panoramic image being generated using the processed plurality of view images.

6. The method of claim 1, wherein presenting the panoramic image in the three-dimensional window comprises: The orientation of the viewpoint of the three-dimensional view is determined based on at least one of the following: the orientation of the viewpoint of the three-dimensional view before the viewpoint switch, and a predetermined orientation associated with the spatial location; as well as The panoramic image is presented based on the orientation of the viewpoint.

7. The method of claim 6, wherein presenting the two-dimensional view in the two-dimensional window further comprises: A two-dimensional view frustum corresponding to the viewpoint of the three-dimensional view is presented in the two-dimensional window; as well as Presenting the three-dimensional view in the three-dimensional window further includes presenting a set of three-dimensional markers in the three-dimensional window that correspond to the set of spatial locations.

8. The method of claim 7, further comprising: In response to receiving an interaction with a 3D marker in the set of 3D markers, the viewpoint of the 3D view is switched to a spatial position corresponding to the 3D marker, so as to present a panoramic image associated with the spatial position in the 3D window.

9. The method according to claim 7, further comprising: In response to the detection of a roaming motion in the three-dimensional window, Update at least one of the position and orientation of the viewpoint in the 3D view; as well as The two-dimensional view frustum is updated in the two-dimensional window based on the updated viewpoint of the three-dimensional view.

10. A device for switching viewpoints in a roaming production application, comprising: The acquisition module is configured to acquire a set of panoramic images associated with a set of spatial locations within a spatial range, wherein the panoramic images in the set of panoramic images represent the surrounding environment at the spatial locations in the set of spatial locations; A two-dimensional rendering module is configured to present a two-dimensional view of the spatial extent in a two-dimensional window of the roaming creation application. The two-dimensional view includes a planar map of the spatial extent and a set of two-dimensional markers in the planar map that are respectively associated with the set of spatial locations. The two-dimensional presentation module is further configured to present material information of production materials for generating the set of panoramic images associated with the set of two-dimensional markers in the material window of the roaming production application, wherein the style of the set of two-dimensional markers indicates the state of the material information of the production materials, and the production materials include multiple view images for generating the panoramic images. A 3D rendering module is configured to render a 3D view generated based on the set of panoramic images in a 3D window of the roaming creation application; as well as A switching module is configured to respond to receiving an interaction with a two-dimensional marker in the set of two-dimensional markers: The viewpoint of the three-dimensional view is switched to a spatial location associated with the two-dimensional marker, so that a panoramic image associated with the spatial location is presented in the three-dimensional window; as well as The multiple view images used to generate the panoramic image are presented in the material window so that the user of the roaming creation application can edit the multiple view images.

11. The apparatus of claim 10, wherein the two-dimensional presentation module comprises: A map rendering module is configured to render the planar map of the spatial extent in the two-dimensional window; as well as A marker rendering module is configured to render a set of two-dimensional markers based on a first style at a set of planar locations in the planar map that correspond to the set of spatial locations.

12. The apparatus of claim 11, further comprising: A determination module is configured to determine the fabrication material of the panoramic image in response to receiving an interaction with the two-dimensional marker, the fabrication material further including descriptive information of the spatial location; as well as The information presentation module is configured to present relevant information about the manufacturing materials.

13. The apparatus of claim 12, wherein the two-dimensional presentation module further comprises: A sensitive style rendering module is configured to render the two-dimensional markup based on a second style different from the first style in response to determining that the plurality of view images contain sensitive information.

14. The apparatus of claim 13, further comprising: A mask style rendering module is configured to render the two-dimensional marker based on a third style that is different from the first style and the second style in response to determining that the sensitive information in the plurality of view images has been processed; as well as An updated image rendering module is configured to render a new panoramic image associated with the spatial location in the three-dimensional window, the new panoramic image being generated using the processed plurality of view images.

15. The apparatus of claim 10, wherein the three-dimensional rendering module comprises: An orientation determination module is configured to determine the orientation of the viewpoint of the three-dimensional view based on at least one of the following: the orientation of the viewpoint of the three-dimensional view before the viewpoint switch, or a predetermined orientation associated with the spatial location; as well as An orientation-based presentation module is configured to present the panoramic image based on the orientation of the viewpoint.

16. The apparatus of claim 15, wherein the two-dimensional presentation module further comprises: A view frustum rendering module is configured to render a two-dimensional view frustum corresponding to the viewpoint of the three-dimensional view in the two-dimensional window; as well as The three-dimensional rendering module further includes a three-dimensional marker rendering module, configured to render a set of three-dimensional markers corresponding to the set of spatial locations in the three-dimensional window.

17. The apparatus of claim 16, further comprising: A 3D switching module is configured to, in response to receiving an interaction with a 3D marker in the set of 3D markers, switch the viewpoint of the 3D view to a spatial position corresponding to the 3D marker, so as to present a panoramic image associated with the spatial position in the 3D window.

18. The apparatus of claim 16, further comprising: A viewpoint update module is configured to update at least one of the position and orientation of the viewpoint of the 3D view in response to detecting a roaming action in the 3D window; and A view frustum update module is configured to update the two-dimensional view frustum in the two-dimensional window based on the updated viewpoint of the three-dimensional view.

19. An electronic device comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 9.

20. A computer-readable storage medium having a computer program stored thereon, the computer program causing the processor to implement the method according to any one of claims 1 to 9 when executed by a processor.

Images