Subspace connection relationship determination method and apparatus, and computer-readable storage medium

Abstract

Embodiments of the present application provide a sub-space connection relationship determination method, device and computer readable storage medium, wherein the method comprises: obtaining a plurality of groups of panoramic images corresponding to a plurality of groups of sub-spaces in a three-dimensional space respectively; determining the orientation of each connection opening according to the position of each connection opening on the corresponding second panoramic image and the camera orientation when each second panoramic image is captured; obtaining the correspondence between two first panoramic images in each group of panoramic images and a part of the second panoramic image in the group of panoramic images to determine the relative orientation of two adjacent sub-spaces in each group of sub-spaces; and identifying the first panoramic images corresponding to the same sub-space in each group of panoramic images to determine the connection relationship of each group of sub-spaces, and determining the connection relationship of each sub-space in the three-dimensional space based on the determined connection relationship of each group of sub-spaces, the orientation of each connection opening and the relative orientation of two adjacent sub-spaces in each group of sub-spaces.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of image processing, and in particular, to a method and apparatus for determining a sub-space connection relationship based on processing of a panoramic image, and a computer readable storage medium. BACKGROUND

[0002] Currently, in the fields of virtual reality (VR), real estate, etc., it is often necessary to obtain a two-dimensional layout image (e.g., a floor plan) reflecting the two-dimensional layout of the entire three-dimensional space, which can enable a user to intuitively understand the sub-space division of the entire three-dimensional space, and also serve as a guide image during virtual roaming. Initially, the two-dimensional layout image is usually obtained by professional personnel through on-site measurement and drawing. In recent years, with the development of computer vision and artificial intelligence technologies, it has been realized to automatically generate a two-dimensional layout image from multiple panoramic images obtained by photographing a three-dimensional scene. In the process of generating a two-dimensional layout image, one of the key steps is to obtain the connection relationship of each sub-space (e.g., each room in a house) in the three-dimensional space.

[0003] In determining the connection relationship of the sub-spaces, a conventional method can determine the relative pose (i.e., a rotation matrix and a translation matrix) of the camera by using the sensor information of the camera when photographing panoramic images in two sub-spaces, respectively. However, this method must ensure that the distance between the two photographing locations is not too far, so as to ensure that the three-dimensional scenes corresponding to the two panoramic images obtained by photographing have sufficient coincidence.

[0004] Therefore, there is a need for a new method for determining the connection relationship of sub-spaces, so as to automatically generate a two-dimensional layout image of a three-dimensional space based on the determined connection relationship of the sub-spaces. SUMMARY

[0005] To solve the above technical problems, according to one aspect of the present application, a method for determining connection relationship of subspaces is provided, comprising: obtaining a plurality of groups of panoramic images respectively corresponding to a plurality of groups of subspaces in a three-dimensional space, wherein each two adjacent subspaces in the three-dimensional space form a group of subspaces, and each group of panoramic images comprises two first panoramic images respectively taken in the two adjacent subspaces in each group of subspaces and a second panoramic image taken at a connection opening connecting the two adjacent subspaces; determining an orientation of each connection opening according to a position of each connection opening on the corresponding second panoramic image and a camera orientation when each second panoramic image is taken; obtaining a correspondence between the two first panoramic images in each group of panoramic images and a part of the second panoramic image in the group of panoramic images, and determining relative orientations of the two adjacent subspaces in each group of subspaces based on the correspondence; and identifying the first panoramic images corresponding to the same subspace in each group of panoramic images, and determining a connection relationship of the groups of subspaces based on the same subspace, and determining a connection relationship of each subspace in the three-dimensional space based on the determined connection relationship of the groups of subspaces, the orientation of each connection opening and the relative orientations of the two adjacent subspaces in each group of subspaces.

[0006] According to another aspect of the present application, a device for determining connection relationship of subspaces is provided, comprising: an obtaining unit configured to obtain a plurality of groups of panoramic images respectively corresponding to a plurality of groups of subspaces in a three-dimensional space, wherein each two adjacent subspaces in the three-dimensional space form a group of subspaces, and each group of panoramic images comprises two first panoramic images respectively taken in the two adjacent subspaces in each group of subspaces and a second panoramic image taken at a connection opening connecting the two adjacent subspaces; a first determining unit configured to determine an orientation of each connection opening according to a position of each connection opening on the corresponding second panoramic image and a camera orientation when each second panoramic image is taken; a second determining unit configured to obtain a correspondence between the two first panoramic images in each group of panoramic images and a part of the second panoramic image in the group of panoramic images, and determine relative orientations of the two adjacent subspaces in each group of subspaces based on the correspondence; and a third determining unit configured to identify the first panoramic images corresponding to the same subspace in each group of panoramic images, and determine a connection relationship of the groups of subspaces based on the same subspace, and determine a connection relationship of each subspace in the three-dimensional space based on the determined connection relationship of the groups of subspaces, the orientation of each connection opening and the relative orientations of the two adjacent subspaces in each group of subspaces.

[0007] According to still another aspect of the present application, there is provided a sub-space connection relationship determining apparatus, comprising: a processor; and a memory having computer program instructions stored therein, wherein the computer program instructions, when executed by the processor, cause the processor to perform the following steps: obtaining a plurality of sets of panoramic images respectively corresponding to a plurality of sets of sub-spaces in a three-dimensional space, wherein each two adjacent sub-spaces in the three-dimensional space forms a set of sub-spaces, and each set of panoramic images comprises two first panoramic images respectively taken within two adjacent sub-spaces in each set of sub-spaces and a second panoramic image taken at a connection opening connecting the two adjacent sub-spaces; determining an orientation of each connection opening according to a position of the connection opening on the corresponding second panoramic image and a camera orientation when the corresponding second panoramic image is taken; obtaining a correspondence between the two first panoramic images in each set of panoramic images and a portion of the second panoramic image in the set of panoramic images, and determining relative orientations of the two adjacent sub-spaces in each set of sub-spaces based on the correspondence; and identifying the first panoramic images corresponding to a same sub-space in the plurality of sets of panoramic images, and determining connection relationships of the plurality of sets of sub-spaces based on the same sub-space, and determining connection relationships of the sub-spaces in the three-dimensional space based on the determined connection relationships of the plurality of sets of sub-spaces, the orientations of the connection openings and the relative orientations of the two adjacent sub-spaces in each set of sub-spaces.

[0008] According to still another aspect of the present application, there is provided a computer readable storage medium having computer program instructions stored therein, wherein the computer program instructions, when executed by a processor, implement a method comprising: obtaining a plurality of sets of panoramic images respectively corresponding to a plurality of sets of sub-spaces in a three-dimensional space, wherein each two adjacent sub-spaces in the three-dimensional space forms a set of sub-spaces, and each set of panoramic images comprises two first panoramic images respectively taken within two adjacent sub-spaces in each set of sub-spaces and a second panoramic image taken at a connection opening connecting the two adjacent sub-spaces; determining an orientation of each connection opening according to a position of the connection opening on the corresponding second panoramic image and a camera orientation when the corresponding second panoramic image is taken; obtaining a correspondence between the two first panoramic images in each set of panoramic images and a portion of the second panoramic image in the set of panoramic images, and determining relative orientations of the two adjacent sub-spaces in each set of sub-spaces based on the correspondence; and identifying the first panoramic images corresponding to a same sub-space in the plurality of sets of panoramic images, and determining connection relationships of the plurality of sets of sub-spaces based on the same sub-space, and determining connection relationships of the sub-spaces in the three-dimensional space based on the determined connection relationships of the plurality of sets of sub-spaces, the orientations of the connection openings and the relative orientations of the two adjacent sub-spaces in each set of sub-spaces.

[0009] According to the sub-space connection relationship determination method, device and computer readable storage medium, the connection relationship of each sub-space in the three-dimensional space can be determined based on the panoramic images of each sub-space in the three-dimensional space and the panoramic images of the connection openings connecting adjacent sub-spaces, and a two-dimensional layout image of the three-dimensional space can be automatically generated based on the determined sub-space connection relationship, thereby improving the generation efficiency of the two-dimensional layout image. BRIEF DESCRIPTION OF DRAWINGS

[0010] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0011] Figure 1 A flow chart of a sub-space connection relationship determination method 100 according to an embodiment of the present application is shown.

[0012] Figure 2 A schematic diagram of a two-dimensional layout image of a three-dimensional space according to an embodiment of the present application is shown.

[0013] Figure 3 A flow chart of a method 300 for determining the orientation of each connection opening is shown.

[0014] Figure 4 An example of a reference orientation according to an embodiment of the present application is shown.

[0015] Figure 5 A schematic diagram of detecting the position of a connection opening according to an embodiment of the present application is shown.

[0016] Figure 6 A schematic diagram of obtaining the correspondence between a first panoramic image and a portion of a second panoramic image is shown.

[0017] Figure 7 A schematic diagram of the determination result of the relative orientation of two adjacent sub-spaces according to an embodiment of the present application is shown.

[0018] Figure 8 A schematic diagram of generating a two-dimensional layout image based on the determined sub-space connection relationship according to an embodiment of the present application is shown.

[0019] Figure 9 A block diagram of a sub-space connection relationship determination device 900 according to an embodiment of the present application is shown.

[0020] Figure 10 A block diagram of a sub-space connection relationship determination device 1000 according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] The subspace connection relationship determination method, device and computer readable storage medium according to embodiments of the present application will be described below with reference to the accompanying drawings. In the drawings, the same reference numerals are used throughout to designate the same elements. It should be understood that the embodiments described herein are merely illustrative and should not be construed as limiting the scope of the present application.

[0022] The subspace connection relationship determination method, device and computer readable storage medium according to embodiments of the present application will be described below with reference to the accompanying drawings. In the drawings, the same reference numerals are used throughout to designate the same elements. It should be understood that the embodiments described herein are merely illustrative and should not be construed as limiting the scope of the present application. Figure 1 The subspace connection relationship determination method according to embodiments of the present application will be described below. Figure 1 A flowchart of the subspace connection relationship determination method 100 is shown.

[0023] As shown in step S101, a plurality of sets of panoramic images respectively corresponding to a plurality of sets of subspaces in a three-dimensional space are acquired, wherein each two adjacent subspaces in the three-dimensional space is a set of subspaces, and each set of panoramic images comprises two first panoramic images respectively taken within two adjacent subspaces in each set of subspaces and a second panoramic image taken at a connection opening connecting the two adjacent subspaces. Figure 1

[0024] In this step, the plurality of sets of panoramic images respectively corresponding to the plurality of sets of subspaces in the three-dimensional space can be acquired in various ways. For example, a panoramic camera (such as a fisheye camera, a binocular camera, etc.) can be used to acquire panoramic images (such as equirectangular projection images). For another example, a plurality of perspective images can be acquired using a normal camera, and the plurality of perspective images can be stitched to acquire the panoramic images respectively.

[0025] Figure 2 A schematic diagram of a two-dimensional layout image of a three-dimensional space according to embodiments of the present application is shown. The three-dimensional space can be composed of a plurality of subspaces, which can be subspaces obtained by various ways of partitioning the three-dimensional space, and two adjacent subspaces can be connected by a connection opening therebetween. For example, when the three-dimensional space is an indoor space of a building, the indoor space of the building can be partitioned into a plurality of rooms independent of each other by objects such as walls, partitions or windows to serve as subspaces, and two adjacent rooms can be connected by a door as the connection opening. According to embodiments of the present application, the connection relationship of each subspace in the three-dimensional space can be determined by panoramic images taken within each subspace in the three-dimensional space and a panoramic image taken below the connection opening, so as to automatically generate a two-dimensional layout image similar to the image shown. Figure 2 As shown in the image shown, the two-dimensional layout image can be generated automatically. In addition, since the panoramic image is taken below the connection opening as the connection image of the images corresponding to the two adjacent subspaces, it is not required that the distance between the places where the two adjacent subspaces are taken cannot be too far.

[0026] In Figure 2 ​In the example, the three-dimensional space can include three subspaces, where every two adjacent subspaces can be a group of subspaces, i.e., Figure 2 Two adjacent subspaces within the dashed box can be considered a set of subspaces, while Figure 2 The two adjacent subspaces within the solid-lined frame can be another set of subspaces. In this step, panoramic images corresponding to each set of subspaces can be acquired. Each set of panoramic images can include two first panoramic images taken within two adjacent subspaces in each set, and a second panoramic image taken through the connecting opening connecting the two adjacent subspaces. For example, for... Figure 2 In this context, one set of panoramic images may include two first panoramic images taken at positions P1 and P3 within two adjacent subspaces, and a second panoramic image taken at the connecting opening O1 (i.e., at position P2) connecting these two adjacent subspaces; while another set of panoramic images may include two first panoramic images taken at positions P4 and P6 within another two adjacent subspaces, and a second panoramic image taken at the connecting opening O2 (i.e., at position P5) connecting these two adjacent subspaces, wherein two panoramic images are taken at positions P3 and P4 within the same subspace. It should be noted that although in Figure 2 The illustration shows a three-dimensional space comprising only three subspaces, but the present invention is not limited to this, and the method for determining the subspace connection relationship according to embodiments of the present invention can be applied to three-dimensional spaces comprising a greater number of subspaces. Furthermore, when more than one panoramic image is captured within the same subspace, multiple captures can be taken at the same location within that subspace to obtain the more than one panoramic image; this is not a limitation.

[0027] In this step, the first and second panoramic images of each subspace can be generated based on a predetermined shooting order, and the groups of panoramic images can be obtained by grouping the first and second panoramic images of each subspace based on the predetermined shooting order. For example, continuing to refer to... Figure 2If the first panoramic image and the second panoramic image of each group of subspaces are generated based on a predetermined shooting order of P1→P2→P3→P4→P5→P6, the first three panoramic images can be divided into one group and the last three panoramic images can be divided into another group based on the predetermined shooting order. For example, if the first panoramic image and the second panoramic image of each group of subspaces are generated based on a predetermined shooting order of P2→P5→P1→P3→P4→P6, the first, third and fourth panoramic images can be divided into one group and the second, fifth and sixth panoramic images can be divided into another group based on the predetermined shooting order. Therefore, the embodiment of the present application can automatically group the input panoramic images according to the predetermined shooting order input or set by the user, thereby facilitating subsequent processing.

[0028] It should be noted that the camera may not be completely horizontally placed during actual shooting, resulting in a certain degree of skew in the panoramic images obtained by shooting. At this time, the first panoramic image and the second panoramic image can be subjected to vertical correction processing before subsequent processing of the first panoramic image and the second panoramic image, so as to improve the accuracy of subsequent processing.

[0029] In step S102, the orientation of each connection opening is determined according to the position of each connection opening on the corresponding second panoramic image and the camera orientation when each second panoramic image is shot.

[0030] Figure 3 A flowchart of a method for determining the orientation of each connection opening is shown. In which, Figure 3 Each step shown is a sub-step of step S102.

[0031] In sub-step S102-1, the camera orientation when each second panoramic image is shot is determined based on the camera sensor data when each second panoramic image is shot.

[0032] Specifically, the camera orientation when any one of the second panoramic images is shot can be taken as a reference orientation, so as to determine the camera orientation when other second panoramic images are shot based on the reference orientation. Figure 4 An example of the reference orientation according to the embodiment of the present application is shown. In which, Figure 4 The left side of shows a second panoramic image 401 shot at connection opening O1, while Figure 4 The arrow direction X shown on the right side of shows the camera orientation when the second panoramic image 401 is shot from the top view, and the camera orientation corresponds to the central image column position of the second panoramic image 401. In which, the central image column position is the position of the most central column of images of the panoramic image in the horizontal direction (for example, Figure 4(The image shows the 0° position in the panoramic image coordinate system). In this example, the camera orientation X when capturing the second panoramic image 401 at the connecting opening O1 can be used as the reference orientation. Furthermore, in other examples, the camera orientation when capturing the second panoramic image at the connecting opening O2 can also be used as the reference orientation; this is not a limitation.

[0033] When the camera orientation X when capturing the second panoramic image 401 through connecting opening O1 is used as a reference orientation, the angle between the camera orientation when capturing other second panoramic images and this reference orientation can be determined based on camera sensor data, thereby obtaining the camera orientation when capturing other second panoramic images. Specifically, based on the camera sensor data when capturing second panoramic images through connecting openings O1 and O2, the angle between the camera orientation when capturing the second panoramic image through connecting opening O2 and the reference orientation X can be determined, thereby obtaining the camera orientation Y when capturing the second panoramic image through connecting opening O2. Figure 4 (Not shown). For example, with the determined included angle α, the camera orientation Y = X + α can be obtained when the second panoramic image is captured at the connecting opening O2. Therefore, the orientations of connecting openings O1 and O2 can be ultimately determined based on the camera orientations X and Y through subsequent steps.

[0034] In sub-step S102-2, the location of the connection opening is detected on each of the second panoramic images.

[0035] In one example, each second panoramic image can be input into a trained object detection model to obtain two bounding boxes corresponding to the connection openings output by the object detection model. The center position of each of these two bounding boxes in the horizontal direction can be used as the location of the detected connection opening. The object detector model can be a model trained based on a deep neural network object detection algorithm, such as the YOLOv3 object detection model; however, other types of object detection models can also be used, without limitation.

[0036] Figure 5 A schematic diagram of detecting the location of a connection opening according to an embodiment of the present invention is shown.

[0037] in, Figure 5 The left side shows the same as Figure 4 The image shown is the same as the second panoramic image 401 taken at the connecting opening O1, and Figure 5 The right side further shows a second panoramic image 402 taken at the connecting opening O2. (See image 402.) Figure 5As shown, by inputting the second panoramic image 401 taken at the connection opening O1 into the object detection model, two bounding boxes 501 and 502 corresponding to the connection opening O1 output by the object detection model can be obtained, and the most central positions 503 and 504 of the bounding boxes 501 and 502 in the horizontal direction can be respectively taken as the positions of the detected connection opening O1, from which Figure 5 As can be seen, it is approximately located at the positions of 90° and -90° in the panoramic image coordinate system.

[0038] In another example, in addition to using the object detection model, the positions of the connection openings can also be detected by calculating the variance of each column of pixels on the respective second panoramic image. Specifically, since the connection openings usually have less variation in texture, color, etc., the difference of the column of pixels corresponding to the connection opening should be the smallest compared to other columns of pixels. Accordingly, the variance of each column of pixels on the second panoramic image can be calculated respectively, and the horizontal coordinate of the column of pixels with the smallest variance can be taken as one of the positions of the detected connection opening. And another horizontal coordinate which is spaced from the horizontal coordinate by half of the horizontal width of the panoramic image can be taken as another position of the connection opening.

[0039] Similarly, the above steps can be repeated on the second panoramic image 402 taken at the connection opening O2, so as to obtain the positions 505 and 506 of the detected connection opening O2. From Figure 5 As can be seen, it is approximately located at the positions of 0° and 180° in the panoramic image coordinate system.

[0040] In sub-step S102-3, the orientation of each connection opening is determined according to the position of the detected connection opening on the respective second panoramic image and the camera orientation.

[0041] In this sub-step, the orientation of each connection opening can be determined according to the difference between the position of the connection opening on the respective second panoramic image and the central image column position and the camera orientation. Wherein, the orientation of the connection opening can be represented by the direction of the straight line passing through the connection opening.

[0042] Continuing to refer to Figure 5 In the second panoramic image 401, the difference between the positions 503 and 504 of the detected connection opening O1 and the central image column position (as mentioned above, the central image column position is the position of 0° in the panoramic image coordinate system, corresponding to the camera orientation X when the second panoramic image 401 is taken) is approximately 90° and -90°, from which it can be determined that the included angle between the orientation of the connection opening O1 and the camera orientation X is 90°, i.e. the orientations of the two are approximately perpendicular to each other.

[0043] Similarly, in the second panoramic image 402, the differences between the detected positions 505 and 506 of the connecting opening O2 and the position of the central image column (as described above, the central image column position is the 0° position in the panoramic image coordinate system, corresponding to the camera orientation Y when the second panoramic image 402 was captured) are approximately 0° and 180°. This difference can be used to determine that the angle between the orientation of the connecting opening O2 and the camera orientation Y is 0°, that is, the orientations of the two are approximately parallel to each other.

[0044] After obtaining the angles between the orientations of each connecting opening and the corresponding camera orientations, the orientations of each connecting opening can be further converted into absolute orientations corresponding to the same reference orientation using the camera orientations, thereby finally determining the orientation of the connecting opening. Specifically, in this example, since the reference orientation has been determined as camera orientation X, and the angle between the orientation of connecting opening O1 and camera orientation X is 90°, the final determined orientation of the connecting opening is X+90°. Furthermore, when the angle between camera orientation Y and the reference orientation is α (i.e., camera orientation Y = X+α), and the angle between the orientation of connecting opening O2 and camera orientation Y is 0°, the final determined orientation of connecting opening O2 is X+α. Through the above process, the orientations of each connecting opening can be determined, and then the process proceeds to step S103 for determining the relative orientations of two adjacent subspaces.

[0045] In step S103, the correspondence between two first panoramic images in each group of panoramic images and a portion of a second panoramic image in the same group of panoramic images is obtained, and the relative orientation of two adjacent subspaces in each group of subspaces is determined based on the correspondence.

[0046] In this step, the second panoramic image can be divided into two sub-images based on the location of the connecting opening detected on the second panoramic image in the set of panoramic images. Subsequently, the similarity between the two first panoramic images in the set of panoramic images and the two sub-images can be compared to obtain the correspondence.

[0047] Figure 6 A schematic diagram illustrating the correspondence between two first panoramic images and a portion of a second panoramic image is shown. For example... Figure 6 As shown, by dividing the second panoramic image 401 in a set of panoramic images by detecting the position of the connecting opening O1, the second panoramic image 401 can be divided into two sub-images 601 and 602. Sub-image 601 is an image obtained by dividing the middle part of the second panoramic image 401, while sub-image 602 is an image obtained by dividing the left and right parts of the second panoramic image 402 and stitching them together. These two sub-images correspond to the two adjacent subspaces connected by the connecting opening O1, respectively.

[0048] Further, Figure 6 It is also shown that the first panoramic image 603 and the first panoramic image 604 in the set of panoramic images are respectively taken within two adjacent subspaces. By comparing the similarity of the first panoramic image 603 and 604 with the two sub-images 601 and 602, it can be determined that the first panoramic image 603 is more similar to the sub-image 601, while the first panoramic image 604 is more similar to the sub-image 602. Thus, it can be determined that the first panoramic image 603 corresponds to the sub-image 601, while the first panoramic image 604 corresponds to the sub-image 602. In this step, a first panoramic image and a sub-image can be input into the trained comparison model, so as to obtain the similarity of the two images output by the model, so as to determine the correspondence between the first panoramic image and the sub-image based on the similarity. Wherein, the comparison model can be a model trained based on a twin neural network, however, the specific type of the model is not limited by the present application. Accordingly, based on the determined correspondence between the first panoramic image and the sub-image, it can be determined that the first panoramic image 603 corresponds to the middle part of the second panoramic image 401, while the first panoramic image 604 corresponds to the two side parts of the second panoramic image 401, so as to obtain the relative orientation of the subspaces corresponding to the two first panoramic images.

[0049] Figure 7 A schematic diagram showing the determination result of the relative orientation of two adjacent subspaces according to an embodiment of the present application is shown. Wherein, the relative orientation defines the orientation of the two adjacent subspaces relative to the connecting opening connecting them, that is, it defines which side or direction the two adjacent subspaces are located relative to the connecting opening. Since it has been determined in step S102 that the orientation of the connecting opening O1 is X+90°, the two adjacent subspaces connected by the connecting opening O1 should be located in the X direction and the -X direction of the connecting opening respectively. Therefore, after determining that the first panoramic image 603 corresponds to the middle part of the second panoramic image 401 and the first panoramic image 604 corresponds to the two side parts of the second panoramic image 401, it can be determined that the relative orientation of the subspace corresponding to the first panoramic image 603 (shown as R2’ in Figure 7 ) is the X direction of the connecting opening O1, while the relative orientation of the subspace corresponding to the first panoramic image 604 (shown as R1’ in Figure 7 ) is the -X direction of the connecting opening O1, that is, the relative orientation of the two adjacent subspaces R1’ and R2’ in the set of subspaces shown on the left of Figure 7 is obtained.

[0050] Similarly, the above step S103 processing can be repeated for another set of panoramic images taken at the connecting opening O2 and within the two adjacent subspaces connected by the connecting opening O2, so as to obtain the relative orientation of the two adjacent subspaces R1’ and R2’ in the set of subspaces shown on the right of Figure 7the relative orientation of two adjacent subspaces R3' and R4' in the other group of subspaces shown on the right side of the three-dimensional space.

[0051] In step S104, the first panoramic image corresponding to the same subspace in each group of panoramic images is identified, and the connection relationship of each group of subspaces is determined based on the same subspace. The connection relationship of each subspace in the three-dimensional space is determined based on the determined connection relationship of each group of subspaces, the orientation of each connection opening, and the relative orientation of two adjacent subspaces in each group of subspaces.

[0052] In this step, the similarity of the first panoramic image in one group of panoramic images and the first panoramic image in another group of panoramic images can be compared to identify the first panoramic image corresponding to the same subspace in each group of panoramic images. As described above, the first panoramic image in one group of panoramic images and the first panoramic image in another group of panoramic images can be input into a comparison model trained based on a twin neural network, so as to obtain the similarity of the two output images, so as to identify the first panoramic image corresponding to the same subspace based on the similarity.

[0053] Continuing to refer to Figure 7 , having determined that there are two groups of subspaces in the three-dimensional space, the connection relationship between the two groups of subspaces will be determined next, i.e., it will be determined that the two groups of subspaces should be connected through which specific subspaces to form a complete three-dimensional space. In this example, when it is determined that the first panoramic image corresponding to the subspace R2' in one group of subspaces and the first panoramic image corresponding to the subspace R3' in the other group of subspaces have a similarity higher than a threshold, it can be identified that the subspaces R2' and R3' are actually the same subspace, and thus the connection relationship of each group of subspaces can be determined based on the same subspace, i.e., it can be determined that the two groups of subspaces should be connected through the subspaces R2' and R3' to form a complete three-dimensional space. Accordingly, the group of subspaces on the left side and the other group of subspaces on the right side can be connected by overlapping the subspace R2' and the subspace R3' with each other. Figure 7 Figure 7

[0054] After connecting each group of subspaces with each other based on the connection relationship of each group of subspaces, the connection relationship of each subspace in the three-dimensional space can be determined based on the orientation of each connection opening and the relative orientation of two adjacent subspaces in each group of subspaces determined in the foregoing steps.

[0055] In addition, after the connection relationship of each subspace is determined, a two-dimensional layout image of the three-dimensional space can be further generated based on the determined connection relationship of each subspace.

[0056] Figure 8 ​​A schematic diagram of generating a two-dimensional layout image based on the determined sub-space connection relationship according to an embodiment of the present application is shown. As shown in Figure 8 Based on the determined connection relationship of each sub-space, by coinciding the sub-space R2' and the sub-space R3' in Figure 7 to form the sub-space R2" in Figure 8 , a two-dimensional layout image similar to the image shown in Figure 2 can be generated.

[0057] In summary, according to the above-mentioned sub-space connection relationship determination method according to an embodiment of the present application, the connection relationship of each sub-space in a three-dimensional space can be determined based on the panoramic images taken in each sub-space in the three-dimensional space and the panoramic images taken at the connection openings connecting each group of adjacent sub-spaces, so as to automatically generate a two-dimensional layout image of the three-dimensional space based on the determined sub-space connection relationship, thereby improving the generation efficiency of the two-dimensional layout image.

[0058] The sub-space connection relationship determination apparatus according to an embodiment of the present application will be described below with reference to Figure 9 . Figure 9 A block diagram of the sub-space connection relationship determination apparatus 900 according to an embodiment of the present application is shown. As shown in Figure 9 , the sub-space connection relationship determination apparatus 900 comprises an acquisition unit 910, a first determination unit 920, a second determination unit 930 and a third determination unit 940. In addition to these units, the sub-space connection relationship determination apparatus 900 can also comprise other components, however, since these components are irrelevant to the content of the embodiments of the present application, the illustration and description thereof are omitted here. Moreover, since the specific details of the operations performed by the image processing apparatus 1100 according to an embodiment of the present application are the same as the details described above with reference to Figures 1-10 , the repeated description of the same details is omitted here to avoid repetition.

[0059] First, the acquisition unit 910 acquires a plurality of groups of panoramic images respectively corresponding to a plurality of groups of sub-spaces in a three-dimensional space, wherein each two adjacent sub-spaces in the three-dimensional space is a group of sub-spaces, and each group of panoramic images comprises two first panoramic images respectively taken within two adjacent sub-spaces in each group of sub-spaces and a second panoramic image taken at a connection opening connecting the two adjacent sub-spaces.

[0060] The first panoramic images and the second panoramic images of each group of sub-spaces can be generated based on a predetermined shooting order, and the plurality of groups of panoramic images can be obtained by grouping the first panoramic images and the second panoramic images of each group of sub-spaces based on the predetermined shooting order. For example, with reference to Figure 2If the first panoramic image and the second panoramic image of each group of subspaces are generated based on a predetermined shooting order of P1→P2→P3→P4→P5→P6, the obtaining unit 910 can divide the first three panoramic images into one group and divide the last three panoramic images into another group based on the predetermined shooting order. For another example, if the first panoramic image and the second panoramic image of each group of subspaces are generated based on a predetermined shooting order of P2→P5→P1→P3→P4→P6, the obtaining unit 910 can divide the first, third and fourth panoramic images into one group and divide the second, fifth and sixth panoramic images into another group based on the predetermined shooting order. Therefore, the obtaining unit 910 of the embodiment of the present application can automatically group the input panoramic images according to the predetermined shooting order input or set by the user in advance, thereby facilitating subsequent processing.

[0061] It should be noted that, since the camera may not be completely horizontally placed during actual shooting, the panoramic images obtained by shooting may be skewed to a certain extent. At this time, before the first panoramic image and the second panoramic image obtained are processed subsequently, the obtaining unit 910 can first perform vertical correction processing on the first panoramic image and the second panoramic image, so as to improve the accuracy of subsequent processing.

[0062] The first determining unit 920 determines the orientation of each connection opening according to the position of each connection opening on the corresponding second panoramic image and the camera orientation when each second panoramic image is shot.

[0063] Firstly, the first determining unit 920 determines the camera orientation when each second panoramic image is shot based on the camera sensor data when each second panoramic image is shot.

[0064] Specifically, the first determining unit 920 can first take the camera orientation when any one second panoramic image is shot as a reference orientation, so as to determine the camera orientation when other second panoramic images are shot based on the reference orientation. Figure 4 An example of the reference orientation according to the embodiment of the present application is shown. Wherein, Figure 4 the left side of Fig. 4 shows a second panoramic image 401 shot at the connection opening O1, and Figure 4 the arrow direction X shown on the right side of Fig. 4 is the camera orientation when the second panoramic image 401 is shot from the overhead direction, and the camera orientation corresponds to the central image column position of the second panoramic image 401. Wherein, the central image column position is the position of the most central column of images of the panoramic image in the horizontal direction (for example, Figure 4(The image shows the 0° position in the panoramic image coordinate system). In this example, the first determining unit 920 can use the camera orientation X when capturing the second panoramic image 401 at the connecting opening O1 as a reference orientation. Furthermore, in other examples, the camera orientation when capturing the second panoramic image at the connecting opening O2 can also be used as a reference orientation; this is not a limitation.

[0065] When the camera orientation X when capturing the second panoramic image 401 through the connecting opening O1 is used as a reference orientation, the first determining unit 920 can determine the angle between the camera orientation when capturing other second panoramic images and the reference orientation based on camera sensor data, thereby obtaining the camera orientation when capturing other second panoramic images. Specifically, the first determining unit 920 can determine the angle between the camera orientation when capturing the second panoramic image through the connecting opening O2 and the reference orientation X based on camera sensor data when capturing second panoramic images through connecting openings O1 and O2, thereby obtaining the camera orientation Y when capturing the second panoramic image through the connecting opening O2. Figure 4 (Not shown). For example, with the determined included angle α, the first determining unit 920 can obtain the camera orientation Y = X + α when capturing the second panoramic image at the connecting opening O2. Therefore, the orientations of connecting openings O1 and O2 can be ultimately determined based on the camera orientations X and Y through subsequent steps.

[0066] Subsequently, the first determining unit 920 detects the location of the connecting opening on each of the second panoramic images.

[0067] In one example, the first determining unit 920 can input each of the second panoramic images into a trained object detection model to obtain two bounding boxes corresponding to the connection openings output by the object detection model. The center position of each of these two bounding boxes in the horizontal direction can be used as the location of the detected connection opening. The object detector model can be a model trained based on a deep neural network object detection algorithm, such as the YOLOv3 object detection model; however, other types of object detection models can also be used, without limitation.

[0068] Figure 5 A schematic diagram of detecting the location of a connection opening according to an embodiment of the present invention is shown.

[0069] in, Figure 5 The left side shows the same as Figure 4 The image shown is the same as the second panoramic image 401 taken at the connecting opening O1, and Figure 5 The right side further shows a second panoramic image 402 taken at the connecting opening O2. (See image 402.) Figure 5As shown, the first determining unit 920 can obtain two bounding boxes 501 and 502 corresponding to the connection opening O1 outputted by the object detection model by inputting the second panoramic image 401 shot at the connection opening O1 into the object detection model, and can take the most central positions 503 and 504 of the bounding boxes 501 and 502 in the horizontal direction as the positions of the detected connection opening O1, respectively. Figure 5 As can be seen, it is approximately located at the positions of 90° and -90° in the panoramic image coordinate system.

[0070] In another example, in addition to using the object detection model, the first determining unit 920 can also detect the positions of the connection openings by calculating the variance of each column of pixels on each second panoramic image. Specifically, since the connection openings usually have less changes in texture, color, etc., the difference of the column of pixels corresponding to the connection openings should be the smallest compared to other columns of pixels. Accordingly, the first determining unit 920 can calculate the variance of each column of pixels on the second panoramic image respectively, and take the horizontal coordinate of the column of pixels with the smallest variance as one of the positions of the detected connection opening. And the first determining unit 920 can take another horizontal coordinate which is spaced from the horizontal coordinate by half of the horizontal width of the panoramic image as another position of the connection opening.

[0071] Similarly, the first determining unit 920 can repeat the above steps on the second panoramic image 402 shot at the connection opening O2, thereby obtaining the positions 505 and 506 of the detected connection opening O2. From Figure 5 As can be seen, it is approximately located at the positions of 0° and 180° in the panoramic image coordinate system.

[0072] Finally, the first determining unit 920 determines the orientations of the respective connection openings according to the positions of the respective connection openings on the corresponding second panoramic images and the camera orientation.

[0073] Among them, the first determining unit 920 can determine the orientation of each connection opening according to the difference between the position of each connection opening on the corresponding second panoramic image and the central image column position, and the camera orientation. The orientation of the connection opening can be represented by the direction of the straight line passing through the connection opening.

[0074] Continuing to refer to Figure 5In the second panoramic image 401, the difference between the detected positions 503 and 504 of the connection opening O1 and the central image column position (as mentioned above, the central image column position is the 0° position in the panoramic image coordinate system, corresponding to the camera orientation X when the second panoramic image 401 is taken) is approximately 90° and -90°, and the first determining unit 920 can determine the included angle between the orientation of the connection opening O1 and the camera orientation X to be 90°, i.e., the orientations of the two are approximately perpendicular to each other.

[0075] Similarly, in the second panoramic image 402, the difference between the detected positions 505 and 506 of the connection opening O2 and the central image column position (as mentioned above, the central image column position is the 0° position in the panoramic image coordinate system, corresponding to the camera orientation Y when the second panoramic image 402 is taken) is approximately 0° and 180°, and the first determining unit 920 can determine the included angle between the orientation of the connection opening O2 and the camera orientation Y to be 0°, i.e., the orientations of the two are approximately parallel to each other.

[0076] After obtaining the included angles between the orientations of the above-described connection openings and the corresponding camera orientations, the first determining unit 920 can further convert the orientations of the connection openings into absolute orientations corresponding to the same reference orientation by using the respective camera orientations, so as to finally determine the orientations of the connection openings. Specifically, in this example, since the reference orientation has been determined to be the camera orientation X, and the included angle between the orientation of the connection opening O1 and the camera orientation X is 90°, the finally determined orientation of the connection opening is X+90°. In addition, when the included angle between the camera orientation Y and the reference orientation is a (i.e., the camera orientation Y = X+a), and the included angle between the orientation of the connection opening O2 and the camera orientation Y is 0°, the finally determined orientation of the connection opening O2 is X+a. Through the above process, the first determining unit 920 can determine the orientations of the respective connection openings.

[0077] The second determining unit 930 obtains a correspondence between two first panoramic images in each group of panoramic images and a part of a second panoramic image in the group of panoramic images, and determines relative positions of two adjacent subspaces in each group of subspaces based on the correspondence.

[0078] The second determining unit 930 can divide the second panoramic image in the group of panoramic images into two sub-images based on the positions of the connection openings detected on the second panoramic image, and then compare the similarity between the two first panoramic images in the group of panoramic images and the two sub-images to obtain the correspondence.

[0079] Figure 6 An example of obtaining the correspondence between the two first panoramic images and the part of the second panoramic image is shown. As shown in FIG. 6, the second panoramic image 402 is divided into two sub-images 402a and 402b, and the two first panoramic images 401a and 401b are compared with the two sub-images 402a and 402b to obtain the correspondence. Figure 6As shown, the second determining unit 930 can divide the second panoramic image 401 in the set of panoramic images by the position of the connection opening O1 detected on the second panoramic image 401, and divide the second panoramic image 401 into two sub-images 601 and 602, where the sub-image 601 is an image divided from the middle part of the second panoramic image 401, and the sub-image 602 is an image divided from the left and right parts of the second panoramic image 402 and spliced, which correspond to two adjacent subspaces communicated by the connection opening O1 respectively.

[0080] In addition, Figure 6 It is also shown that the first panoramic image 603 and the first panoramic image 604 in the set of panoramic images are respectively taken in the two adjacent subspaces. By comparing the similarity of the first panoramic images 603 and 604 with the two sub-images 601 and 602, the second determining unit 930 can determine that the first panoramic image 603 is more similar to the sub-image 601, and the first panoramic image 604 is more similar to the sub-image 602. Thus, the second determining unit 930 can determine that the first panoramic image 603 corresponds to the sub-image 601, and the first panoramic image 604 corresponds to the sub-image 602. Wherein, the second determining unit 930 can input a first panoramic image and a sub-image into a trained comparison model, so as to obtain the similarity of the two images output by the model, so as to determine the correspondence between the first panoramic image and the sub-image based on the similarity. Wherein, the comparison model can be a model trained based on a twin neural network, however, the specific type of the model is not limited by the present application. Accordingly, based on the determined correspondence between the first panoramic image and the sub-image, the second determining unit 930 can determine that the first panoramic image 603 corresponds to the middle part of the second panoramic image 401, and the first panoramic image 604 corresponds to the two side parts of the second panoramic image 401, so as to obtain the relative positions of the subspaces corresponding to the two first panoramic images.

[0081] Figure 7 A schematic diagram of the determination result of the relative positions of two adjacent subspaces according to an embodiment of the present application is shown. Wherein, the relative positions define the positions of the two adjacent subspaces relative to the connection opening communicating them, that is, define which side or direction of the connection opening the two adjacent subspaces are located in. Since the first determining unit 920 has determined that the orientation of the connection opening O1 is X+90°, the two adjacent subspaces communicated by the connection opening O1 should be located in the X direction and the -X direction of the connection opening respectively. Therefore, after the second determining unit 930 determines that the first panoramic image 603 corresponds to the middle part of the second panoramic image 401 and the first panoramic image 604 corresponds to the two side parts of the second panoramic image 401, it can be determined that the subspaces corresponding to the first panoramic image 603 are located in the X direction of the connection opening O1, and the subspaces corresponding to the first panoramic image 604 are located in the -X direction of the connection opening O1.Figure 7 The relative orientation of the connection opening O1 is shown as R2', while the subspace corresponding to the first panoramic image 604 (in Figure 7 The relative orientation of the connection opening O1 (shown as R1') is the -X direction, which is equivalent to obtaining the following: Figure 7 The left side shows the relative orientation of two adjacent subspaces R1' and R2' in a set of subspaces.

[0082] Similarly, the second determining unit 930 can repeat the above-described step S103 processing on another set of panoramic images captured at the connecting opening O2 and in two adjacent subspaces connected by the connecting opening O2, thereby obtaining, as shown in the figure. Figure 7 The right side shows the relative orientations of two adjacent subspaces R3' and R4' in another set of subspaces.

[0083] The third determining unit 940 identifies the first panoramic image corresponding to the same subspace in each group of panoramic images, and determines the connection relationship of each group of subspaces based on the same subspace. Based on the determined connection relationship of each group of subspaces, the orientation of each connection opening and the relative orientation of two adjacent subspaces in each group of subspaces, the connection relationship of each subspace in the three-dimensional space is determined.

[0084] The third determining unit 940 can compare the similarity between a first panoramic image in one set of panoramic images and a first panoramic image in another set of panoramic images to identify the first panoramic image corresponding to the same subspace in each set of panoramic images. As described above, the third determining unit 940 can input the first panoramic image in one set of panoramic images and the first panoramic image in another set of panoramic images into a comparison model trained based on a Siamese neural network to obtain the similarity between the two output images, so as to identify the first panoramic image corresponding to the same subspace based on the similarity.

[0085] Continue to refer to Figure 7, the third determining unit 940 will determine the connection relationship between the two groups of subspaces, i.e., determine through which specific subspaces the two groups of subspaces should be connected to form the complete three-dimensional space. In this example, when the third determining unit 940 determines that the first panoramic image corresponding to the subspace R2' in one group of subspaces and the first panoramic image corresponding to the subspace R3' in another group of subspaces have a similarity degree higher than a threshold, it can be identified that the subspace R2' and the subspace R3' are actually the same subspace, and thus the third determining unit 940 can determine the connection relationship of the groups of subspaces based on the same subspace, i.e., it can be determined that the two groups of subspaces should be connected through the subspace R2' and the subspace R3' to form the complete three-dimensional space. Accordingly, the third determining unit 940 can connect the group of subspaces on the left and the group of subspaces on the right by overlapping the subspace R2' and the subspace R3' with each other. Figure 7 the group of subspaces on the left and Figure 7 the group of subspaces on the right.

[0086] After connecting the groups of subspaces based on the connection relationship of the groups of subspaces, the third determining unit 940 can determine the connection relationship of each subspace in the three-dimensional space based on the orientation of each connection opening determined in the foregoing step and the relative orientation of two adjacent subspaces in each group of subspaces.

[0087] In addition, after determining the connection relationship of each subspace, the image generating unit (not shown) can further generate a two-dimensional layout image of the three-dimensional space based on the determined connection relationship of each subspace.

[0088] Figure 8 A schematic diagram of generating a two-dimensional layout image based on the determined subspace connection relationship according to an embodiment of the present application is shown. As Figure 8 shown, the image generating unit can generate a two-dimensional layout image similar to the image shown in Figure 7 by overlapping the subspace R2' and the subspace R3' in Figure 8 to form the subspace R2" in Figure 2 based on the determined connection relationship of each subspace.

[0089] In summary, the above-described subspace connection relationship determining apparatus according to an embodiment of the present application can determine the connection relationship of each subspace in a three-dimensional space based on the panoramic images taken of each subspace in the three-dimensional space and the panoramic images taken of the connection openings connecting each group of adjacent subspaces, so as to automatically generate a two-dimensional layout image of the three-dimensional space based on the determined subspace connection relationship, thereby improving the generation efficiency of the two-dimensional layout image.

[0090] In the following, reference Figure 10This describes a subspace connection relationship determination apparatus according to an embodiment of the present invention. Figure 10 A block diagram of a subspace connectivity determination apparatus 1000 according to an embodiment of the present invention is shown. Figure 10 As shown, the device 1000 can be a computer or a server.

[0091] like Figure 10 As shown, the subspace connection relationship determination device 1000 includes one or more processors 1010 and a memory 1020. Of course, in addition to these, the subspace connection relationship determination device 1000 may also include input devices, output devices (not shown), etc., and these components can be interconnected via a bus system and / or other forms of connection mechanisms. It should be noted that... Figure 10 The components and structures of the subspace connection relationship determination device 1000 shown are merely exemplary and not limiting. The subspace connection relationship determination device 1000 may also have other components and structures as needed.

[0092] The processor 1010 may be a central processing unit (CPU) or other processing unit with data processing and / or instruction execution capabilities, and may utilize computer program instructions stored in memory 1020 to perform desired functions, including: acquiring panoramic images corresponding to each set of subspaces in three-dimensional space, wherein each pair of adjacent subspaces in three-dimensional space constitutes a set of subspaces, and each set of panoramic images includes two first panoramic images captured in the two adjacent subspaces of each set of subspaces and a second panoramic image captured at a connecting opening connecting the two adjacent subspaces; and according to each connecting opening, in the corresponding second panoramic image... The orientation of each connecting opening is determined by the position of the camera on the screen and the camera orientation when capturing each second panoramic image; the correspondence between two first panoramic images in each group of panoramic images and a portion of the second panoramic image in that group of panoramic images is obtained, and the relative orientation of two adjacent subspaces in each group of subspaces is determined based on the correspondence; and the first panoramic images corresponding to the same subspace in each group of panoramic images are identified, and the connection relationship of each group of subspaces is determined based on the same subspace, and the connection relationship of each subspace in the three-dimensional space is determined based on the determined connection relationship of each group of subspaces, the orientation of each connecting opening, and the relative orientation of two adjacent subspaces in each group of subspaces.

[0093] The memory 1020 can include one or more computer program products that can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. One or more computer program instructions can be stored on the computer-readable storage media, which can be run by the processor 1010 to implement the functions of the subspace connection relationship determination apparatus of the embodiments of the application described above and / or other desired functions, and / or to perform the visual mileage calculation method according to the embodiments of the application. Various application programs and various data can also be stored in the computer-readable storage media.

[0094] In the following, a computer-readable storage medium according to an embodiment of the application is described, which stores computer program instructions, wherein the computer program instructions are executed by a processor to implement the following steps: obtaining a plurality of groups of panoramic images respectively corresponding to a plurality of groups of subspaces in a three-dimensional space, wherein each two adjacent subspaces in the three-dimensional space form a group of subspaces, and each group of panoramic images includes two first panoramic images respectively taken within two adjacent subspaces in each group of subspaces and a second panoramic image taken at a connection opening connecting the two adjacent subspaces; determining an orientation of each connection opening according to a position of the connection opening on the corresponding second panoramic image and a camera orientation when each second panoramic image is taken; obtaining a correspondence between the two first panoramic images in each group of panoramic images and a part of the second panoramic image in the group of panoramic images, and determining relative orientations of the two adjacent subspaces in each group of subspaces based on the correspondence; and identifying first panoramic images in the groups of panoramic images corresponding to the same subspace, and determining a connection relationship of the groups of subspaces based on the same subspace, and determining a connection relationship of each subspace in the three-dimensional space based on the determined connection relationship of the groups of subspaces, the orientation of each connection opening and the relative orientations of the two adjacent subspaces in each group of subspaces.

[0095] Of course, the above-described specific embodiments are only exemplary and not limiting, and those skilled in the art can combine and combine some steps and apparatuses from the above separately described various embodiments according to the concept of the application to achieve the effect of the application, and such combined and combined embodiments are also included in the application, and such combination and combination is not described here.

[0096] Note that the advantages, advantages, effects, etc. mentioned in the present application are only examples and not limitations, and these advantages, advantages, effects, etc. cannot be considered as necessary for each embodiment of the present application. In addition, the above specific details of the application are only for the purpose of example and for the purpose of understanding, and not for the purpose of limitation, and the above details do not limit the application to the above specific details.

[0097] The block diagrams of the devices, apparatuses, equipment, systems described herein are merely illustrative examples and are not intended to require or imply constrictions, arrangements, configurations as shown in the block diagrams. As will be appreciated by one of ordinary skill in the art, the devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "including," "containing," "comprising," and the like are to be construed in an open-ended fashion, indicating the described elements or steps are among those included, but are not limited to only those elements or steps; use of terms such as "or," "and," and "comprising" to link the elements or steps are intended merely to be illustrative of the open-ended meaning and are not a limitation on the scope of the use of such terms to introduce innovation. The words "or" and "and" as used herein, in the context of using these words to join a list of elements or items, covers all of the following interpretations of even the alternative disjunctive and conjunctive lists of an alternative. That is, for example, a list of "a," "b," or "c" covers any of the possibilities: "a," "b," "c" alone; "a," "b," "c" in any combination; "a," "b," or "c"; "a" or "b"; "a" or "c"; "b" or "c"; "a" and "b"; "a" and "c"; "b" and "c"; or "a," "b," and "c." The words "comprising" (and any form of comprising, such as "comprise" and "comprises"), "having" (and any form of having, such as "have" and "has"), "including" (and any form of including, such as "includes" and "include") or "containing" (and any form of containing, such as "contains" and "contain") are inclusive / inclusive-sense rather than exclusive / exclusive-sense and do not exclude the presence of additional elements or steps.

[0098] The flowchart diagrams and above method descriptions are merely illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the order presented. As will be appreciated by one of ordinary skill in the art, the order of the steps in the above embodiments can be changed, as can other implementations of the disclosure. Words such as "thereafter," "then," "next," etc. are not intended to limit the order of the steps; these words are simply used to guide the reader through the description of the methods. Further, any reference to claim elements in the singular, for example, using the articles "one," "a" or "an," is not to be construed as limiting the element to the singular.

[0099] Further, the steps and apparatuses in the various embodiments herein are not limited to performing only within one embodiment, but in fact, parts of steps and parts of apparatuses in the various embodiments herein can be combined to conceive new embodiments according to the concepts of the present disclosure, and these new embodiments are also included in the scope of the present disclosure.

[0100] The various operations of methods described above can be performed by any suitable means capable of performing the corresponding functions. The means can include various hardware and / or software component(s) and / or module(s), including, but not limited to a circuit, an application specific integrated circuit (ASIC), or processor. The processor can be a general purpose processor, a digital signal processor (DSP), an ASIC, a field programmable gate array signal (FPGA), or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein.

[0101] The various illustrative logical blocks, modules, and circuits described in connection with the present disclosure can be implemented or performed with a general purpose processor, a digital signal processor (DSP), an ASIC, a field programmable gate array signal (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor can be a microprocessor, but in the alternative, the processor can be any commercially available processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0102] The steps of a method or algorithm described in connection with the present disclosure can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in any form of tangible storage medium. Some examples of storage media that can be used include random access memory (RAM), read only memory (ROM), a flash memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM and so forth. A storage medium can be coupled to a processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The software module can comprise single instruction or many instructions, and can be distributed over several different code segments, among different programs, and across multiple storage media.

[0103] The methods of the present invention include one or more acts for accomplishing the described methods. The methods and / or acts can be interchanged between methods, unless specifically stated to the contrary.

[0104] The functions described can be implemented in hardware, software, firmware or any combination thereof. If implemented in software, the functions can be stored as one or more instructions on a tangible computer-readable medium. A storage media can be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other tangible medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk herein, refers to compact discs, laser discs, optical discs, digital versatile discs (DVD), floppy disks and Blu-ray discs, as examples.

[0105] Thus, a computer program product can perform the operations presented herein. For example, such a computer program product can be a computer readable tangible medium having instructions stored thereon that, when executed by one or more processors, perform the operations described herein. The computer program product can also include packaging materials.

[0106] Software or instructions can also be transmitted over a transmission medium. For example, software can be transmitted from a website, server, or other remote source using a transmission medium such as a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave. Thus, a transmission medium can transmit information for execution by a machine.

[0107] Moreover, the modules and / or other appropriate means for performing the methods and techniques described herein can be downloaded and / or otherwise obtained by a user terminal and / or base station as applicable, such as from the server. For example, such a device can be coupled to the server to facilitate the transfer of means for performing the methods described herein. Alternatively, various methods described herein can be provided via a storage means (e.g., RAM, ROM, a physical storage medium such as a CD or floppy disk, etc.), such that a user terminal and / or base station can obtain the various methods upon coupling or providing the storage means to the device.

[0108] Other examples and implementations are within the scope and spirit of the disclosure and appended claims. For example, due to the nature of software, many modifications such as those relating to software

[0109] Various changes, modifications and alterations to the techniques described herein can be made without departing from the teachings of the disclosure as defined by the appended claims. Moreover, the scope of the claims should not be limited to the particular aspects described in the specification. Any of the various aspects described in the specification can be implemented in hardware, software or firmware, or a combination thereof. The specification is to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be within the scope of the present disclosure.

[0110] The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0111] The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the application to forms disclosed herein. Although several example aspects and embodiments have been discussed, those skilled in the art will recognize certain variations, modifications, changes, additions, and sub-combinations thereof.

Claims

1. A method for determining connection relationships of subspaces, comprising: obtaining a plurality of sets of panoramic images respectively corresponding to a plurality of sets of subspaces in a three-dimensional space, wherein each two adjacent subspaces in the three-dimensional space forms a set of subspaces, and each set of panoramic images comprises two first panoramic images respectively taken within two adjacent subspaces in each set of subspaces and a second panoramic image taken at a connection opening connecting the two adjacent subspaces; determining orientations of the connection openings according to positions of the connection openings on the corresponding second panoramic images and camera orientations when the corresponding second panoramic images are taken; obtaining a correspondence between the two first panoramic images in each set of panoramic images and a portion of the second panoramic image in the set of panoramic images, and determining relative orientations of the two adjacent subspaces in each set of subspaces based on the correspondence; and identifying the first panoramic images corresponding to the same subspace in the sets of panoramic images, and determining connection relationships of the sets of subspaces based on the same subspace, and determining connection relationships of the subspaces in the three-dimensional space based on the determined connection relationships of the sets of subspaces, the orientations of the connection openings and the relative orientations of the two adjacent subspaces in each set of subspaces; wherein the obtaining the correspondence between the two first panoramic images in each set of panoramic images and the portion of the second panoramic image in the set of panoramic images comprises: dividing the second panoramic image in each set of panoramic images into two sub-images based on the positions of the connection openings detected on the second panoramic image; and comparing the two first panoramic images in each set of panoramic images with the two sub-images to obtain the correspondence. 2.The method of claim 1, wherein the first panoramic images and the second panoramic images of each set of subspaces are generated based on a predetermined shooting order; and the sets of panoramic images are obtained by grouping the first panoramic images and the second panoramic images of the sets of subspaces based on the predetermined shooting order. The determining the orientations of the connection openings according to the positions of the connection openings on the corresponding second panoramic images and the camera orientations when the corresponding second panoramic images are taken comprises: determining the camera orientations when the corresponding second panoramic images are taken based on camera sensor data when the corresponding second panoramic images are taken; detecting the positions of the connection openings on the corresponding second panoramic images; and determining the orientations of the connection openings according to the detected positions of the connection openings on the corresponding second panoramic images and the camera orientations. The determining the orientations of the connection openings according to the detected positions of the connection openings on the corresponding second panoramic images and the camera orientations comprises: determining the orientations of the connection openings according to differences between the positions of the connection openings on the corresponding second panoramic images and a center image column position and the camera orientations. The identifying the first panoramic images corresponding to the same subspace in the sets of panoramic images comprises: comparing the first panoramic images in one set of panoramic images with the first panoramic images in another set of panoramic images to identify the first panoramic images corresponding to the same subspace in the sets of panoramic images. ​ ​ ​ ​ ​ ​ 3. The method of claim 1, wherein, ​ ​ ​ ​ 4. The method of claim 3, wherein, ​ 5. The method of claim 1, wherein, ​ ​ 6. The method of any one of claims 1-5, further comprising: generate a two-dimensional layout image of the three-dimensional space based on the determined connection relationship of each sub-space in the three-dimensional space. 7.A sub-space connection relationship determining apparatus, comprising: an acquisition unit configured to acquire a plurality of sets of panoramic images respectively corresponding to a plurality of sets of sub-spaces in a three-dimensional space, wherein each two adjacent sub-spaces in the three-dimensional space forms a set of sub-spaces, and each set of panoramic images comprises two first panoramic images respectively taken within the two adjacent sub-spaces in each set of sub-spaces and a second panoramic image taken at a connection opening connecting the two adjacent sub-spaces; a first determination unit configured to determine an orientation of each connection opening according to a position of each connection opening on a corresponding second panoramic image and a camera orientation when each second panoramic image is taken; a second determination unit configured to obtain a correspondence between the two first panoramic images in each set of panoramic images and a portion of the second panoramic image in the set of panoramic images, and determine relative orientations of the two adjacent sub-spaces in each set of sub-spaces based on the correspondence; and a third determination unit configured to identify the first panoramic images corresponding to a same sub-space in the plurality of sets of panoramic images, and determine a connection relationship of each set of sub-spaces based on the same sub-space, and determine a connection relationship of each sub-space in the three-dimensional space based on the determined connection relationship of each set of sub-spaces, the orientation of each connection opening and the relative orientations of the two adjacent sub-spaces in each set of sub-spaces; wherein the second determination unit divides the second panoramic image in the set of panoramic images into two sub-images based on the position of the connection opening detected on the second panoramic image; and compares the two first panoramic images in the set of panoramic images with the two sub-images in similarity to obtain the correspondence. 8.A sub-space connection relationship determining apparatus, comprising: a processor; and a memory having computer program instructions stored therein, wherein the computer program instructions, when executed by the processor, cause the processor to perform the following steps: acquire a plurality of sets of panoramic images respectively corresponding to a plurality of sets of sub-spaces in a three-dimensional space, wherein each two adjacent sub-spaces in the three-dimensional space forms a set of sub-spaces, and each set of panoramic images comprises two first panoramic images respectively taken within the two adjacent sub-spaces in each set of sub-spaces and a second panoramic image taken at a connection opening connecting the two adjacent sub-spaces; determine an orientation of each connection opening according to a position of each connection opening on a corresponding second panoramic image and a camera orientation when each second panoramic image is taken; obtain a correspondence between the two first panoramic images in each set of panoramic images and a portion of the second panoramic image in the set of panoramic images, and determine relative orientations of the two adjacent sub-spaces in each set of sub-spaces based on the correspondence; and identifying first panoramic images in each group of panoramic images corresponding to the same sub-space, and determining the connection relationship of each group of sub-spaces based on the same sub-space, determining the connection relationship of each sub-space in the three-dimensional space based on the determined connection relationship of each group of sub-spaces, the orientation of each connection opening, and the relative orientation of two adjacent sub-spaces in each group of sub-spaces; wherein the obtaining of the correspondence between the two first panoramic images in each group of panoramic images and a part of the second panoramic image in the group of panoramic images comprises: dividing the second panoramic image into two sub-images based on the position of the connection opening detected on the second panoramic image in the group of panoramic images; and comparing the similarity between the two first panoramic images in the group of panoramic images and the two sub-images to obtain the correspondence.

9. A computer readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by a processor, implement the following method: obtaining a plurality of groups of panoramic images respectively corresponding to a plurality of groups of sub-spaces in a three-dimensional space, wherein every two adjacent sub-spaces in the three-dimensional space is a group of sub-spaces, and each group of panoramic images includes two first panoramic images respectively taken in the two adjacent sub-spaces in each group of sub-spaces, and a second panoramic image taken at a connection opening connecting the two adjacent sub-spaces; determining the orientation of each connection opening according to the position of each connection opening on the corresponding second panoramic image and the camera orientation when taking each second panoramic image; obtaining the correspondence between the two first panoramic images in each group of panoramic images and a part of the second panoramic image in the group of panoramic images, and determining the relative orientation of the two adjacent sub-spaces in each group of sub-spaces based on the correspondence; and identifying first panoramic images in each group of panoramic images corresponding to the same sub-space, and determining the connection relationship of each group of sub-spaces based on the same sub-space, determining the connection relationship of each sub-space in the three-dimensional space based on the determined connection relationship of each group of sub-spaces, the orientation of each connection opening, and the relative orientation of two adjacent sub-spaces in each group of sub-spaces; wherein the obtaining of the correspondence between the two first panoramic images in each group of panoramic images and a part of the second panoramic image in the group of panoramic images comprises: dividing the second panoramic image into two sub-images based on the position of the connection opening detected on the second panoramic image in the group of panoramic images; and comparing the similarity between the two first panoramic images in the group of panoramic images and the two sub-images to obtain the correspondence. The computer program instructions, when executed by a processor, implement the following method:

Images