Spatial relationship calculation method, device and storage medium

Abstract

The embodiments of the present application provide a spatial relationship calculation method, device and storage medium. In the embodiments of the present application, when predicting the connection relationship between multiple single spaces, based on the panoramic images of multiple spaces corresponding to the multiple single spaces and the panoramic images of the multiple spatial connections connecting the multiple single spaces, the candidate connection relationship and connection weights of the multiple entrances corresponding to the multiple single spaces can be preliminarily estimated. Based on the candidate connection relationship and connection weights, an undirected weighted graph corresponding to the multiple entrances can be created; the connection relationship of the multiple entrances is calculated based on the undirected weighted graph, which effectively improves the robustness of the spatial relationship calculation method and the reliability of the calculation results.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a spatial relationship calculation method, device and storage medium. Background Art

[0002] The 3D panoramic navigation technology is developing day by day, providing more convenience for people's lives. For example, it can provide users with a remote house viewing service with a strong sense of immersion based on the 3D indoor navigation map, or provide users with a more intuitive and easy-to-understand station navigation service based on the 3D station navigation map.

[0003] Three-dimensional modeling of spatial scenes is an important part of realizing three-dimensional panoramic navigation. In the process of three-dimensional modeling of spatial scenes, if the spatial scene contains multiple single spaces, it is necessary to determine the association relationship between the multiple single spaces.

[0004] The spatial relationship calculation method provided by the prior art has poor robustness and reliability. Therefore, a new solution is needed. Summary of the invention

[0005] Multiple aspects of the present application provide a spatial relationship calculation method, device, and storage medium to improve the robustness and reliability of the spatial relationship calculation method.

[0006] An embodiment of the present application also provides a spatial relationship calculation method, including: acquiring multiple image data of a target space, wherein the target space includes multiple subspaces; constructing a graph model based on the multiple image data, wherein the graph model uses entrances corresponding to the multiple subspaces as nodes; identifying the connection relationship between the multiple subspaces based on the graph model; and displaying the connection relationship.

[0007] An embodiment of the present application also provides a spatial relationship calculation method, including: obtaining multiple spatial panoramic images corresponding to multiple single spaces and panoramic images of multiple spatial connections connecting the multiple single spaces; estimating candidate connection relationships and connection weights of multiple entrances corresponding to the multiple single spaces based on the multiple spatial panoramic images and the panoramic images of the multiple spatial connections; creating an undirected weighted graph corresponding to the multiple entrances based on the candidate connection relationships and connection weights; and calculating the connection relationships of the multiple entrances based on the undirected weighted graph to determine the connection relationships of the multiple single spaces based on the connection relationships of the multiple entrances.

[0008] An embodiment of the present application also provides a data processing method, including: acquiring multiple image data of a target space, wherein the target space includes multiple subspaces; reconstructing the target space based on the multiple image data; identifying the multiple image data to identify objects contained in each of the multiple subspaces; displaying a reconstructed image corresponding to the target space, and marking the objects contained in each of the multiple subspaces on the reconstructed image.

[0009] An embodiment of the present application also provides a data processing method, including: acquiring multiple image data of a target space, wherein the target space includes multiple subspaces; reconstructing the target space based on the multiple image data; displaying a reconstructed image corresponding to the target space; and responding to an adjustment operation on the reconstructed image, dynamically displaying the adjustment effect of the reconstructed image.

[0010] An embodiment of the present application also provides a data processing method, including: acquiring multiple image data of a target space, the multiple image data including multiple spatial panoramic images corresponding to multiple single spaces and panoramic images of multiple spatial connections connecting the multiple single spaces; determining a first spatial panoramic image and a panoramic image of a first spatial connection having an overlapping area from the multiple image data based on feature points contained in the multiple spatial panoramic images and the panoramic images of the multiple spatial connections; and correcting the first spatial panoramic image and the panoramic image of the first spatial connection based on pixel distribution characteristics of the overlapping area.

[0011] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to: execute the steps in the spatial relationship calculation method or the steps in the data processing method provided in the embodiment of the present application.

[0012] The embodiment of the present application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the spatial relationship calculation method or data processing method provided in the embodiment of the present application.

[0013] In the spatial relationship calculation method provided in the embodiment of the present application, when predicting the connection relationship between multiple single spaces, based on the multiple spatial panoramic images corresponding to the multiple single spaces and the panoramic images of the multiple spatial connections connecting the multiple single spaces, the candidate connection relationships and connection weights of the multiple entrances corresponding to the multiple single spaces can be preliminarily estimated; based on the candidate connection relationships and connection weights, an undirected weighted graph corresponding to the multiple entrances can be created; and the connection relationships of the multiple entrances are calculated based on the undirected weighted graph, which effectively improves the robustness of the spatial relationship calculation method and the reliability of the calculation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0015] Figure 1 A schematic diagram of a flow chart of a spatial relationship calculation method provided for an exemplary embodiment of the present application;

[0016] Figure 2 A schematic flow chart of a spatial relationship calculation method provided for another exemplary embodiment of the present application;

[0017] Figure 3a An exemplary embodiment of the present application provides a method for obtaining an initial maximum spanning tree according to an undirected weighted graph;

[0018] Figure 3b An exemplary embodiment of the present application provides an illustration of optimizing the edges in the initial maximum spanning tree;

[0019] Figure 4a A schematic flow chart of a spatial relationship calculation method provided for another exemplary embodiment of the present application;

[0020] Figure 4b A flowchart of a data processing method provided for an exemplary embodiment of the present application;

[0021] Figure 4c A flowchart of a data processing method provided for an exemplary embodiment of the present application;

[0022] Figure 4d A flowchart of a data processing method provided for an exemplary embodiment of the present application;

[0023] Figure 5 A schematic diagram for predicting the connection relationship between multiple rooms in a house provided by an exemplary embodiment of the present application;

[0024] Figure 6 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0026] In order to solve the technical problems of poor robustness and reliability of spatial relationship calculation methods, a solution is provided in some embodiments of the present application. The technical solutions provided in each embodiment of the present application are described in detail below in conjunction with the accompanying drawings.

[0027] Figure 1 A schematic diagram of a spatial relationship calculation method provided by an exemplary embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0028] Step 101: Acquire a plurality of spatial panoramic images corresponding to a plurality of single spaces and a panoramic image of a plurality of spatial connection points connecting the plurality of single spaces.

[0029] Step 102: estimate candidate connection relationships and connection weights of multiple entrances corresponding to the multiple single spaces based on the multiple spatial panoramic images and the panoramic images of the multiple spatial connections.

[0030] Step 103: Create an undirected weighted graph corresponding to the multiple entries according to the candidate connection relationship and the connection weight.

[0031] Step 104: Calculate the connection relationship of the multiple entrances according to the undirected weighted graph, so as to determine the connection relationship of the multiple single spaces according to the connection relationship of the multiple entrances.

[0032] When performing 3D modeling for indoor scenes, the complete indoor scene can be defined as a combination of multiple single spaces, where different single spaces are separated by physical walls or virtual walls, and adjacent single spaces are connected at the doorway (or virtual door frame). Among them, virtual walls refer to virtual obstacles defined to separate different single spaces.

[0033] For example, the indoor scene is a house, and the kitchen, living room, dining room, bathroom, and bedroom in the house can be defined as multiple single spaces in the house. The kitchen and the living room are separated by a physical wall, and the living room and the kitchen are connected at the kitchen door. The dining room and the living room are separated by a virtual wall, and are connected at the virtual dining room door. The living room and the bathroom are separated by a physical wall, and the living room and the bathroom are connected at the bathroom door. The bedroom and the living room are separated by a physical wall, and the bedroom and the living room are connected at the bedroom door.

[0034] In this embodiment, for the convenience of description, the doorways or virtual door frames connecting adjacent single spaces are described as the entrances of the single spaces, and each single space may include one or more entrances. For example, taking the above example of a house, the kitchen may include an entrance such as the kitchen door, the bathroom may include an entrance such as the bathroom door, and the living room may include multiple entrances such as the entrance door, the kitchen door, the virtual dining room door, the bathroom door, and the bedroom door.

[0035] In step 101, when acquiring image data for reconstructing an indoor scene, a panoramic device can be used to shoot a spatial panoramic image in each single space to obtain multiple spatial panoramic images, and a panoramic image can be shot at each spatial connection to obtain panoramic images of multiple spatial connections.

[0036] The panoramic image at the spatial connection includes some image features of the single space connected to it, for example, the panoramic image at the kitchen door includes some image features of the kitchen. Based on the overlapping features in the panoramic images of multiple spaces and the panoramic images at the spatial connection, the accuracy of the spatial connection relationship prediction can be further improved.

[0037] Among them, a panoramic image refers to a wide-angle image, whose horizontal viewing angle ranges from -180° to 180° and vertical viewing angle ranges from -90° to 90°. For any single space, according to the panoramic image corresponding to the single space, the cube corresponding to the single space can be reconstructed, that is, a cube formed by six faces: top, bottom, front, back, left, and right.

[0038] In some embodiments, the captured panoramic image presents a French equirectangular projection, and the aspect ratio of the panoramic photo is fixed at 2:1, which is not limited in this embodiment.

[0039] In step 102, after obtaining a plurality of spatial panoramic images and a plurality of panoramic images of spatial connections, candidate connection relationships and connection weights of a plurality of entrances corresponding to a plurality of single spaces may be estimated.

[0040] Among them, multiple entrances corresponding to multiple single spaces can be obtained from multiple spatial panoramic images. Candidate connection relationships refer to connection relationships obtained by preliminary spatial relationship prediction, and the candidate connection relationships need to be corrected through subsequent steps. Any two entrances are connected, and it can be considered that the two entrances correspond to a spatial connection in reality. For example, a door in the bedroom space panorama is connected to a door in the living room space panorama, then it can be considered that the door in the bedroom space panorama image and the door in the living room space panorama image correspond to the spatial connection between the bedroom and the living room in the actual space, that is, the bedroom door.

[0041] Among them, the connection weight can be used to characterize the reliability of the candidate connection relationship between the entrances, or the probability that a connection relationship exists between the entrances. The panoramic image at the spatial connection is an image obtained by panoramic photography at the entrance of a single space. Therefore, if two single spaces are connected, the feature points in the two single spaces may appear in the panoramic image at the spatial connection between the two single spaces. Therefore, after determining multiple entrances from multiple spatial panoramic images, the reliability of the connection relationship between the multiple entrances can be estimated based on the matching feature points in the multiple spatial panoramic images and the panoramic images of the multiple spatial connections. This will be explained in detail later and will not be repeated here.

[0042] In step 103, after the candidate connection relationships and connection weights of the multiple entries are determined, an undirected weighted graph corresponding to the multiple entries may be created according to the candidate connection relationships and connection weights.

[0043] An undirected weighted graph is a graph model used to represent the relationship between different objects. An undirected weighted graph consists of nodes (or vertices) and edges (straight lines or curves) connecting these nodes. The connecting edges in an undirected weighted graph do not distinguish directions, and each edge corresponds to a value, which is called the weight corresponding to the edge.

[0044] Based on this, when creating an undirected weighted graph, multiple entrances corresponding to multiple single spaces can be used as nodes, and the connecting edges between the multiple nodes can be determined based on the predicted candidate connection relationships between the multiple entrances. And the weights on the connecting edges between the multiple nodes can be determined based on the connection weights corresponding to the multiple entrances.

[0045] In step 104, after the undirected weighted graph is created, the connection relationship of multiple entrances is calculated.

[0046] Among them, multiple connected subgraphs contained in the undirected weighted graph can be obtained as spanning trees. The spanning tree contains all the nodes in the undirected weighted graph and contains the minimum edges that can maintain connectivity. The weight of the spanning tree is the sum of the weights of all the connecting edges contained in the spanning tree.

[0047] The weight of the connecting edge in an undirected weighted graph represents the probability that the nodes at both ends of the connecting edge are connected. The larger the weight of the connecting edge, the higher the probability that the two nodes are connected. Based on this, the larger the weight of the spanning tree, the more reliable the connection relationship between the nodes represented by the spanning tree.

[0048] Therefore, in this embodiment, a spanning tree whose weights meet the set conditions can be found in an undirected weighted graph to obtain a node connection relationship with high reliability. After obtaining the connection relationship of multiple entrances, the connection relationship of multiple single spaces can be determined according to the connection relationship of the multiple entrances. For example, if entrance 1 in space 1 has a connection relationship with entrance 2 in space 2, then space 1 and space 2 are considered to be connected, and no further description is given.

[0049] In this embodiment, when predicting the connection relationship between multiple single spaces, based on the multiple spatial panoramic images corresponding to the multiple single spaces and the panoramic images of the multiple spatial connections connecting the multiple single spaces, the candidate connection relationships and connection weights of the multiple entrances corresponding to the multiple single spaces can be preliminarily estimated; based on the candidate connection relationships and connection weights, an undirected weighted graph corresponding to the multiple entrances can be created; and the connection relationship of the multiple entrances is calculated based on the undirected weighted graph, which effectively improves the robustness of the spatial relationship calculation method and the reliability of the calculation results.

[0050] It should be noted that in the above and below embodiments of the present application, after obtaining multiple spatial panoramic images, multiple single spaces can be reconstructed. When reconstructing a single space, the geometric information in the spatial panoramic image can be preprocessed. Optionally, for each spatial panoramic image, the wall line data in the spatial panoramic image can be optimized. For example, the wall line data can be optimized by the Manhattan algorithm to obtain a standard single room with four or six walls. Optionally, on the basis of the optimized wall lines, the door line data in each spatial panoramic image is optimized so that the door line fits the wall surface.

[0051] After geometric information is preprocessed in the spatial panoramic image, the area where the spatial connection is located can be extracted from the preprocessed spatial panoramic image, that is, multiple entrances corresponding to multiple single spaces can be identified. Next, the connection relationship between the multiple entrances can be predicted to determine the connection relationship between the multiple single spaces.

[0052] In the above and following embodiments of the present application, the optional manner of estimating the candidate connection relationships at the multiple entrances according to the multiple spatial panoramic images and the panoramic images at the multiple spatial connections includes:

[0053] Implementation method 1: Based on the posture data of the image acquisition device, estimate the candidate connection relationships at multiple entrances.

[0054] Optionally, multiple entrances may be identified from multiple spatial panoramic images, that is, for any spatial panoramic image, all entrances included in the spatial panoramic image are identified, and the position of each entrance in the spatial panoramic image is determined.

[0055] Next, the posture data of the image acquisition device when capturing each panoramic image of the space is obtained to determine the orientation information of multiple entrances. The image acquisition device is equipped with an IMU (Inertial measurement unit), which can record the posture data of the image acquisition device when taking the panoramic image. The orientation of each entrance can be determined by combining the posture data of the image acquisition device and the direction of the wall line and door line in the panoramic image of the space.

[0056] The orientation of the entrance is an absolute direction, for example, east, west, south, north, etc. After obtaining the orientation information of each of the multiple entrances, it can be estimated that there is a candidate connection relationship between entrances with opposite orientations. For example, if entrance 1 in spatial panoramic image 1 faces east, and entrance 2 in spatial panoramic image 2 faces west, it can be estimated that entrance 1 and entrance 2 have a connection relationship.

[0057] Implementation method 2: based on the shooting time information, estimate the candidate connection relationships at multiple entrances.

[0058] Optionally, when capturing each spatial panoramic image and the panoramic image of the spatial connection, the image acquisition device can save the shooting time in the image in a certain format, for example, saving the shooting time in the captured panoramic image in Exif format. These time information records the time sequence of each acquisition point in the shooting process. Generally, if the shooting time of two spatial panoramic images is similar, it can be considered that the probability that the single spaces corresponding to the two spatial panoramic images are adjacent is relatively high.

[0059] Based on the above, the acquisition order of multiple spatial panoramic images and panoramic images of multiple spatial connections can be determined according to the image acquisition time information; then, according to the acquisition order, spatial panoramic images whose acquisition order difference is within the set order difference range are screened from the multiple spatial panoramic images, and the entrances contained in the spatial panoramic images whose acquisition order difference is within the set order difference range are estimated as entrances with candidate connection relationships.

[0060] When shooting, the shooting route can be set along the adjacent order of the single space, so that the image acquisition order obtained by shooting can reflect the adjacent characteristics between the single spaces to a certain extent. Among them, setting the sequence difference range can include: the acquisition sequence difference is 1 or 2.

[0061] In one scenario, the shooting order is: shooting in single space 1, shooting in single space 2, and shooting at the spatial connection between single space 1 and single space 2. Then, the difference in the acquisition order between the spatial panoramic image corresponding to single space 1 and the spatial panoramic image corresponding to single space 2 is 1. If entrance 1 is on the spatial panoramic image corresponding to single space 1, and entrance 2 is on the spatial panoramic image corresponding to single space 2, it can be estimated that entrance 1 and entrance 2 are connected.

[0062] In another scenario, the shooting order is: shooting in single space 1, shooting at the spatial connection of single space 1 and single space 2, and shooting in single space 2. Then, the difference in the acquisition order of the spatial panoramic image corresponding to single space 1 and the spatial panoramic image corresponding to single space 2 is 2. If entrance 1 is on the spatial panoramic image corresponding to single space 1, and entrance 2 is on the spatial panoramic image corresponding to single space 2, it can be estimated that entrance 1 and entrance 2 are connected.

[0063] In the above and the following embodiments of the present application, after estimating the candidate connection relationship between multiple entries, the connection edges between multiple nodes corresponding to the multiple entries can be determined in the weighted undirected graph according to the candidate connection relationship. For ease of description and distinction, the connection edge determined according to the candidate connection relationship can be described as a first type of connection edge. The first type of connection edge is an estimated normal connection edge (Normal Edge).

[0064] Further optionally, a spatial panoramic image containing multiple entrances can be determined from multiple spatial panoramic images. For ease of description and distinction, the spatial panoramic image containing multiple entrances is described as a third spatial panoramic image. Then, in the undirected weighted graph, a second type of connection edge is added between the nodes corresponding to the multiple entrances contained in the third space. That is, if a spatial panoramic image contains multiple different entrances, a connection edge can be added between the nodes corresponding to the multiple different entrances. This second type of connection edge can be called a constant edge.

[0065] In addition to the above two types of connection edges, the undirected weighted graph also contains a third type of connection edge. In the process of constructing a spanning tree based on an undirected weighted graph, there may be some nodes that are not connected to existing nodes in the spanning tree. For the convenience of description, the nodes that are not connected to the existing nodes in the spanning tree are described as candidate nodes. At this time, a third type of connection edge can be added between the candidate node and any existing node in the spanning tree. The third type of connection edge is a fake edge. With the help of fake edges, it can be ensured that the spanning tree contains all the nodes in the undirected weighted graph, which is conducive to finding reasonable node connection relationships through iterative algorithms in the subsequent process.

[0066] On the basis of the above embodiment, after determining the connection edges between the nodes in the undirected weighted graph, the weights on the connection edges can be determined. Optionally, for the second type of connection edges, a fixed weight can be set for them. The size of the fixed weight can be set according to actual needs, and this embodiment does not limit it. Optionally, the fixed weight can be a larger value to ensure that when the initial maximum spanning tree is created from the undirected weighted graph later, the second type of connection edges can be added to the initial maximum spanning tree.

[0067] Optionally, the weights on the second type of connection edge and the third type of connection edge may be determined according to the connection weights between the entries corresponding to the nodes at both ends of the connection edge, which will be further described below.

[0068] In some exemplary embodiments, since each entrance has its own spatial panoramic image, the weight between every two entrances can be determined by the weight between the spatial images where the two entrances are located.

[0069] An optional implementation method of estimating connection weights of multiple entrances based on multiple spatial panoramic images and panoramic images of multiple spatial joints can be implemented based on feature detection and feature matching operations.

[0070] Optionally, after acquiring multiple spatial panoramic images and panoramic images of multiple spatial connections, feature detection can be performed on each spatial panoramic image and the panoramic image at each spatial connection, and the number of matching feature points between each spatial panoramic image and the panoramic image at each spatial connection can be calculated.

[0071] Feature detection refers to using a computer to extract image information and determine whether each image point belongs to an image feature. The result of feature detection is to divide the points on the image into different subsets, which often belong to isolated points, continuous curves or continuous regions. The task of feature matching is to find pairs of feature points with correct correspondence in different images.

[0072] Optionally, the feature detection operation can be implemented based on an ASIFT (affine-Affine-Scale-invariant feature transform) algorithm. The feature matching operation can be implemented based on a RANSAC (Random Sample Consensus) algorithm. Optionally, in order to improve computational efficiency, the panoramic image can be reduced, for example, to a size of 300*600.

[0073] The following section will take two entries with candidate connection relationships as an example to exemplify an optional implementation method of calculating the connection weight between the two entries. For ease of description and distinction, the two entries with candidate connection relationships are labeled as: a first entry and a second entry.

[0074] For the first entrance and the second entrance, a first space panoramic image where the first entrance is located and a second space panoramic image where the second entrance is located can be obtained respectively; then, based on the image features of the first space panoramic image and the second space panoramic image, the connection weight between the first entrance and the second entrance is estimated.

[0075] Optionally, feature points that match between the first spatial panoramic image and the panoramic images at the multiple spatial connections can be obtained respectively to obtain multiple feature point matching numbers, which are marked as multiple first feature point matching numbers. Also, feature points that match between the second spatial panoramic image and the panoramic images at the multiple spatial connections can be obtained respectively to obtain multiple feature point matching numbers, which are marked as multiple second feature point matching numbers.

[0076] Next, the minimum value among the matching numbers of the multiple first feature points is selected as the first minimum value; and the minimum value among the matching numbers of the multiple second feature points is selected as the second minimum value, and the maximum value between the first minimum value and the second minimum value is selected as the connection weight between the first entrance and the second entrance.

[0077] Optionally, the following will take the first spatial connection among the multiple spatial connections as an example to further illustrate the process of obtaining feature points that match between the first spatial panoramic image and the panoramic images of the multiple spatial connections. The first spatial connection is any spatial connection among the multiple spatial connections.

[0078] Taking the first spatial connection as an example, the feature points that match between the first spatial panoramic image and the panoramic image at the first spatial connection can be obtained, and the number of matched feature points is obtained, which is described as the third feature point matching number for easy distinction. If the third feature point matching number is greater than the set feature point number threshold, the third feature point matching number is used as the first feature point matching number. Among them, the feature point number threshold can be set according to actual needs, and this embodiment does not limit it.

[0079] On the contrary, if the third feature point matching number is less than or equal to the feature point number threshold, the first feature point matching number is set to a set value, where the set value may be a minimum value or may be 0, which is not limited in this embodiment.

[0080] It should be understood that the above-mentioned “first”, “second” and “third” are only used for convenience of description and distinction, and do not constitute a limitation on the order or size of the objects they define.

[0081] The following will combine the i-th spatial panoramic image R i and the jth spatial panoramic image R j , the calculation process of the connection weight is further explained, where i and j are positive integers used to represent the image number, i≠j. k represents the kth panoramic image at the spatial connection, k = [1, n], and n represents the number of panoramic images at the spatial connection.

[0082] For the spatial panoramic image R i , can traverse all the panoramic images D1-D at the spatial connection n, and perform feature matching operations on each panoramic image at the spatial connection point traversed. Let S() represent the feature matching operation, and the feature matching operation result S(R i ,D1)~S(R i ,D n ).

[0083] At the same time, for the spatial panoramic image R j , can traverse all the panoramic images D1-D at the spatial connection n , and perform feature matching operations on each panoramic image at the spatial connection point traversed to obtain the feature matching operation result S(R j ,D1)~S(R j ,D n ).

[0084] The weight C(R i ,R j ), can be expressed by the following formula:

[0085] C(R i ,R j )=max{min[S(R i ,D1),S(R j ,D1)]...min[S(R i ,D k ),S(R j ,D k )]…min[S(R i ,D n ),S(R j ,D n )]}

[0086] Among them, min() represents the function of finding the minimum value, and max() represents the function of finding the maximum value.

[0087] For R i , after traversing all the panoramic images D1-D at the spatial connections n In the process of feature matching, the number of feature points matched by each feature matching operation is determined. k For example, mark S(R i ,D k )=N ik , if N ik If it is greater than the set feature point number threshold N, it can be considered that D k The entrance represented by is most likely the spatial image R i At this time, we can mark S(R i ,D k )=Nik , otherwise mark S(R i ,D k )=0, that is:

[0088]

[0089] Based on the above method, the weight between any two spatial panoramic images can be calculated to obtain the connection weight between the entrances contained in any two spatial panoramic images. Based on the connection weight between the entrances contained in any two spatial panoramic images, the weights on the first type of connection edge and the second type of connection edge in the undirected weighted graph can be obtained, which will not be repeated.

[0090] Based on the above embodiments, the nodes in the undirected weighted graph, the connecting edges between the nodes, and the weights on the connecting edges are determined.

[0091] In some optional implementations, the undirected weighted graph may be further subjected to a "pruning" operation to remove unnecessary connection edges in the undirected weighted graph. Based on the "pruning" operation, the connection edges in the undirected weighted graph may be reduced to reduce the complexity of the undirected weighted graph and improve subsequent processing efficiency.

[0092] Optionally, based on multiple spatial panoramic images and panoramic images of multiple spatial joints, multiple groups of entrances without connection relationships can be screened from multiple entrances, each group of entrances containing two entrances. The following will be illustrated by taking a group of entrances as an example. For ease of description and distinction, in the following embodiments, the two entrances contained in any group of entrances without connection relationships are described as the third entrance and the fourth entrance. After the third entrance and the fourth entrance are screened out, the connection edge between the nodes corresponding to the third entrance and the fourth entrance can be deleted in the undirected weighted graph to implement the pruning operation of the graph model.

[0093] The following will further explain in detail how to perform the "pruning" operation on an undirected weighted graph in combination with different implementation methods:

[0094] In implementation mode A, the third entrance and the fourth entrance that do not have a connection relationship may be screened based on the spatial visibility feature.

[0095] Optionally, the visible range of the third entrance may be obtained based on the panoramic image of the space where the third entrance is located. For example, a single space may be reconstructed based on the panoramic image of the space where the third entrance is located, and the visible range of the third entrance may be obtained from the reconstructed single space.

[0096] Next, the feature points that match between the panoramic image of the space where the third entrance is located and the panoramic image of the second space connection can be obtained; wherein the correlation between the panoramic image of the second space connection and the panoramic image of the space where the fourth entrance is located is greater than the set correlation threshold; that is, the panoramic image of the second space connection is obtained by panoramic shooting at the fourth entrance with a high probability. If the matching feature point is not within the visible range of the third entrance, it is determined that there is no candidate connection relationship between the third entrance and the fourth entrance.

[0097] For example, the spatial panoramic image R1 includes the entrance d1, and the spatial panoramic image R2 includes the entrance d2. After the panoramic image D1 at the spatial connection is matched with the spatial panoramic image R1 for feature points, a matching feature point M is obtained, and the position of the feature point M in the spatial panoramic image R1 can be determined. If most of the feature points in the feature point M are within the visible range of the entrance d1, it can be considered that the correlation between d1 and D1 is low. In other words, the probability that D1 was not taken at d1 is high. If the correlation between D2 and d2 is high, the probability that D2 was taken at d2 is high. Based on the above analysis, it can be considered that there is a high probability that there is no connection relationship between d1 and d2.

[0098] In implementation mode B, the third entry and the fourth entry that do not have a connection relationship may be screened based on the semantic relationship of the image names.

[0099] The image name is a name obtained by naming the collected panoramic image. For example, the collected panoramic image can be named as the name of the room and the name of the space connection. For example, the master bedroom door, master bedroom, living room, entrance door, etc.

[0100] If the name of the panoramic image at the connection of the third space includes the name of the panoramic image where the third entrance is located, and the panoramic image where the third entrance is located only includes the third entrance, then it is determined that the panoramic image at the connection of the third space is associated with the third entrance; if the degree of association between the panoramic image where the fourth entrance is located and the panoramic image at the connection of the third space is less than or equal to the set association degree threshold, then it is determined that there is no candidate connection relationship between the third entrance and the fourth entrance. This will be explained in conjunction with specific examples below.

[0101] If the name of the panoramic image D1 at the spatial connection includes the name of the spatial panoramic image R1, and the panoramic image R1 only includes one entrance d1, then D1 can be considered to be photographed at d1. For example, if one image is named "Master Bedroom Door" and another image is named "Master Bedroom", and the image named "Master Bedroom" only includes one door, then the image named "Master Bedroom Door" can be considered to be photographed at the door in the image named "Master Bedroom".

[0102] On this basis, if the spatial panoramic image R2 has a low correlation with the panoramic image D1 at the spatial connection, it can be considered that d1 is not captured at the entrance d2 included in the spatial panoramic image R2. Further, it can be considered that d1 and d2 have no connection relationship.

[0103] In implementation C, the third entrance and the fourth entrance that have no connection relationship can be screened out based on spatial rationality.

[0104] If the spatial panoramic image where the third entrance is located only includes the third entrance, and the spatial panoramic image where the fourth entrance is located only includes the fourth entrance, it is determined that there is no candidate connection relationship between the third entrance and the fourth entrance. In other words, if each of two or more single spaces has only one entrance, it can be determined that there is no connection relationship between the two single spaces.

[0105] In implementation mode D, the third entrance and the fourth entrance that do not have a connection relationship can be screened out according to the geometric relationship of the entrances.

[0106] If the geometric shapes of the third entrance and the fourth entrance do not match, it is determined that there is no candidate connection relationship between the third entrance and the fourth entrance. For example, when the entrance is implemented as a door in a single space, the geometric shapes of the two doors do not match, including: the height difference of the doors is greater than a set height difference threshold and / or the width difference of the doors is greater than a set width difference threshold.

[0107] It should be noted that, since the gate line is optimized and the marking has errors, the height difference threshold and the width difference threshold can be set to a larger range, which will not be described in detail.

[0108] In addition to the various implementations described above, the connection relationship between different entrances can be further determined by calculating the floor color of the panoramic image of the space where different entrances are located, which will not be described in detail.

[0109] Based on the above “pruning” operation, the weights of the connected edges that have not been “pruned” can be further updated. An exemplary description will be given below.

[0110] In some optional embodiments, for any two spatial panoramic images, the connection weights corresponding to the entrances included in the two spatial panoramic images may be further updated according to the acquisition order between the two spatial panoramic images. The following will continue to be described using the first spatial panoramic image and the second spatial panoramic image as examples.

[0111] If the acquisition order difference between the first spatial panoramic image and the second spatial panoramic image is within the set order difference range, an additional weight may be added to the estimated connection weight of the first entrance and the second entrance. The set order difference range may refer to the description of the aforementioned embodiment and may be 1 or 2. The additional weight may be set according to actual needs and is not limited in this embodiment.

[0112] Take the spatial panoramic image R i and the jth spatial panoramic image R j For example, i and j represent the acquisition order. If |ij|<=2, and after other branch reduction operations in the algorithm, the spatial panoramic image R i Entrance d i and the jth spatial panoramic image R j Entrance d j If there is a connection relationship between and, update d i and d j The weight between the corresponding gate nodes is: W`(d i ,d j )=W(d i ,d j )+W time , where W time is a fixed weight value, that is, an additional weight value. W(d i ,d j ) can be calculated based on the feature matching operation provided in the aforementioned embodiment, which will not be described here.

[0113] Figure 2 is a flowchart of a spatial relationship calculation method provided by another exemplary embodiment of the present application. Figure 2 As shown, the method includes:

[0114] Step 201: Acquire a plurality of spatial panoramic images corresponding to a plurality of single spaces and a panoramic image of a plurality of spatial connection points connecting the plurality of single spaces.

[0115] Step 202: Identify multiple entrances corresponding to multiple single spaces from multiple spatial panoramic images, and create an undirected weighted graph using the multiple entrances as multiple nodes.

[0116] Step 203: Estimate candidate connection relationships and connection weights of multiple entrances based on the multiple spatial panoramic images and the panoramic images of the multiple spatial connections.

[0117] Step 204: Determine a first type of connection edge between a portion of nodes according to the candidate connection relationship, and determine to add a second type of connection edge between another portion of nodes located on the same spatial panoramic image.

[0118] Step 205: Determine the weight on the first type of connection edge according to the connection weight, and set a fixed weight for the second type of connection edge.

[0119] Step 206: Prune the undirected weighted graph to update the undirected weighted graph.

[0120] Step 207: Obtain multiple spanning trees that meet the set weight conditions from the undirected weighted graph.

[0121] Step 208: Estimate the architectural features of the multiple single spaces according to the node connection relationships represented by the multiple spanning trees.

[0122] Step 209: Select a target spanning tree from the multiple spanning trees according to the architectural features of the multiple single spaces.

[0123] Step 210: Determine the connection relationship of the multiple entries according to the node connection relationship represented by the target spanning tree.

[0124] Among them, the optional implementation methods of steps 201 to 206 can refer to the records of the aforementioned embodiments and are not repeated here.

[0125] In step 207, multiple spanning trees satisfying the set weight condition may be determined from the undirected weighted graph.

[0126] Optionally, an initial maximum spanning tree may be created according to the weights on the connecting edges in the undirected weighted graph.

[0127] Among them, the creation of the initial maximum spanning tree can be implemented by using the Kruskal algorithm. Based on the Kruskal algorithm, a new graph model can be created, which contains all the nodes in the undirected weighted graph, but has no connecting edges. Then, according to the order of the weights on all the connecting edges in the undirected weighted graph from large to small, the connecting edges are selected in sequence starting from the edge with the largest weight, and the connecting edges are added between the corresponding nodes in the newly created graph model. Figure 3a The process of creating an initial maximum spanning tree from an undirected weighted graph is illustrated.

[0128] Since the previous step performs a "pruning" operation on the undirected weighted graph model, in the process of creating the initial maximum spanning tree, some candidate nodes will not be connected to other nodes in the initial maximum spanning tree. At this time, a pseudo edge, i.e., the third type of connection edge, can be added between the candidate node and any other node. Alternatively, if multiple disconnected subtrees are generated during the creation of the initial maximum spanning tree, the multiple subtrees can be merged through pseudo edges to obtain a connected initial maximum spanning tree.

[0129] Next, based on the initial maximum spanning tree, a new spanning tree without the third type of connecting edges can be found.

[0130] Optionally, the first type of connection edge and / or the second type of connection edge in the undirected weighted graph that is not included in the initial maximum spanning tree can be used to replace the third type of connection edge in the initial maximum spanning tree. That is, if the undirected weighted graph contains normal edges and / or fixed edges that do not appear in the initial maximum spanning tree, the normal edges and / or fixed edges can be used to replace the pseudo edges in the initial maximum spanning tree while ensuring that the initial maximum spanning tree still has connectivity.

[0131] like Figure 3b As shown, assuming that the connection edge f in the initial maximum spanning tree is a pseudo edge, the connection edge f can be deleted. At this time, node 7 will become an isolated node. To ensure the connectivity of the spanning tree, the connection edge e that does not appear in the initial maximum spanning tree can be determined from the undirected weighted graph. If the connection edge e is not a pseudo edge, the connection edge e can be added to the initial spanning tree to complete the connection edge replacement operation.

[0132] Optionally, after performing the above-mentioned connection edge replacement operation, it can be checked whether the updated initial maximum spanning tree contains a loop. If a loop is contained, an appropriate edge is selected from the loop for deletion. The appropriate edge does not include a second type of connection edge, and does not include a first type of connection edge whose weight is greater than a set weight threshold. In other words, the edges that can be deleted in the loop may include pseudo edges and first type connection edges with smaller weights.

[0133] The above-mentioned operation of replacing pseudo edges and processing connecting edges in the loop can be repeated multiple times until the initial maximum spanning tree is updated to a new spanning tree without pseudo edges. Then, the edges of the new spanning tree can be transformed to obtain multiple new spanning trees without pseudo edges. Optionally, if the undirected weighted graph contains connecting edges of the first type or the second type that do not appear in the new spanning tree, any edge of the new spanning tree can be replaced with the connecting edges of the first type or the second type, while ensuring that the new spanning tree still has connectivity, so as to obtain other spanning trees different from the new spanning tree.

[0134] After obtaining multiple spanning trees, a spanning tree with higher rationality may be selected from the multiple spanning trees. Optionally, the process of selecting a spanning tree with higher rationality may be further implemented in combination with the information of a single space. The following will be described in combination with step 208 and step 209.

[0135] In step 208, the architectural features of the multiple single spaces may be estimated according to the node connection relationships represented by the multiple spanning trees, and a target spanning tree may be selected from the multiple spanning trees according to the reconstructed architectural features of the multiple single spaces.

[0136] The multiple single spaces refer to single spaces reconstructed according to the spatial panorama. The following will take the first spanning tree among the multiple spanning trees as an example for exemplary description.

[0137] Optionally, for the first spanning tree, the distribution positions of multiple single spaces can be determined based on the depth-first traversal algorithm and the connection relationship between the multiple entrances represented by the first spanning tree. Then, according to the distribution positions of the multiple single spaces, the overlapping areas of the multiple single spaces are calculated as the overlapping areas corresponding to the first spanning tree. For example, the relative transformation matrix between the multiple single spaces can be calculated based on the connection relationship between the entrances, and the absolute positions between the multiple single spaces can be calculated. Then, a mask can be drawn in the area where the multiple single spaces are located to obtain a matrix of a set size as the image to be calculated, and the overlapping area between the single spaces is calculated from the image to be calculated. Optionally, the set size can be 400x400.

[0138] It should be understood that the smaller the overlapping area between the single spaces, the higher the rationality of the architectural design of the area formed by the combination of the single spaces. Based on this, in some optional implementations, a spanning tree with an overlapping area less than a set overlapping area threshold can be selected from the multiple spanning trees according to the overlapping areas corresponding to the multiple spanning trees, as the target spanning tree.

[0139] Further optionally, considering the space utilization problem, the bounding boxes of the areas where the multiple single spaces are located can be drawn according to the distribution positions of the multiple single spaces, and the areas of the bounding boxes can be calculated as the bounding box areas corresponding to the first spanning tree. Next, the reasonable spatial distribution index corresponding to the first spanning tree can be calculated according to the overlapping area and the bounding box area corresponding to the first spanning tree. Optionally, the weighted sum of the overlapping area and the bounding box area corresponding to the first spanning tree can be calculated to obtain the reasonable spatial distribution index corresponding to the first spanning tree. This embodiment includes but is not limited to this.

[0140] Based on the above method, the spatial distribution rationality index corresponding to each spanning tree can be calculated. Then, according to the spatial distribution rationality indexes corresponding to the multiple spanning trees, a spanning tree with a spatial distribution rationality index greater than a set index threshold can be selected from the multiple spanning trees as the target spanning tree. Alternatively, a spanning tree with the highest ranking of the spatial distribution rationality index or several spanning trees with the highest ranking can be selected as the target spanning tree.

[0141] In step 210, after the target spanning tree is determined, the connection relationship of the multiple entrances can be determined according to the node connection relationship represented by the target spanning tree. After the connection relationship of the multiple entrances is determined, the multiple three-dimensional single spaces reconstructed according to the multiple spatial panoramas can be combined to obtain a reconstruction result of the complete scene.

[0142] In this embodiment, by using the panoramic image information taken at sparse points, the basic IMU orientation information, and combining the geometric and semantic information in the scene, a weight graph of the association relationship between each single space is established, and finally a unified spanning tree algorithm is used to infer the optimal spatial connection relationship in the graph. Its advantages are that it reduces the cost of image acquisition equipment, reduces the requirements for collecting image data, and improves the efficiency of the panoramic image acquisition process; at the same time, in addition to image information, IMU information, geometric shape constraints and semantic information are further introduced to predict the connection relationship, which is more robust.

[0143] Figure 4a A flowchart of a spatial relationship calculation method provided by another exemplary embodiment of the present application is shown in FIG. Figure 4a As shown, the method includes:

[0144] Step 401a: Acquire multiple image data of a target space, wherein the target space includes multiple subspaces.

[0145] Step 402a: construct a graph model based on the multiple image data, wherein the graph model uses the entrances corresponding to the multiple subspaces as nodes.

[0146] Step 403a: Identify the connection relationship between the multiple subspaces according to the graph model.

[0147] Step 404a: display the connection relationship.

[0148] The target space can be any indoor space, such as a residential house, an office space, an exhibition hall, a shopping mall, etc. The target space includes multiple subspaces (such as the single space described in the above embodiment). For example, a residential house includes multiple subspaces such as a kitchen, a living room, a dining room, a bathroom, and a bedroom. An office space includes multiple subspaces such as a conference room, an office area, a printing room, and a tea room.

[0149] The multiple image data of the target space may include multiple panoramic images captured in the target space, for example, a panoramic image of a subspace captured in each subspace and a panoramic image captured at a spatial connection between adjacent subspaces.

[0150] Based on multiple image data of the target space image, a graph model can be constructed. Optionally, the graph model can be implemented as an undirected weighted graph. The undirected weighted graph includes multiple nodes, and the multiple nodes are connected by connecting edges, and each connecting edge corresponds to a weight. In the graph model constructed in this embodiment, the entrances corresponding to the multiple subspaces can be regarded as nodes. Furthermore, by optimizing the connection relationship between the nodes in the graph model, the connection relationship between the multiple entrances can be predicted, and based on the predicted connection relationship between the multiple entrances, the connection relationship between the multiple subspaces can be identified.

[0151] Among them, the optional implementation method of constructing a graph model based on multiple image data can refer to the records of the aforementioned embodiments and will not be repeated here.

[0152] In this embodiment, when predicting the connection relationship between multiple subspaces contained in the target space, the calculation method of the spatial relationship can be converted into an optimization method of the graphical model based on multiple image data corresponding to the target space, thereby reducing the cost required for the spatial relationship calculation and improving the robustness of the spatial relationship calculation method.

[0153] Figure 4b A flowchart of a data processing method provided by an exemplary embodiment of the present application is shown in FIG. Figure 4b As shown, the method includes:

[0154] Step 401b: Acquire multiple image data of a target space, wherein the target space includes multiple subspaces.

[0155] Step 402b: reconstruct the target space based on the multiple image data.

[0156] Step 403b: Identify the multiple image data to identify the objects contained in each of the multiple subspaces.

[0157] Step 404b: display the reconstructed image corresponding to the target space, and mark the objects contained in each of the multiple subspaces on the reconstructed image.

[0158] Among them, the optional implementation method of reconstructing the target space according to multiple image data can refer to the records of the above embodiments, which will not be repeated here. On the basis of reconstructing the target space, based on the data processing method provided in this embodiment, the reconstruction result can be further annotated. The following will be explained in combination with step 403b and step 404b.

[0159] In step 403b, the method for identifying the multiple image data can be implemented by a method for image subject recognition based on a neural network, which is not limited in this embodiment. Based on the image recognition step, the objects contained in each of the multiple image data can be identified, and the objects contained in the multiple subspaces can be determined based on the correspondence between the image data and the subspace.

[0160] The objects contained in the subspace refer to the objects, furniture, equipment, decorations, etc. contained in the subspace. For example, a panoramic image is recognized to obtain objects such as "sofa", "tea table", "TV cabinet", and "floor lamp". If the panoramic image corresponds to the subspace corresponding to the living room, it can be determined that the living room contains furniture such as "sofa", "tea table", "TV cabinet", and "floor lamp".

[0161] In step 404b, the reconstructed image corresponding to the target space is displayed. Optionally, the reconstructed image may be two-dimensional or three-dimensional, which is not limited in this embodiment. When displaying the reconstructed image, the objects contained in each subspace may be marked on the reconstructed image according to the objects contained in each subspace, so as to facilitate user viewing.

[0162] In some optional embodiments, the shooting points corresponding to the plurality of image data may be identified according to the image contents contained in the plurality of image data, wherein the shooting points include shooting points for shooting the plurality of subspaces and shooting points at spatial connections corresponding to the plurality of subspaces.

[0163] After obtaining the shooting points, the shooting points can be marked in the reconstructed image. For example, in a reconstructed image of a house, the shooting points in the living room, the shooting points in the bedroom, the shooting points of the bedroom door, the shooting points of the entrance door, etc. can be marked. Optionally, the shooting point marking method can be a text marking method, a highlight marking method, or a star marking method, etc., which is not limited in this embodiment.

[0164] In this embodiment, after the target space is reconstructed, information is annotated on the reconstructed image, which can more intuitively display the reconstruction effect and facilitate user viewing.

[0165] Figure 4c A flowchart of a data processing method provided for another exemplary embodiment of the present application is shown in FIG. Figure 4c As shown, the method includes:

[0166] Step 401c: Acquire multiple image data of a target space, wherein the target space includes multiple subspaces.

[0167] Step 402c: reconstruct the target space based on the multiple image data.

[0168] Step 403c: display the reconstructed image corresponding to the target space.

[0169] Step 404c: In response to the adjustment operation on the reconstructed image, dynamically display the adjustment effect of the reconstructed image.

[0170] The optional implementation of reconstructing the target space according to the multiple image data can refer to the records of the above embodiments, which will not be described in detail here. On the basis of reconstructing the target space, based on the data processing method provided in this embodiment, the reconstruction result can be further dynamically displayed.

[0171] The adjustment operation of the reconstructed image may be initiated by the user. For example, when the target space is implemented as a house, the user may be a house viewing user or a decoration designer user, etc. The user may adjust the reconstructed image according to the needs.

[0172] Optionally, the adjustment operation performed on the reconstructed image includes: a movement operation on an object in the reconstructed image, a deletion operation on an object in the reconstructed image, and an operation of adding at least one object in the reconstructed image.

[0173] Based on the above operations, the layout operation or decoration operation of the target space can be simulated. For example, for a reconstructed image of a house, the user can move the furniture or decorative items in the house by touching or dragging the mouse to simulate the layout operation of the room; or, the furniture replacement operation can be simulated by deleting existing furniture and adding other furniture.

[0174] In this embodiment, after the target space is reconstructed, the reconstructed image of the target space is displayed, and the adjustment effect of the reconstructed image can be dynamically displayed in response to the adjustment operation on the reconstructed image. Furthermore, based on the reconstructed image, the user can be provided with a simulated adjustment operation for the target space.

[0175] Figure 4d A flowchart of a data processing method provided for another exemplary embodiment of the present application is shown in FIG. Figure 4d As shown, the method includes:

[0176] Step 401d: Acquire multiple image data of the target space, wherein the multiple image data include multiple spatial panoramic images corresponding to multiple single spaces and panoramic images of multiple spatial connection points connecting the multiple single spaces.

[0177] Step 402d: Determine a first spatial panoramic image and a first spatial connection panoramic image having an overlapping area from the multiple image data according to the feature points respectively included in the multiple spatial panoramic images and the multiple spatial connection panoramic images.

[0178] Step 403d: correct the panoramic image of the first space and the panoramic image at the connection of the first space according to the pixel distribution characteristics of the overlapping area.

[0179] Among them, based on the feature points respectively contained in the multiple spatial panoramic images and the panoramic images at the multiple spatial connections, the operation of determining the first spatial panoramic image and the panoramic image at the first spatial connection where there are overlapping areas from multiple image data can be achieved by feature point detection and feature point matching. For details, please refer to the records in the aforementioned embodiments, which will not be repeated here.

[0180] There is an overlapping area between the panoramic image of the first space and the panoramic image at the connection of the first space, which means that there is an overlapping area between the spatial area represented by the panoramic image of the first space and the spatial area represented by the panoramic image at the connection of the first space. For example, the spatial area represented by the panoramic image corresponding to the kitchen is the kitchen, and the spatial area represented by the panoramic image corresponding to the kitchen door includes a part of the kitchen and a part of the restaurant. Then, there is a part of the spatial area represented by the two, that is, a part of the kitchen. Based on the pixel distribution characteristics of the overlapping area, the panoramic image of the first space and the panoramic image at the connection of the first space can be corrected. Among them, the correction may include correcting the pixel points with errors in the overlapping area, standardizing the geometric lines in the image, or compensating the clarity of the blurred area in the image, etc. This embodiment includes but is not limited to this.

[0181] In this embodiment, the spatial panoramic image and the panoramic image at the spatial connection can be corrected according to the overlapping area between the spatial panoramic image and the panoramic image at the spatial connection, so that higher quality images can be obtained, which is conducive to providing more stable data support for spatial reconstruction.

[0182] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 201 to 204 can be device A; for another example, the execution subject of steps 201 and 202 can be device A, and the execution subject of step 203 can be device B; and so on.

[0183] In addition, some of the processes described in the above embodiments and drawings include multiple operations that appear in a specific order, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The operation serial numbers, such as 201, 202, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order.

[0184] In addition, these processes may include more or fewer operations, and these operations may be performed sequentially or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to different types.

[0185] Figure 5 A typical application scenario of the embodiment of the present application is illustrated. Figure 5In the illustration, a panoramic image of each room in a house and a panoramic image of the room door can be taken in the adjacent order of the rooms. The panoramic image of each room and the panoramic image of the room door are input into a computer device, and the connection relationship between different rooms is predicted by the computer device, and the house is reconstructed based on the connection relationship and the captured panoramic images.

[0186] When predicting the connection relationship between different rooms, the computer device can pre-process the geometric information of each room panoramic image to correct the wall lines and door lines in the reconstructed single room, and extract the door area from the room panoramic image to obtain multiple room doors.

[0187] Next, the computer device can obtain the orientation information of multiple room doors based on the IMU data of the image acquisition device, and estimate the connection relationship between the room doors based on the orientation information of the room doors. In addition, the computer device can further estimate the room doors that may have a connection relationship based on the time sequence of image acquisition.

[0188] Next, the computer device may perform a feature point detection operation for each panoramic image, and determine matching feature points on the panoramic image of the room and the panoramic image at the door of the room. Based on the matching feature points on the panoramic image of the room and the panoramic image at the door of the room, the connection weight between the door of the room may be calculated. Furthermore, the connection weight between the door of the room may be corrected based on the time sequence of image acquisition.

[0189] Next, the computer device can use multiple room doors as door nodes, and construct a graph model, i.e., an undirected weighted graph, based on the connection relationship between the multiple room doors and the connection weights between the multiple room doors. Then, the graph model pruning operation can be performed, and a spanning tree can be calculated based on the pruned graph model. Based on the spanning tree, a relatively accurate connection relationship between rooms can be predicted, and based on the connection relationship between rooms, multiple reconstructed single spaces can be combined to obtain a reconstructed house.

[0190] Figure 6 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application, such as Figure 6 As shown, the electronic device includes: a memory 601 and a processor 602.

[0191] The memory 601 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0192] Among them, the memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0193] Processor 602 is coupled to memory 601 and is used to execute a computer program in memory 601, so as to: obtain multiple spatial panoramic images corresponding to multiple single spaces and panoramic images of multiple spatial connections connecting the multiple single spaces; estimate candidate connection relationships and connection weights of multiple entrances corresponding to the multiple single spaces based on the multiple spatial panoramic images and the panoramic images of the multiple spatial connections; create an undirected weighted graph corresponding to the multiple entrances based on the candidate connection relationships and connection weights; and calculate the connection relationship of the multiple entrances based on the undirected weighted graph, so as to determine the connection relationship of the multiple single spaces based on the connection relationship of the multiple entrances.

[0194] Further optionally, when the processor 602 estimates the candidate connection relationships at the multiple entrances corresponding to the multiple single spaces based on the multiple spatial panoramic images and the panoramic images at the multiple spatial connections, it is specifically used to: identify the multiple entrances corresponding to the multiple single spaces from the multiple spatial panoramic images; determine the orientation information of the multiple entrances based on the posture data when the image acquisition device acquires the spatial panoramic images; and estimate the existence of the candidate connection relationships between the entrances with opposite orientations based on the orientation information of the multiple entrances.

[0195] Further optionally, when the processor 602 estimates the candidate connection relationships at the multiple entrances corresponding to the multiple single spaces based on the multiple spatial panoramic images and the panoramic images at the multiple spatial connections, it is specifically used to: determine the acquisition order of the multiple spatial panoramic images and the panoramic images at the multiple spatial connections based on the image acquisition time information; filter, from the multiple spatial panoramic images, the spatial panoramic images whose acquisition order differences are within a set order difference range according to the acquisition order; and estimate the entrances contained in the spatial panoramic images whose acquisition order differences are within the set order difference range as entrances where the candidate connection relationships exist.

[0196] Further optionally, when the processor 602 estimates the connection weights of the multiple entrances corresponding to the multiple single spaces based on the multiple spatial panoramic images and the panoramic images at the multiple spatial connections, it is specifically used to: for the first entrance and the second entrance, obtain the first spatial panoramic image where the first entrance is located and the second spatial panoramic image where the second entrance is located; and estimate the connection weight between the first entrance and the second entrance based on image features of the first spatial panoramic image and the second spatial panoramic image.

[0197] Further optionally, when the processor 602 estimates the connection weight between the first entrance and the second entrance based on the image features of the first spatial panoramic image and the second spatial panoramic image, it is specifically used to: respectively obtain the feature points that match between the first spatial panoramic image and the panoramic images at the multiple spatial connections, and obtain multiple first feature point matching numbers; and respectively obtain the feature points that match between the second spatial panoramic image and the panoramic images at the multiple spatial connections, and obtain multiple second feature point matching numbers; select the first minimum value among the multiple first feature point matching numbers and the second minimum value among the multiple second feature point matching numbers; and select the maximum value between the first minimum value and the second minimum value as the connection weight between the first entrance and the second entrance.

[0198] Further optionally, when the processor 602 respectively obtains the feature points that match between the first spatial panoramic image and the panoramic images of the multiple spatial connections to obtain multiple first feature point matching numbers, it is specifically used to: for the first spatial connection among the multiple spatial connections, obtain the feature points that match between the first spatial panoramic image and the panoramic image of the first spatial connection to obtain a third feature point matching number; if the third feature point matching number is greater than a set feature point number threshold, use the third feature point matching number as the first feature point matching number; if the third feature point matching number is less than or equal to the feature point number threshold, set the first feature point matching number to the set value.

[0199] Further optionally, the processor 602 is also used to: determine the acquisition order of the multiple spatial panoramic images and the panoramic images at the multiple spatial connections based on the image acquisition time information; if the acquisition order difference between the first spatial panoramic image and the second spatial panoramic image is within a set order difference range, then add an additional weight to the estimated connection weight of the first entrance and the second entrance.

[0200] Further optionally, when the processor 602 creates an undirected weighted graph corresponding to the multiple entrances based on the candidate connection relationships and the connection weights, it is specifically used to: take the multiple entrances as multiple nodes, and determine the first type of connection edges between the multiple nodes based on the candidate connection relationships; determine the weights on the first type of connection edges based on the connection weights to obtain the undirected weighted graph.

[0201] Further optionally, the processor 602 is also used to: determine a third spatial panoramic image from the multiple spatial panoramic images, wherein the third spatial panoramic image includes multiple entrances; in the undirected weighted graph, add a second type of connecting edges between the nodes corresponding to the multiple entrances included in the third space, and set a fixed weight for the second type of connecting edges.

[0202] Further optionally, the processor 602 is also used to: filter out a third entrance and a fourth entrance with no connection relationship from the multiple entrances based on the multiple spatial panoramic images and the panoramic images at the multiple spatial connections; and delete the connecting edges between the nodes corresponding to the third entrance and the fourth entrance in the undirected weighted graph.

[0203] Further optionally, when the processor 602 screens the third entrance and the fourth entrance with no connection relationship from the multiple entrances based on the multiple spatial panoramic images and the panoramic images of the multiple spatial connections, the processor 602 is specifically used to: obtain the visible range of the third entrance based on the spatial panoramic image where the third entrance is located; obtain matching feature points between the spatial panoramic image where the third entrance is located and the panoramic image at the second spatial connection; wherein the correlation between the panoramic image at the second spatial connection and the spatial panoramic image where the fourth entrance is located is greater than a set correlation threshold; if the matching feature points are not within the visible range of the third entrance, it is determined that the candidate connection relationship does not exist between the third entrance and the fourth entrance.

[0204] Further optionally, when the processor 602 filters out the third entrance and the fourth entrance with no connection relationship from the multiple entrances based on the multiple spatial panoramic images and the panoramic images at the multiple spatial connections, it is specifically used to: if the name of the panoramic image at the third spatial connection contains the name of the panoramic image where the third entrance is located, and the panoramic image where the third entrance is located only contains the third entrance, then it is determined that the panoramic image at the third spatial connection is associated with the third entrance; if the degree of correlation between the panoramic image where the fourth entrance is located and the panoramic image at the third spatial connection is less than or equal to a set correlation threshold, then it is determined that the third entrance and the fourth entrance do not have the candidate connection relationship.

[0205] Further optionally, when the processor 602 screens the third entrance and the fourth entrance with no connection relationship from the multiple entrances based on the multiple spatial panoramic images and the panoramic images of the multiple spatial connections, it is specifically used to: if the spatial panoramic image where the third entrance is located only contains the third entrance, and the spatial panoramic image where the fourth entrance is located only contains the fourth entrance, then it is determined that the third entrance and the fourth entrance do not have the candidate connection relationship.

[0206] Further optionally, when the processor 602 screens the third entrance and the fourth entrance with no connection relationship from the multiple entrances based on the multiple spatial panoramic images and the panoramic images of the multiple spatial connections, it is specifically used to: if the geometric shapes of the third entrance and the fourth entrance do not match, determine that the candidate connection relationship does not exist between the third entrance and the fourth entrance.

[0207] Further optionally, when the processor 602 calculates the connection relationship of the multiple entrances according to the undirected weighted graph, it is specifically used to: obtain multiple spanning trees that meet the set weight conditions from the undirected weighted graph; estimate the architectural characteristics of the multiple single spaces according to the node connection relationship represented by the multiple spanning trees; select a target spanning tree from the multiple spanning trees according to the architectural characteristics of the multiple single spaces; and determine the connection relationship of the multiple entrances according to the node connection relationship represented by the target spanning tree.

[0208] Further optionally, when the processor 602 determines multiple spanning trees that meet the set weight conditions from the undirected weighted graph, it is specifically used to: create an initial maximum spanning tree according to the weights on the connecting edges in the undirected weighted graph; repeat the following steps until the initial maximum spanning tree is updated to a new spanning tree that does not contain the third type of connecting edges, and transform the edges in the new spanning tree to obtain the multiple spanning trees: use the first type of connecting edges and / or the second type of connecting edges in the undirected weighted graph that are not included in the initial maximum spanning tree to replace the third type of connecting edges in the initial maximum spanning tree; if after performing the replacement operation, the connecting edges in the new spanning tree form a loop, then delete the third type of connecting edges in the loop and / or the first type of connecting edges whose weights are less than the set weight threshold.

[0209] Further optionally, when the processor 602 estimates the architectural features of the multiple single spaces after reconstruction based on the node connection relationship represented by the multiple spanning trees, it is specifically used to: determine, for a first spanning tree among the multiple spanning trees, the distribution positions of the multiple single spaces based on a depth-first traversal algorithm and the connection relationship between the multiple entrances represented by the first spanning tree; calculate, based on the distribution positions of the multiple single spaces, the overlapping areas of the multiple single spaces as the overlapping areas corresponding to the first spanning tree; accordingly, when the processor 602 selects a target spanning tree from the multiple spanning trees based on the architectural features of the multiple single spaces after reconstruction, it is specifically used to: select, based on the overlapping areas corresponding to each of the multiple spanning trees, a spanning tree whose overlapping area is less than a set overlapping area threshold as the target spanning tree.

[0210] Further optionally, the processor 602 is also used to: draw a bounding box of the area where the multiple single spaces are located according to the distribution positions of the multiple single spaces; calculate the area of ​​the bounding box as the bounding box area corresponding to the first spanning tree; calculate the spatial distribution rationality index corresponding to the first spanning tree according to the overlapping area and the bounding box area corresponding to the first spanning tree; accordingly, when the processor 602 selects a target spanning tree from the multiple spanning trees according to the architectural features after the reconstruction of the multiple single spaces, it is specifically used to: select a spanning tree whose spatial distribution rationality index is greater than a set index threshold from the multiple spanning trees as the target spanning tree according to the spatial distribution rationality index corresponding to each of the multiple spanning trees.

[0211] Further, if Figure 6 As shown, the electronic device also includes: a communication component 603, a display component 604, a power supply component 605 and other components. Figure 6 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 6 Components shown.

[0212] Among them, the communication component 603 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can be implemented based on near field communication (NFC) technology, radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0213] Among them, the display component 604 includes a screen, and the screen may include a liquid crystal display component (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0214] The power supply component 605 provides power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply component is located.

[0215] In this embodiment, when predicting the connection relationship between multiple single spaces, based on the multiple spatial panoramic images corresponding to the multiple single spaces and the panoramic images of the multiple spatial connections connecting the multiple single spaces, the candidate connection relationships and connection weights of the multiple entrances corresponding to the multiple single spaces can be preliminarily estimated; based on the candidate connection relationships and connection weights, an undirected weighted graph corresponding to the multiple entrances can be created; and the connection relationship of the multiple entrances is calculated based on the undirected weighted graph, which effectively improves the robustness of the spatial relationship calculation method and the reliability of the calculation results.

[0216] In addition to the processing logic described in the above embodiments, Figure 6 The electronic device shown in the figure may also execute the following processing logic: obtaining multiple image data of a target space through the processor 602, wherein the target space includes multiple subspaces; constructing a graph model based on the multiple image data, wherein the graph model uses the entrances corresponding to the multiple subspaces as nodes; identifying the connection relationship between the multiple subspaces according to the graph model; and displaying the connection relationship. No further details will be given.

[0217] In addition to the processing logic described in the above embodiments, Figure 6 The illustrated electronic device can also execute the following processing logic: obtaining multiple image data of the target space through the processor 602, wherein the target space includes multiple subspaces; reconstructing the target space based on the multiple image data; identifying the multiple image data to identify the objects contained in each of the multiple subspaces; displaying the reconstructed image corresponding to the target space, and marking the objects contained in each of the multiple subspaces on the reconstructed image.

[0218] Further optionally, the processor 602 is also used to: identify shooting points corresponding to the multiple image data based on the image content contained in the multiple image data; mark the shooting points in the reconstructed image; wherein the shooting points include shooting points in the multiple subspaces and shooting points at the spatial connections corresponding to the multiple subspaces.

[0219] In addition to the processing logic described in the above embodiments, Figure 6 The illustrated electronic device can also execute the following processing logic: obtaining multiple image data of the target space through the processor 602, wherein the target space includes multiple subspaces; reconstructing the target space based on the multiple image data; displaying the reconstructed image corresponding to the target space; and responding to an adjustment operation on the reconstructed image, dynamically displaying the adjustment effect of the reconstructed image.

[0220] Further optionally, the adjustment operation includes at least one of the following: an operation of moving an object in the reconstructed image; an operation of deleting an object in the reconstructed image; an operation of adding at least one object in the reconstructed image.

[0221] In addition to the processing logic described in the above embodiments, Figure 6 The illustrated electronic device can also execute the following processing logic: obtaining multiple image data of the target space through the processor 602, the multiple image data including multiple spatial panoramic images corresponding to multiple single spaces and panoramic images of multiple spatial connections connecting the multiple single spaces; determining a first spatial panoramic image and a panoramic image at a first spatial connection where there is an overlapping area from the multiple image data based on feature points contained in the multiple spatial panoramic images and the panoramic images at the multiple spatial connections; and correcting the first spatial panoramic image and the panoramic image at the first spatial connection based on pixel distribution characteristics of the overlapping area.

[0222] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be executed by an electronic device in the above method embodiment.

[0223] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0224] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0225] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0226] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0227] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0228] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0229] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0230] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0231] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A spatial relationship calculation method, characterized in that: include: Acquire a plurality of image data of a target space, wherein the target space is an indoor space including a plurality of subspaces; Based on the multiple image data, construct a graph model with the entrances corresponding to the multiple subspaces as nodes; based on the multiple image data, constructing the graph model includes: based on the multiple image data, constructing the graph model by feature detection and feature matching; According to the graph model, identifying a connection relationship between the plurality of subspaces; The connection relationship is displayed.

2. A spatial relationship calculation method, characterized in that: include: Acquire a plurality of spatial panoramic images corresponding to a plurality of single spaces and a panoramic image of a plurality of spatial connection points connecting the plurality of single spaces; estimating candidate connection relationships and connection weights of multiple entrances corresponding to the multiple single spaces according to the multiple spatial panoramic images and the panoramic images of the multiple spatial connections; Creating an undirected weighted graph corresponding to the multiple entrances according to the candidate connection relationships and connection weights; According to the undirected weighted graph, calculating the connection relationship of the plurality of entrances, so as to determine the connection relationship of the plurality of single spaces according to the connection relationship of the plurality of entrances; Among them, the estimating the candidate connection relationship and connection weights of the multiple entrances corresponding to the multiple single spaces based on the multiple spatial panoramic images and the panoramic images of the multiple spatial connections includes: estimating the candidate connection relationship and connection weights of the multiple entrances corresponding to the multiple single spaces by using feature detection and feature matching based on the multiple spatial panoramic images and the panoramic images of the multiple spatial connections.

3. The method according to claim 2, characterized in that According to the plurality of spatial panoramic images and the panoramic images of the plurality of spatial connections, estimating candidate connection relationships at the plurality of entrances corresponding to the plurality of single spaces, further comprising: identifying the plurality of entrances corresponding to the plurality of single spaces from the plurality of spatial panoramic images; Determining orientation information of the plurality of entrances according to posture data when an image acquisition device acquires the spatial panoramic image; According to the orientation information of the plurality of entrances, it is estimated that the candidate connection relationship exists between entrances with opposite orientations.

4. The method according to claim 2, characterized in that: According to the plurality of spatial panoramic images and the panoramic images of the plurality of spatial connections, estimating candidate connection relationships at the plurality of entrances corresponding to the plurality of single spaces, further comprising: Determining, according to the image acquisition time information, an acquisition order of the plurality of spatial panoramic images and the panoramic images at the plurality of spatial connections; According to the acquisition sequence, selecting, from the plurality of spatial panoramic images, spatial panoramic images whose acquisition sequence differences are within a set sequence difference range; The entrances included in the spatial panoramic image whose acquisition sequence difference is within the set sequence difference range are estimated as the entrances having the candidate connection relationship.

5. The method according to claim 2, characterized in that: According to the panoramic images of the multiple spaces and the panoramic images of the multiple spaces connected, the connection weights of the multiple entrances corresponding to the multiple single spaces are estimated by feature detection and feature matching, including: For a first entrance and a second entrance, obtaining a first spatial panoramic image where the first entrance is located and a second spatial panoramic image where the second entrance is located; Determine the first spatial panoramic image and the second spatial panoramic image by means of feature detection; The connection weight between the first entrance and the second entrance is estimated by matching feature points of the first spatial panoramic image and the second spatial panoramic image.

6. The method according to claim 5, characterized in that The method of estimating a connection weight between the first entrance and the second entrance by matching feature points of the first spatial panoramic image and the second spatial panoramic image includes: Respectively acquiring feature points that match between the first spatial panoramic image and the panoramic images at the plurality of spatial connections, to obtain a plurality of first feature point matching numbers; and Respectively acquiring feature points matched between the second spatial panoramic image and the panoramic images at the plurality of spatial connections, to obtain a plurality of second feature point matching numbers; Selecting a first minimum value among the plurality of first feature point matching quantities and a second minimum value among the plurality of second feature point matching quantities; A maximum value between the first minimum value and the second minimum value is selected as a connection weight between the first entrance and the second entrance.

7. The method according to claim 6, characterized in that Respectively acquiring the feature points that match between the first spatial panoramic image and the panoramic images at the plurality of spatial connections to obtain a plurality of first feature point matching numbers, including: For a first spatial connection among the multiple spatial connections, acquiring feature points that match between the first spatial panoramic image and the panoramic image of the first spatial connection to obtain a third number of feature points; If the third feature point matching number is greater than the set feature point number threshold, the third feature point matching number is used as the first feature point matching number; If the third feature point matching number is less than or equal to the feature point number threshold, the first feature point matching number is set to a set value.

8. The method according to claim 5, characterized in that Also includes: Determining, according to the image acquisition time information, an acquisition order of the plurality of spatial panoramic images and the panoramic images at the plurality of spatial connections; If the acquisition sequence difference between the first spatial panoramic image and the second spatial panoramic image is within a set sequence difference range, an additional weight is added to the estimated connection weight of the first entrance and the second entrance.

9. The method according to any one of claims 2 to 8, characterized in that: Creating an undirected weighted graph corresponding to the multiple entrances according to the candidate connection relationships and the connection weights, including: Taking the multiple entrances as multiple nodes, and determining a first type of connection edge between the multiple nodes according to the candidate connection relationship; According to the connection weights, the weights on the first type of connection edges are determined to obtain the undirected weighted graph.

10. The method according to claim 9, characterized in that Also includes: Determining a third spatial panoramic image from the plurality of spatial panoramic images, wherein the third spatial panoramic image includes a plurality of entrances; In the undirected weighted graph, second-type connection edges are added between nodes corresponding to a plurality of entrances included in the third space, and fixed weights are set for the second-type connection edges.

11. The method according to claim 9, characterized in that Also includes: Filtering a third entrance and a fourth entrance that have no connection relationship from the plurality of entrances according to the plurality of spatial panoramic images and the panoramic images of the plurality of spatial connections; In the undirected weighted graph, a connection edge between nodes corresponding to the third entrance and the fourth entrance is deleted.

12. The method according to claim 11, characterized in that According to the plurality of spatial panoramic images and the panoramic images of the plurality of spatial connections, a third entrance and a fourth entrance that do not have a connection relationship are screened from the plurality of entrances, including: Acquire a visible range of the third entrance according to a panoramic image of the space where the third entrance is located; Acquire matching feature points between the panoramic image of the space where the third entrance is located and the panoramic image at the connection of the second space; wherein the degree of correlation between the panoramic image at the connection of the second space and the panoramic image of the space where the fourth entrance is located is greater than a set correlation threshold; If the matched feature point is not within the visible range of the third entrance, it is determined that the candidate connection relationship does not exist between the third entrance and the fourth entrance.

13. The method according to claim 11, characterized in that According to the plurality of spatial panoramic images and the panoramic images of the plurality of spatial connections, a third entrance and a fourth entrance that do not have a connection relationship are screened from the plurality of entrances, including: If the name of the panoramic image at the connection point of the third space includes the name of the panoramic image where the third entrance is located, and the panoramic image where the third entrance is located only includes the third entrance, then it is determined that the panoramic image at the connection point of the third space is associated with the third entrance; If the correlation between the panoramic image where the fourth entrance is located and the panoramic image at the connection point of the third space is less than or equal to the set correlation threshold, it is determined that the candidate connection relationship does not exist between the third entrance and the fourth entrance.

14. The method according to claim 11, characterized in that According to the plurality of spatial panoramic images and the panoramic images of the plurality of spatial connections, a third entrance and a fourth entrance that do not have a connection relationship are screened from the plurality of entrances, including: If the spatial panoramic image where the third entrance is located only includes the third entrance, and the spatial panoramic image where the fourth entrance is located only includes the fourth entrance, it is determined that the candidate connection relationship does not exist between the third entrance and the fourth entrance.

15. The method according to claim 11, characterized in that According to the plurality of spatial panoramic images and the panoramic images of the plurality of spatial connections, a third entrance and a fourth entrance that do not have a connection relationship are screened from the plurality of entrances, including: If the geometric shapes of the third entrance and the fourth entrance do not match, it is determined that the candidate connection relationship does not exist between the third entrance and the fourth entrance.

16. The method according to claim 10, characterized in that Calculating the connection relationship of the plurality of entrances according to the undirected weighted graph includes: Obtaining multiple spanning trees satisfying set weight conditions from the undirected weighted graph; estimating architectural features of the plurality of single spaces according to the node connection relationships represented by the plurality of spanning trees; selecting a target spanning tree from the plurality of spanning trees according to architectural features of the plurality of single spaces; The connection relationship of the multiple entries is determined according to the node connection relationship represented by the target spanning tree.

17. The method according to claim 16, characterized in that Determining a plurality of spanning trees satisfying a set weight condition from the undirected weighted graph includes: Creating an initial maximum spanning tree according to the weights on the connecting edges in the undirected weighted graph; Repeat the following steps until the initial maximum spanning tree is updated to a new spanning tree that does not include the third type of connection edge, and the edges in the new spanning tree are transformed to obtain the multiple spanning trees: Using the first type of connection edges and / or the second type of connection edges in the undirected weighted graph that are not included in the initial maximum spanning tree to replace the third type of connection edges in the initial maximum spanning tree; If the connection edges in the new spanning tree form a loop after performing the replacement operation, the connection edges of the third type in the loop and / or the connection edges of the first type whose weights are less than a set weight threshold are deleted.

18. The method according to claim 16, characterized in that According to the node connection relationships represented by the multiple spanning trees, estimating the architectural features of the multiple single spaces after reconstruction includes: For a first spanning tree among the multiple spanning trees, based on a depth-first traversal algorithm and a connection relationship between multiple entrances represented by the first spanning tree, determining distribution positions of the multiple single spaces; Calculating, according to the distribution positions of the multiple single spaces, overlapping areas of the multiple single spaces as overlapping areas corresponding to the first spanning tree; Selecting a target spanning tree from the plurality of spanning trees according to the architectural features after the plurality of single-space reconstructions includes: According to the overlapping areas corresponding to the multiple spanning trees respectively, a spanning tree whose overlapping area is smaller than a set overlapping area threshold is selected from the multiple spanning trees as the target spanning tree.

19. The method according to claim 18, characterized in that Also includes: Drawing bounding boxes of areas where the multiple single spaces are located according to the distribution positions of the multiple single spaces; Calculating the area of ​​the bounding box as the area of ​​the bounding box corresponding to the first spanning tree; Calculating a reasonable spatial distribution index corresponding to the first spanning tree according to an overlapping area and a bounding box area corresponding to the first spanning tree; Selecting a target spanning tree from the plurality of spanning trees according to the architectural features after the plurality of single-space reconstructions includes: According to the reasonable spatial distribution indexes corresponding to the multiple spanning trees, a spanning tree having a reasonable spatial distribution index greater than a set index threshold is selected from the multiple spanning trees as the target spanning tree.

20. A data processing method, characterized in that: include: Acquire a plurality of image data of a target space, wherein the target space is an indoor space including a plurality of subspaces; Based on the multiple image data, construct a graph model with the entrances corresponding to the multiple subspaces as nodes; based on the multiple image data, constructing the graph model includes: based on the multiple image data, constructing the graph model by feature detection and feature matching; According to the graph model, identifying the connection relationship between the multiple subspaces to reconstruct the target space; Recognize the multiple image data to identify the objects contained in each of the multiple subspaces; The reconstructed image corresponding to the target space is displayed, and the objects contained in each of the multiple subspaces are marked on the reconstructed image.

21. The method according to claim 20, characterized in that Also includes: Identifying shooting points corresponding to the plurality of image data according to image contents contained in the plurality of image data; marking the shooting point in the reconstructed image; The shooting points include shooting points in the multiple subspaces and shooting points at spatial connections corresponding to the multiple subspaces.

22. A data processing method, characterized in that: include: Acquire a plurality of image data of a target space, wherein the target space is an indoor space including a plurality of subspaces; Based on the multiple image data, construct a graph model with the entrances corresponding to the multiple subspaces as nodes; based on the multiple image data, constructing the graph model includes: based on the multiple image data, constructing the graph model by feature detection and feature matching; According to the graph model, identifying the connection relationship between the multiple subspaces to reconstruct the target space; Displaying a reconstructed image corresponding to the target space; In response to the adjustment operation on the reconstructed image, the adjustment effect of the reconstructed image is dynamically displayed.

23. The method according to claim 22, characterized in that The adjustment operation includes at least one of the following: moving an object in the reconstructed image; Deleting the object in the reconstructed image; An operation of adding at least one object to the reconstructed image.

24. A data processing method, characterized in that: include; Acquire a plurality of image data of the target space, wherein the plurality of image data includes a plurality of spatial panoramic images corresponding to a plurality of single spaces and a panoramic image of a plurality of spatial connection points connecting the plurality of single spaces; estimating candidate connection relationships and connection weights of multiple entrances corresponding to the multiple single spaces according to the multiple spatial panoramic images and the panoramic images of the multiple spatial connections; Creating an undirected weighted graph corresponding to the plurality of entrances according to the candidate connection relationships and the connection weights, so as to determine the first spatial panoramic image with overlapping areas and the panoramic image at the first spatial connection from the plurality of image data; Correcting the panoramic image of the first space and the panoramic image at the connection of the first space according to the pixel distribution characteristics of the overlapping area; Among them, based on the multiple spatial panoramic images and the panoramic images of the multiple spatial connections, estimating the candidate connection relationships and connection weights of the multiple entrances corresponding to the multiple single spaces includes: based on the multiple spatial panoramic images and the panoramic images of the multiple spatial connections, using feature detection and feature matching methods, estimating the feature points respectively contained in the candidate connection relationships and connection weights of the multiple entrances corresponding to the multiple single spaces.

25. An electronic device, characterized in that: include: Memory and processor; The memory is used to store one or more computer instructions; The processor is used to execute the one or more computer instructions to: execute the steps in the spatial relationship calculation method described in any one of claims 1-19 or the data processing method described in any one of claims 20-24.

26. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it can implement the spatial relationship calculation method described in any one of claims 1 to 19 or the data processing method described in any one of claims 20 to 24.

Images