Shooting point location line connection method and device based on directed graph, equipment and medium

Abstract

The embodiment of the invention provides a shooting point location line connection method and device based on a directed graph, equipment and a medium. According to the embodiment of the invention, the method comprises the steps: obtaining a panoramic image which is shot at a first shooting point and contains depth information; the panoramic image comprises a plane image and a gate image; performing scene information extraction based on the panoramic image to obtain door point information and plane information; extracting at least one effective plane from the panoramic image by using the plane information; and predicting a next shooting point position meeting a screening condition from the panoramic image by using the effective plane and / or gate point information. The user only needs to move the camera to the predicted next shooting point to shoot the second panoramic image, so that the shot image coherence effect can be ensured, and meanwhile, excessive repeated image content cannot be shot. The multi-point continuous shooting requirement for generating the panoramic image can be met, and a shooting user can be better guided to continuously and efficiently shoot the panoramic image.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment and medium for connecting shooting points based on a directed graph. Background Art

[0002] With the continuous development of virtual reality technology, panoramas are widely used in house rental, hotel homestay, home decoration, various exhibitions and other fields. Compared with ordinary single-perspective images, panoramas can provide broader field of view information, allowing people to restore the scene to a certain extent, and users can get a better sense of immersion when watching. Therefore, panorama previews are becoming more and more popular and sought after by the public.

[0003] In the fields of 3D reconstruction and VR (Virtual Reality), more and more panoramic data are used to construct subsequent tasks. However, in the process of selecting shooting points, different people have different ideas. They may shoot particularly dense points, resulting in slow processing speed and longer shooting time. They may also shoot some isolated points, which will affect the subsequent automatic registration algorithm and make the registration result very poor. There will also be various other uncontrollable situations that affect the effect and efficiency of panoramic image construction and increase the workload of image shooting users. Therefore, there is a need for a solution that can guide users to correctly shoot panoramic images. Summary of the invention

[0004] Multiple aspects of the present application provide a method, device, equipment and medium for connecting shooting points based on a directed graph, so as to solve the technical problem of how to reasonably arrange the shooting points of a panoramic image.

[0005] In a first aspect, an exemplary embodiment of the present application provides a method for connecting shooting points based on a directed graph, the method comprising:

[0006] Acquire the first shooting point information when shooting an image in the house at the first shooting point, display a directed graph containing the information of each shooting point in the panoramic image of the house, and establish a directed connection between each point in the directed graph;

[0007] In response to a manual movement instruction of the user to the first shooting point, determining an evaluation score of a directed connection between the first shooting point and surrounding shooting points;

[0008] Based on the evaluation scores, a directed connection relationship is established between the first shooting point and other shooting points in the surrounding shooting points.

[0009] Optionally, after obtaining the first shooting point information when the image of the information inside the house is shot at the first shooting point, the method includes:

[0010] Determine, according to the shooting order of the shooting points, a first point ID corresponding to the first shooting point;

[0011] According to the first point ID, the point IDs corresponding to the surrounding shooting points are grouped to obtain a first point group and a second point group; wherein each point ID in the first point group is smaller than the first point ID, and each point ID in the second point group is larger than the first point ID.

[0012] Optionally, in response to a manual movement instruction of the user to the first shooting point, determining an evaluation score of a directed connection between the first shooting point and surrounding shooting points; and establishing a directed connection relationship between the first shooting point and other points in the surrounding shooting points based on the evaluation score, comprises:

[0013] Based on the first point grouping, a candidate point list is established; the candidate point list includes point IDs;

[0014] In response to a manual movement instruction of the user for the first shooting point, selecting a first candidate point from the candidate point list;

[0015] Acquire the position information corresponding to the first candidate point, and calculate a first distance value between the first candidate point and the first shooting point; and,

[0016] Calculating a first degree of overlap using point cloud information of the first candidate point and the first shooting point;

[0017] Calculate the first evaluation score corresponding to each of the candidate points in the candidate point list corresponding to the first point group by using the first distance value and the first overlap degree;

[0018] A candidate point is selected from the candidate point list as a target shooting point for establishing a connection relationship with the first shooting point, so as to establish a directed connection relationship between the first shooting point and the target shooting point.

[0019] Optionally, the calculating a first degree of overlap by using the point cloud information of the first candidate point and the first shooting point includes:

[0020] Acquire first point cloud information captured at the first candidate point;

[0021] Acquire second point cloud information obtained by photographing at the first photographing point;

[0022] Counting the number of point clouds that appear in the first point cloud information and the second point cloud information;

[0023] The ratio of the number of the point cloud and the number of the point cloud appearing in the second point cloud is calculated, and the ratio is used as the first overlap degree.

[0024] Optionally, the calculating, by using the first distance value and the first overlap degree, the evaluation score corresponding to each of the candidate points in the candidate point list corresponding to the first point grouping includes:

[0025] Acquire a first coordinate of the first candidate point, and acquire a second coordinate of the first shooting point;

[0026] Calculate the first distance value using the first coordinate and the second coordinate;

[0027] The first distance value and the first overlap degree are input into a first calculation formula:

[0028] score=1 / (dist+1e-6)+overlap;

[0029] Among them, score is the first evaluation score, distb represents the first distance value, and overlap represents the first repetition degree.

[0030] Optionally, before inputting the first distance value and the first overlap into a first calculation formula, the method further includes:

[0031] Determining whether the first distance value is greater than a distance threshold, and determining whether the first overlap degree is greater than an overlap degree threshold;

[0032] If the judgment result is that the first distance value is greater than the distance threshold, and / or the first overlap is less than the overlap threshold, a maximum value is assigned to the target shooting point.

[0033] Optionally, in response to a manual movement instruction of the user to the first shooting point, determining an evaluation score of a directed connection between the first shooting point and surrounding shooting points; and establishing a directed connection relationship between the first shooting point and other points in the surrounding shooting points based on the evaluation score, comprises:

[0034] Based on the second point grouping, a candidate point list is established; the candidate point list includes point IDs;

[0035] In response to a user's manual movement instruction for the first shooting point, selecting a second candidate point from the candidate point list;

[0036] Acquire the posture information corresponding to the second candidate point, and calculate the second distance value between the second candidate point and the first shooting point; and,

[0037] Calculating a second degree of overlap using point cloud information of the second candidate point and the first shooting point;

[0038] Calculate the second evaluation score corresponding to each of the candidate points in the candidate point list corresponding to the first point group by using the second distance value and the second overlap degree;

[0039] A candidate point is selected from the candidate point list as a target shooting point for establishing a connection relationship with the first shooting point, so as to establish a directed connection relationship between the first shooting point and the target shooting point.

[0040] Optionally, it also includes:

[0041] If the parent node corresponding to the second candidate point is assigned a maximum value, the parent node corresponding to the second candidate point replaces the second candidate node;

[0042] If the parent node corresponding to the second candidate point is not assigned a maximum value, calculating a third evaluation score corresponding to the parent node corresponding to the second candidate point; and,

[0043] If the second evaluation score is greater than the third evaluation score, the second candidate node is replaced by the parent node corresponding to the second candidate point.

[0044] In a second aspect, an exemplary embodiment of the present application provides a shooting point connection device based on a directed graph, the device comprising:

[0045] The acquisition module is used to obtain the first shooting point information when shooting an image inside the house at the first shooting point, display a directed graph containing the information of each shooting point in the house panoramic view, and establish directed connections between the points in the directed graph.

[0046] The judgment module is used to judge the evaluation score of the directed connection between the first shooting point and surrounding shooting points in response to the user's manual movement instruction for the first shooting point.

[0047] A connection establishing module is used to establish a directed connection relationship between the first shooting point and other points in the surrounding shooting points based on the size of the evaluation score.

[0048] In a third aspect, an exemplary embodiment of the present application further provides an electronic device, comprising: a memory and a processor; wherein the memory is used to store a computer program; the processor is coupled to the memory, and is used to execute the computer program for executing the steps in the above-mentioned method of connecting shooting points based on a directed graph.

[0049] In a fourth aspect, an exemplary embodiment of the present application provides a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, causes the one or more processors to execute the steps in the above-mentioned method for connecting shooting points based on a directed graph.

[0050] In this embodiment, when shooting in a house environment, since there are many rooms in the house, it is necessary to shoot comprehensively from multiple different shooting points to generate a complete panoramic image. However, the user who performs the shooting task cannot know whether the current shooting point is suitable. The present application proposes a directed graph for displaying the relationship between shooting points, in which each shooting point is included, and a directed connection relationship between two points is established according to the ID value of the shooting point. Specifically, the first shooting point information when the information inside the house is imaged at the first shooting point is obtained, and a directed graph containing the information of each shooting point in the house panoramic view is displayed, and a directed connection is established between each point in the directed graph; in response to the user's manual movement instruction for the first shooting point, the positional relationship between the first shooting point and the surrounding shooting points is determined, and a directed connection relationship between the first shooting point and other points in the surrounding shooting points is established; based on the connection information, the directed connection relationship is re-established for the first shooting point. Through the above scheme, when taking images of the house at different shooting points, it is necessary to determine whether the shooting points are reasonable, because if the shooting points are unreasonable, the panoramic image cannot be established or the established panoramic image has defects. If it is reasonable, a directed connection relationship between each shooting point will be established in the directed graph. If the first shooting point cannot establish a directed connection relationship with the surrounding shooting positions, it may be that the position of the first shooting point is unreasonable. At this time, the user can manually move the position of the first shooting point as needed, so that the position of the first shooting point in the directed graph is consistent with the actual shooting point. When the first shooting point is moved, it is necessary to re-correct the connection relationship between the first shooting point and the surrounding shooting points to obtain a complete directed graph, so that the shooting user can have an intuitive feeling of the overall shooting effect through the directed graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0052] Figure 1 A schematic diagram of a flow chart of a method for connecting shooting points based on a directed graph provided in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of a directed graph for illustrating an embodiment of the present application;

[0054] Figure 3 A wiring scheme based on a directed graph is illustrated as an example in the embodiment of the present application;

[0055] Figure 4 A schematic diagram of the structure of a shooting point connection device based on a directed graph provided in an embodiment of the present application;

[0056] Figure 5 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0058] With the continuous development of virtual reality technology, panoramas are widely used in house rental, hotel homestay, home decoration, various exhibitions and other fields. Compared with ordinary single-perspective images, panoramas can provide broader field of view information, allowing people to restore the scene to a certain extent. Therefore, panorama previews are becoming more and more popular and sought after by the public.

[0059] In the fields of 3D reconstruction and VR (Virtual Reality), more and more panoramic data are used to construct subsequent tasks. During the shooting process, the photographer often chooses the shooting point based on his own experience, but the overall shooting effect cannot be understood in time. In other words, the photographer does not know whether the shooting point he currently chooses is reasonably matched with other shooting points. For example, it is unknown whether the distance, angle, etc. are reasonable. Therefore, a method is needed to judge whether the shooting point is reasonable from an objective perspective.

[0060] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0061] Figure 1 The following is a flow chart of a method for connecting shooting points based on a directed graph provided in an embodiment of the present application. Figure 1 As shown, the directed graph-based connection method provided in the embodiment of the present application can be run on a server side, which can be a local server or a cloud server. The method specifically includes the following steps:

[0062] Step 101: obtaining first shooting point information when shooting an image inside a house at a first shooting point, displaying a directed graph including each shooting point information in a panoramic image of the house, and establishing directed connections between each point in the directed graph.

[0063] Step 102: In response to a manual movement instruction of the first shooting point by a user, determining an evaluation score of a directed connection between the first shooting point and surrounding shooting points.

[0064] Step 103: Based on the evaluation scores, a directed connection relationship is established between the first shooting point and other shooting points in the surrounding shooting points.

[0065] The first shooting point mentioned here can be understood as the point that the current user is shooting or has just finished shooting. Whenever a new shooting point is added, a point mark will be added to the corresponding position in the directed graph, so that users can timely understand the current shooting progress and the distribution of each point. Each point has its own point information, including the coordinate information and ID value of the point. The ID value of each point is different. The surrounding shooting points with a larger ID value than the first shooting point are called parent node points, and the surrounding shooting points with a smaller ID value than the first shooting point are called child node points. When establishing a directed connection, the shooting point corresponding to the large ID value generally points to the shooting point corresponding to the small ID value, that is, the child node points to the parent node. These shooting points with directional connection relationships and the top view of the shooting scene (such as an indoor scene) together constitute a directed graph.

[0066] like Figure 2 Schematic diagram of a directed graph for illustrating an embodiment of the present application. Figure 2 As can be seen from the figure, there are multiple points in the directed graph, which are 30 to 40 points. Figure 2 It can be seen that point 40 is a point that has just been photographed or is being photographed. Point 39 does not establish a directed connection relationship with the parent node of the previous shooting point, which may be because the distance is too far, or because the overlap is too low, or the evaluation score is unqualified. The specific process of establishing a directed connection relationship will be described in the following embodiments, and you can refer to the following embodiments, so it will not be repeated here. Figure 2 The direction of the directed connection can be from a small ID value to a large ID value, that is, from a parent node to a child node.

[0067] When shooting a video of a house, in order to obtain a complete video of the house, it is necessary to shoot at multiple points in the house, and then perform image synthesis to obtain a panoramic image that can present the complete image information of the house. Therefore, the selection of shooting points is crucial during the shooting process. If the shooting position layout is unreasonable, the image synthesis effect will be poor, or the complete image cannot be synthesized.

[0068] It should be noted that the shooting scene mentioned here is inside a house (for example, a residential house, an office building, etc.). In fact, it can also be applied to many outdoor scenes, such as an outdoor maze, a children's playground, etc. In addition, since the overall shooting environment is relatively complex, in order to help users better understand the current shooting status, the location of each shooting point inside the house, whether the shooting point is reasonable, etc., the present application proposes to display the shooting points in a directed graph created based on the top view of the house, and establish a directed connection relationship between each shooting point. Mark each shooting point in the top view of the scene obtained by shooting. These shooting points are consistent with the actual shooting position. If they are inconsistent, the user can adjust them manually.

[0069] In order to meet the user's manual adjustment requirements for the first shooting point, the user can directly drag the shooting point in the displayed directed graph by hand. If it is displayed on the computer side, it can also be dragged by a mouse or other means. During the manual dragging process, the connection line with the first shooting point being dragged will automatically adjust with the position change. Specifically, when the position of the first shooting point changes, the position relationship between the first shooting point and the surrounding shooting points will be further determined. According to the adjusted position relationship, the directed connection relationship between the first shooting point and other points in the surrounding shooting points is updated. For example, after the first shooting point is manually moved to the left, the directed connection relationship with the right point will be disconnected, and a directed connection relationship will be established with the left point. In practical applications, the connection relationship can be updated in many ways, such as adding a new directed connection relationship, or reducing an existing directed connection relationship, or reducing an existing directed connection relationship while adding a new directed connection relationship. When updating the connection relationship, it is necessary to traverse and evaluate the parent node and child node corresponding to the first shooting point, and then comprehensively determine how to update the directed connection relationship between the first shooting point and other points.

[0070] Through the above method, the first shooting point after adjusting the directed connection relationship can accurately establish a directed connection with each point, so that the user can intuitively understand the position of each node in the directed graph, and whether a valid directed connection relationship is established between each node. If a first shooting point does not have a directed connection relationship, it means that the position of the first shooting point is not very reasonable and needs further adjustment, or a new shooting point is added. In addition, since the shooting unit is often marked after the shooting equipment automatically positions it in the directed graph, position deviations are inevitable. Therefore, in order to make the shooting point positioning more reasonable and accurate, the user is allowed to move it manually. It can effectively improve the generation efficiency and effect of the directed graph.

[0071] In one or more embodiments of the present application, after obtaining the first shooting point information when capturing an image of information inside a house at the first shooting point, the method includes: determining the first point ID corresponding to the first shooting point according to the shooting order of the shooting points; grouping the point IDs corresponding to the surrounding shooting points according to the first point ID to obtain a first point group and a second point group; wherein each point ID in the first point group is smaller than the first point ID, and each point ID in the second point group is larger than the first point ID.

[0072] In practical applications, in order to distinguish different shooting points, a unique ID value can be assigned to each shooting point. After shooting at the first shooting point, it is assumed that the assigned ID value is the first point ID. In order to facilitate the subsequent traversal of the positional relationship of the surrounding shooting points, a set or list of point IDs can be established. When establishing, grouping can be performed according to the first point ID. In other words, the surrounding points with larger ID values ​​than the first point are grouped into a group, which can be called the second point grouping here. The surrounding points with smaller ID values ​​than the first point are grouped into a group, which can be called the first point grouping here.

[0073] After the groups are created, the points contained in the groups can be traversed according to the groups. For example, the first point group can be traversed first, and then the second point group can be traversed. The traversal order of the groups can be adjusted as needed.

[0074] Assume that each point in the first point group and the second point group is stored in a list form, and in the list, the points are sorted according to the size of the ID value. When traversing, the points are traversed in order. The specific traversal calculation process will be described in the following embodiment, and will not be repeated here.

[0075] Generally speaking, the ID value corresponding to the shooting point that appears later is larger, and the ID value corresponding to the shooting point that appears earlier is smaller. Of course, in actual applications, users can selectively set the ID value according to their actual needs. It should be noted that the ID value of each shooting point is unique.

[0076] Through the above grouping method, the point traversal process is made simpler and clearer, and the management of the traversal process and results is convenient.

[0077] In one or more embodiments of the present application, in response to a user's manual movement instruction for the first shooting point, determining an evaluation score of a directed connection between the first shooting point and surrounding shooting points; and establishing a directed connection relationship between the first shooting point and other points in the surrounding shooting points based on the evaluation score, includes:

[0078] Based on the first point grouping, a candidate point list is established; the candidate point list includes point IDs;

[0079] In response to a manual movement instruction of the user for the first shooting point, selecting a first candidate point from the candidate point list;

[0080] Acquire the position information corresponding to the first candidate point, and calculate a first distance value between the first candidate point and the first shooting point; and,

[0081] Calculating a first degree of overlap using point cloud information of the first candidate point and the first shooting point;

[0082] Calculate the first evaluation score corresponding to each of the candidate points in the candidate point list corresponding to the first point group by using the first distance value and the first overlap degree;

[0083] A candidate point is selected from the candidate point list as a target shooting point for establishing a connection relationship with the first shooting point, so as to establish a directed connection relationship between the first shooting point and the target shooting point.

[0084] In actual applications, a unique, continuously increasing point ID is assigned to each shooting point in the order of the shooting plan, ensuring that the ID assignment is both logical and convenient for subsequent processing.

[0085] In an optional solution, the grouping principle is clear: The first point grouping includes all points that are located before the first shooting point in the shooting order. These points may have been recorded in the previous shooting process, or will be shot in the subsequent shooting according to the shooting plan but before the first shooting point. The second point grouping includes all points that are located after the first shooting point in the shooting order. These points need to be shot next or may be shot at a certain point in the future, or are points shot after the first shooting point. When grouping, group according to the size of the ID. Generally speaking, the ID value of the point shot first is smaller than the ID value of the point shot later.

[0086] After creating groups in the above way, you can use data structures (such as lists, arrays, or database tables) to store the IDs and related information of all shooting points. Traverse these data structures and assign the points to the corresponding groups according to the size of the point IDs.

[0087] As an option, you can further filter the candidate points when creating the list. Make sure the candidate points are visible from the first shooting point and are not blocked by obstacles. In addition, there are distance restrictions. According to the shooting requirements and equipment capabilities, set a reasonable distance range to exclude points that are too far or too close. For the perspective requirements, you can consider the perspective coherence and completeness of the panorama and select points that can supplement or enhance the current perspective.

[0088] When the list of candidate points is generated, based on the above screening conditions, points that meet the conditions are selected from the second point group as candidate points. If necessary, special points can also be selected as candidates in the first point group.

[0089] An intuitive and easy-to-use user interface can be designed as needed, allowing users to manually move the first shooting point by dragging, clicking, etc. The candidate point list and the currently selected candidate point are displayed in real time, and a preview function is provided.

[0090] When the user issues a move command, the system should immediately parse the command and determine the user's intention (for example, to move the first shooting point to a certain location). If the command is unclear or cannot be executed (such as moving to another unreachable point or moving a distance beyond the range of the house's overhead view), the system should give a corresponding prompt or error message.

[0091] In order to obtain accurate pose information, that is, the first shooting point information, sensors (such as gyroscopes, accelerometers) or SLAM (simultaneous localization and mapping) technology can be used to obtain the precise pose information of the first candidate point. If this information is not available or not accurate enough, other methods (such as manual measurement, map matching, etc.) can be considered for estimation.

[0092] According to the position information of the first shooting point and the first candidate point, the physical distance between them (which may be a straight-line distance, a feasible distance along a path, etc.) is calculated.

[0093] The overlap between two points is calculated using point cloud data or captured image data through algorithms (such as ICP, NDT, etc.). Furthermore, the quality of the overlapping area (such as clarity, texture richness, etc.) can also be evaluated to more accurately reflect the visual correlation between the two points. The specific calculation process of the overlap will be described in the following embodiments and will not be repeated here.

[0094] In addition, in practical applications, a comprehensive evaluation function is designed, which should take into account multiple factors such as distance and overlap (such as lighting conditions, occlusion, and shooting priority). An evaluation score is calculated for each candidate point, and the points are sorted or filtered according to the score.

[0095] After the comprehensive evaluation is completed, the candidate point with the highest evaluation score can be selected as the target shooting point. If there are multiple candidate points with similar scores and it is difficult to choose, you can consider using additional decision-making mechanisms (such as user confirmation, random selection, polling, etc.). A step further, a directed connection line can be established to record the directed connection relationship between the first shooting point and the target shooting point in the system. This includes information such as the starting point, end point, weight (possibly based on the evaluation score), direction, etc. of the connection. Update the scene graph to reflect the new connection relationship. Ensure that these updates are real-time and accurate.

[0096] In one or more embodiments of the present application, the step of calculating the first overlap degree using the point cloud information of the first candidate point and the first shooting point includes:

[0097] Acquire first point cloud information captured at the first candidate point;

[0098] Acquire second point cloud information obtained by photographing at the first photographing point;

[0099] Counting the number of point clouds that appear in the first point cloud information and the second point cloud information;

[0100] The ratio of the number of the point cloud and the number of the point cloud appearing in the second point cloud is calculated, and the ratio is used as the first overlap degree.

[0101] In practical applications, when calculating the first overlap using the point cloud information of the first candidate point and the first shooting point, it is necessary to ensure the accuracy of the point cloud data and the rationality of the processing method. The specific implementation process is as follows:

[0102] First, the point cloud data obtained by shooting the first candidate point and the first shooting point are preprocessed, including denoising, filtering, registration and other steps, so as to improve the quality of the point cloud data and facilitate the subsequent statistical overlap of the point cloud.

[0103] During registration, for example, the iterative closest point (ICP) algorithm or other point cloud registration algorithms can be used to register the point cloud of the first candidate point (first point cloud information) with the point cloud of the first shooting point (second point cloud information) so that they are in the same coordinate system for subsequent comparison and analysis. After the registration is completed, the overlapping area detection can be performed by comparing the points in the two point cloud data sets to find the point clouds that appear in both (i.e., overlapping point clouds). This usually involves calculating the distance from each point in the point cloud to all points in the other point cloud, and determining a threshold to determine whether it belongs to the overlapping area.

[0104] For example, point cloud data needs to be read from a storage medium (such as a file, a database, etc.). Point cloud data is usually stored in the form of files, such as PCD (Point Cloud Data), PLY (Polygon File Format) or LAS (Lidar Point Cloud Data) formats.

[0105] After reading, the point cloud data needs to be parsed into a suitable data structure, such as pcl::PointCloud in the Point Cloud Library (PCL) <pcl::pointxyz>Or other custom data structures. This structure should be able to store the 3D coordinates (X, Y, Z) of each point and other possible attributes (such as color, intensity, etc.).

[0106] After traversing the parsed point cloud data structure, count the number of points in it. This is usually a simple iterative process, implemented by accumulating the number of points. Identifying duplicate point clouds is a more advanced task in point cloud processing, which helps reduce data redundancy and improve the efficiency of subsequent processing. The following is a basic method for identifying duplicate point clouds:

[0107] The point cloud repetition can be understood as being judged based on the spatial distance between points, that is, if the distance between two points is less than a preset threshold, the two points are considered to be repeated.

[0108] Use spatial data structures: In order to efficiently identify duplicate point clouds, spatial data structures can be used to organize point cloud data, such as KD tree (K-dimensional tree), octree (Octree) or spatial grid (Spatial Grid). These data structures can speed up the search and comparison process between points.

[0109] Using the selected spatial data structure, a search operation is performed on each point to find nearby points. These points are then compared with the current point and marked as duplicates if the distance between them is less than a preset threshold.

[0110] Furthermore, the number of overlapping point clouds can be counted, and the ratio of this number to the total number of point clouds in the second point cloud can be calculated. This ratio is the first overlap degree, which reflects the overlap degree of visual information between two points.

[0111] By the above method, the overlap degree of the point cloud between the first shooting point and the first candidate point can be calculated, so as to evaluate the candidate point by using the calculated overlap degree.

[0112] In one or more embodiments of the present application, the use of the first distance value and the first overlap to calculate the evaluation score corresponding to each of the candidate points in the candidate point list corresponding to the first point group includes: obtaining the first coordinate of the first candidate point, and obtaining the second coordinate of the first shooting point; using the first coordinate and the second coordinate to calculate the first distance value; inputting the first distance value and the first overlap into a first calculation formula: score = 1 / (dist+1e -6 )+overlap; wherein score is the first evaluation score, distb represents the first distance value, and overlap represents the first repetition degree.

[0113] It should be noted that the first candidate point mentioned here is a first candidate point (that is, a parent node point) selected sequentially from the parent list. Before inputting the first distance value and the first overlap into the first calculation formula, it is necessary to first determine whether the first distance value and the first overlap meet the preset conditions. Specifically, determine whether the first distance value is greater than the distance threshold, and determine whether the first overlap is greater than the overlap threshold; if the judgment result is that the first distance value is greater than the distance threshold, and / or the first overlap is less than the overlap threshold, then the target shooting point is assigned a maximum value.

[0114] For example, the distance threshold can be set to 6 meters and the overlap threshold can be set to 0.25. If the judgment result is that the first distance value is greater than the distance threshold, it means that the distance is too far and cannot meet the panoramic image registration synthesis requirements. If the judgment result is that the first overlap is not greater than the overlap threshold, it means that the image content cannot meet the registration synthesis requirements due to the low overlap. In this case, in order to facilitate the subsequent adjustment of the candidate points, the ID value of the first candidate point can be temporarily assigned a maximum value, and it is not used as a candidate point for evaluation this time. For candidate points that do not meet the threshold conditions, a very large evaluation score (such as positive infinity) is assigned to ensure that they will not be selected in the subsequent selection process. In addition, there are other methods, such as directly removing these candidate points from the list to avoid unnecessary calculations. Here, it is not necessary to mark it as a maximum value so that the first candidate point can be re-evaluated later when needed. If it is removed from the list, it is a temporary removal method, such as transferring it to a substitute list so that the candidate point can be re-evaluated later when needed.

[0115] If the judgment result is that the first distance value is less than the distance threshold, and the first overlap is greater than the overlap threshold, it means that the distance of the first candidate point is appropriate, and the first overlap between the image of the first candidate point and the image of the first shooting point is greater than the overlap threshold, it means that the first candidate point meets the basic requirements and can participate in the score evaluation. Threshold judgment can help exclude candidate points that are too far away or have too low overlap, thereby improving the quality and efficiency of the panorama.

[0116] After the above threshold judgments for distance and overlap, the evaluation score of the first candidate point can be further calculated.

[0117] When calculating the evaluation score corresponding to each candidate point in the candidate point list, two factors, distance and overlap, need to be considered comprehensively.

[0118] When calculating the distance, the calculation is performed in the same coordinate system. The first distance value is calculated using the first coordinate of the first candidate point and the second coordinate of the first shooting point by the Euclidean distance formula. This distance value reflects the physical distance between the two points. In indoor scenes, the influence of walls, indoor doors, etc. on the calculated distance should be considered. Specifically, if the first candidate point and the first shooting point are located on both sides of the wall, when calculating the physical distance, the straight-line distance between the two points cannot be directly calculated, but the influence of the wall should be considered. If the first candidate point is located on both sides of the door, since the door is a moving object that can be opened and closed, when calculating the distance, it can be calculated according to the straight-line distance, and the influence of the door does not need to be considered.

[0119] Evaluation score calculation: Input the first distance value and the first overlap into the first calculation formula to calculate the evaluation score. However, please note that the formula score=1 / (dist+1e -6 )+overlap may need to be adjusted, because the overlap is usually a value between 0 and 1, and direct addition may cause the evaluation score to exceed the expected range. A more reasonable approach may be to multiply the overlap by a weight factor, or to normalize the distance term and overlap term separately before calculation.

[0120] As an alternative, the above formula can be optimized:

[0121] Assume that overlap is already a value between 0 and 1, dist is the distance value, and dist_max is the set maximum distance threshold.

[0122] dist_normalized = 1 / (dist+1e -6 ), and normalize the distance.

[0123] overlap_weight=0.5, where it is assumed that the weight of overlap is 0.5.

[0124] score=dist_normalized*(1-overlap_weight)+overlap*overlap_weight.

[0125] It should be noted that 1-overlap_weight here is the weight of the distance term, and its sum with the weight of the overlap is 1, ensuring the rationality of the evaluation score.

[0126] The candidate point list contains multiple candidate points. Existing candidate points are generally points where shooting has been completed. When the user shoots at a new shooting point (that is, the first shooting point mentioned above), it is necessary to evaluate whether the current shooting point is appropriate and reasonable. The appropriate and reasonable here refers to the convenience of subsequent image synthesis. In the actual evaluation, the distance between points and the overlap of point clouds are used for evaluation, and the rationality of the points can be comprehensively evaluated from the perspective of image content and image effect.

[0127] In one or more embodiments of the present application, in response to a user's manual movement instruction for the first shooting point, determining an evaluation score of a directed connection between the first shooting point and surrounding shooting points; and establishing a directed connection relationship between the first shooting point and other points in the surrounding shooting points based on the evaluation score, includes:

[0128] Based on the second point grouping, a candidate point list is established; the candidate point list includes point IDs;

[0129] In response to a user's manual movement instruction for the first shooting point, selecting a second candidate point from the candidate point list;

[0130] Acquire the posture information corresponding to the second candidate point, and calculate the second distance value between the second candidate point and the first shooting point; and,

[0131] Calculating a second degree of overlap using point cloud information of the second candidate point and the first shooting point;

[0132] Calculate the second evaluation score corresponding to each of the candidate points in the candidate point list corresponding to the first point group by using the second distance value and the second overlap degree;

[0133] A candidate point is selected from the candidate point list as a target shooting point for establishing a connection relationship with the first shooting point, so as to establish a directed connection relationship between the first shooting point and the target shooting point.

[0134] In addition to evaluating the first candidate point, the second candidate point also needs to be evaluated. The second candidate point, the ID value of the second candidate point is any one of the multiple points with a greater ID value than the first shooting point. The calculation method of the second evaluation score is similar to the calculation method of the first evaluation score, as follows:

[0135] Select a second candidate point child from the children list in sequence. Before inputting the second distance value and the second overlap into the second calculation formula, it is necessary to first determine whether the second distance value and the second overlap meet the preset conditions. Specifically, determine whether the second distance value is greater than the distance threshold, and determine whether the second overlap is greater than the overlap threshold; if the judgment result is that the second distance value is greater than the distance threshold, and / or the second overlap is less than the overlap threshold, then the target shooting point is assigned a maximum value.

[0136] For example, the distance threshold can be set to 6 meters and the overlap threshold can be set to 0.25. If the judgment result is that the second distance value is greater than the distance threshold, it means that the distance is too far and cannot meet the panoramic image registration synthesis requirements. If the judgment result is that the second overlap is not greater than the overlap threshold, it means that the image content cannot meet the registration synthesis requirements due to the low overlap. In this case, in order to facilitate the subsequent adjustment of the candidate points, the ID value of the second candidate point can be temporarily assigned a maximum value, and it is not used as a candidate point for evaluation this time. For candidate points that do not meet the threshold conditions, a very large evaluation score (such as positive infinity) is assigned to ensure that they will not be selected in the subsequent selection process. In addition, there are other methods, such as directly removing these candidate points from the list to avoid unnecessary calculations. Here, it is not necessary to mark it as a maximum value so that the second candidate point can be re-evaluated later when needed. If it is removed from the list, it is a temporary removal method, such as transferring it to a substitute list so that the candidate point can be re-evaluated later when needed.

[0137] If the judgment result is that the second distance value is less than the distance threshold, and the second overlap is greater than the overlap threshold, it means that the distance of the second candidate point is appropriate, and the second overlap between the image of the second candidate point and the image of the second shooting point is greater than the overlap threshold, it means that the second candidate point meets the basic requirements and can participate in the score evaluation. Threshold judgment can help exclude candidate points that are too far away or have too low overlap, thereby improving the quality and efficiency of the panorama.

[0138] After the above threshold judgments for distance and overlap, the evaluation score of the second candidate point can be further calculated.

[0139] When calculating the evaluation score corresponding to each candidate point in the candidate point list, two factors, distance and overlap, need to be considered comprehensively.

[0140] When calculating the distance, the calculation is performed in the same coordinate system. The second distance value is calculated using the second coordinates of the second candidate point and the second coordinates of the second shooting point using the Euclidean distance formula. This distance value reflects the physical distance between the two points. In indoor scenes, the influence of walls, indoor doors, etc. on the calculated distance should be considered. Specifically, if the second candidate point and the second shooting point are located on both sides of the wall, when calculating the physical distance, the straight-line distance between the two points cannot be directly calculated, but the influence of the wall should be considered. If the second candidate point is located on both sides of the door, since the door is a moving object that can be opened and closed, when calculating the distance, it can be calculated according to the straight-line distance, and the influence of the door does not need to be considered.

[0141] Evaluation score calculation: Input the second distance value and the second overlap into the second calculation formula to calculate the evaluation score. However, please note that the formula score=1 / (dist+1e -6 )+overlap may need to be adjusted, because the overlap is usually a value between 0 and 1, and direct addition may cause the evaluation score to exceed the expected range. A more reasonable approach may be to multiply the overlap by a weight factor, or to normalize the distance term and overlap term separately before calculation.

[0142] In one or more embodiments of the present application, if the parent node corresponding to the second candidate point is assigned a maximum value, the parent node corresponding to the second candidate point replaces the second candidate node. If the parent node corresponding to the second candidate point is not assigned a maximum value, the third evaluation score corresponding to the parent node corresponding to the second candidate point is calculated. And, if the second evaluation score is greater than the third evaluation score, the parent node corresponding to the second candidate point replaces the second candidate node.

[0143] First of all, it should be noted that the child node ID value is greater than the parent node ID value. In actual applications, as the first shooting point is manually moved, the relative position relationship between the various points will also change accordingly. This may cause the original directed connection relationship to be changed, for example, the original directed connection relationship is cancelled and a new directed connection relationship is established.

[0144] When modifying a directed connection relationship, there are two situations:

[0145] One is that the parent node corresponding to the child node is assigned a maximum value, that is, when judging the threshold, it is found that the parent node parent does not meet the requirements and cannot obtain the evaluation score between the parent node and the first shooting point, then the parent node is assigned a maximum value. And the parent node is replaced by the first shooting point that has been moved. In other words, the parent node with a directed connection relationship is reassigned to the candidate point.

[0146] Another is that if the parent node corresponding to the child node (the second candidate point mentioned above) is not assigned a maximum value, it is necessary to further calculate the third evaluation score between the second candidate point (also a child node) and the corresponding parent node. Compare the calculated third evaluation score with the previously calculated second evaluation score between the second candidate point and the first shooting point. The larger the evaluation score, the closer the association between the two points. If the second evaluation score is greater than the third evaluation score, it means that the directed connection relationship between the second candidate point and the original parent node is not the optimal relationship. At this time, the parent node corresponding to the second candidate point can be replaced by the mobile point (that is, the first shooting point). After the replacement, a directed connection relationship can be established directly according to the previously replaced parent node. Of course, it can also be recalculated and then decide how to establish the connection relationship. As long as a directed connection relationship is established, it is considered that the two points are reasonably aligned, and the images taken at these two points can be used for panoramic image generation.

[0147] If the second evaluation score is less than the third evaluation score, there is no need to replace the parent node corresponding to the second candidate point. When establishing a directed connection relationship for the first shooting point, a selection can be made based on the second evaluation scores of multiple second candidate points in the list. A second candidate point with a high second evaluation score is preferentially selected to establish a directed connection relationship with the first shooting point.

[0148] For ease of understanding, the following will be described by way of specific examples. Figure 3 The directed graph-based wiring scheme is illustrated in the embodiments of the present application.

[0149] When establishing directed lines for each point in a directed graph, you can connect them manually or automatically. If you use automatic connection, you can use the information from automatic registration to connect the point registered with the current first shooting point, establish a directed connection relationship between the two points, and make a directed graph based on the whole house top view.

[0150] If the automatic connection method is not used, the user can manually move the points to obtain the connection line. That is, the user can choose to drag one or more shooting points to the appropriate position as needed. The specific implementation process is as follows:

[0151] The existing points are grouped into two groups: one group is the child nodes whose point ID value is larger than the first shooting point ID value, and the other group is the parent nodes whose point ID value is smaller than the first shooting point ID value. The two groups are respectively created into group lists, and the points in each list are traversed.

[0152] First, 1) consider the directed connection relationship between the parent node point with a smaller ID value than the first shooting point moved manually and the first shooting point:

[0153] a) Get all parent information of the manually moved point (that is, the first shooting point), and create a first list of all parents. And add the first shooting point to the first list. Since the first shooting point is the latest shooting position, the ID value corresponding to the first shooting point is the maximum ID value, which can be marked as candi_1.

[0154] b) Traverse all the first shooting points in the first list (for example, candi_i represents one of them).

[0155] I) Select the point with a smaller ID value than the first candidate point as the parents.

[0156] II) Sequentially select a parent node from parents as a candidate point.

[0157] i. Filter points where the distance between parent and candi_i is too far (for example, 6 meters).

[0158] ii. Get the position information of parent and candi_i, solve the distance information, and solve the normal information.

[0159] iii. Calculate the overlap between parent and candi_i. For example, the kdtree algorithm can be used for searching, and the directional relationship between the first shooting point and the first candidate point can be considered.

[0160] iv. Score it, which can be calculated using the following formula: score = 1 / (dist+1e -6 )+overlap (the scoring mainly considers the distance and overlap. The closer the distance, the higher the score. The higher the overlap, the higher the score. Other forms of score calculation formulas are also acceptable. For details, please refer to the previous embodiment, which will not be repeated here).

[0161] v. Get the parent with the highest score. When its overlap is greater than the threshold (empirical value 0.25) and the distance is less than the threshold (empirical value 6m), return the ID value corresponding to the parent node with the highest score. If the overlap is less than the overlap threshold (for example, less than 0.25) and / or the distance is greater than the distance threshold (for example, 6 meters), return a maximum value, that is, assign a maximum value to the first candidate node, indicating that the first candidate node does not participate in the evaluation value calculation.

[0162] c) When the user manually moves multiple moving points at the same time, when candi_id connects to multiple parents, a selection is made based on the ID values ​​of the multiple moving points. Specifically, the point with the largest ID value corresponding to the first shooting point is selected to establish a directed connection relationship.

[0163] 2) Consider the directed connection relationship between the parent node point with a larger ID value than the first shooting point moved manually and the first shooting point:

[0164] a) Get the second candidate point children nodes (child_id represents one of them) with a larger ID value than the moving point (that is, the first shooting point). Add all these child nodes to the second list.

[0165] b) Traverse the children in the second list.

[0166] i) Select a child node from the second list as the second candidate point, and calculate the second evaluation score in the same way as in step 1) b) ii) above, that is, the second evaluation score of the first shooting point and the second candidate point.

[0167] ii) If the parent node parent corresponding to the second candidate point is a maximum value (that is, it is assigned a maximum value in step 1), the parent node parent corresponding to the second candidate point will be replaced by the first shooting point, because the first shooting point and the second candidate point can calculate a suitable evaluation score.

[0168] iii) If the parent node corresponding to the second candidate point is not assigned a maximum value, the evaluation score of the second candidate point and the corresponding parent node parent, that is, the third evaluation score, is calculated. If the second evaluation score is greater than the third evaluation score, the parent node of the second candidate point is replaced with the mobile point.

[0169] c) Similarly, when the user manually moves multiple moving points at the same time, when candi_id connects to multiple parents, a selection is made based on the ID values ​​of the multiple moving points. Specifically, the point with the largest ID value corresponding to the first shooting point is selected to establish a directed connection relationship.

[0170] like Figure 4 The following is a schematic diagram of the structure of a device for connecting shooting points based on a directed graph provided in an embodiment of the present application. Figure 4 As can be seen in the figure, the device includes:

[0171] The acquisition module 41 is used to acquire the first shooting point information when shooting an image inside the house at the first shooting point, display a directed graph containing the information of each shooting point in the house panoramic image, and establish directed connections between the points in the directed graph.

[0172] The determination module 42 is configured to determine the positional relationship between the first shooting point and surrounding shooting points in response to a manual movement instruction of the first shooting point by a user.

[0173] The connection establishing module 43 is used to establish a directed connection relationship between the first shooting point and other points in the surrounding shooting points based on the position relationship.

[0174] An acquisition module 41 is used to determine a first point ID corresponding to the first shooting point according to the shooting order of the shooting points;

[0175] According to the first point ID, the point IDs corresponding to the surrounding shooting points are grouped to obtain a first point group and a second point group; wherein each point ID in the first point group is smaller than the first point ID, and each point ID in the second point group is larger than the first point ID.

[0176] A judgment module 42 is used to establish a candidate point list based on the first point grouping; the candidate point list includes point IDs;

[0177] In response to a manual movement instruction of the user for the first shooting point, selecting a first candidate point from the candidate point list;

[0178] Acquire the position information corresponding to the first candidate point, and calculate a first distance value between the first candidate point and the first shooting point; and,

[0179] Calculating a first degree of overlap using point cloud information of the first candidate point and the first shooting point;

[0180] Calculate the first evaluation score corresponding to each of the candidate points in the candidate point list corresponding to the first point group by using the first distance value and the first overlap degree;

[0181] A candidate point is selected from the candidate point list as a target shooting point for establishing a connection relationship with the first shooting point, so as to establish a directed connection relationship between the first shooting point and the target shooting point.

[0182] A judgment module 42 is used to obtain first point cloud information photographed at the first candidate point;

[0183] Acquire second point cloud information obtained by photographing at the first photographing point;

[0184] Counting the number of point clouds that appear in the first point cloud information and the second point cloud information;

[0185] The ratio of the number of the point cloud and the number of the point cloud appearing in the second point cloud is calculated, and the ratio is used as the first overlap degree.

[0186] A judgment module 42 is used to obtain the first coordinates of the first candidate point, and to obtain the second coordinates of the first shooting point;

[0187] Calculate the first distance value using the first coordinate and the second coordinate;

[0188] The first distance value and the first overlap degree are input into a first calculation formula:

[0189] score = 1 / (dist+1e -6 )+overlap;

[0190] Among them, score is the first evaluation score, distb represents the first distance value, and overlap represents the first repetition degree.

[0191] A judging module 42, configured to judge whether the first distance value is greater than a distance threshold, and whether the first overlap degree is greater than an overlap degree threshold;

[0192] If the judgment result is that the first distance value is greater than the distance threshold, and / or the first overlap is less than the overlap threshold, a maximum value is assigned to the target shooting point.

[0193] A judgment module 42 is used to establish a candidate point list based on the second point grouping; the candidate point list includes point IDs;

[0194] In response to a user's manual movement instruction for the first shooting point, selecting a second candidate point from the candidate point list;

[0195] Acquire the posture information corresponding to the second candidate point, and calculate the second distance value between the second candidate point and the first shooting point; and,

[0196] Calculating a second degree of overlap using point cloud information of the second candidate point and the first shooting point;

[0197] Calculate the second evaluation score corresponding to each of the candidate points in the candidate point list corresponding to the first point group by using the second distance value and the second overlap degree;

[0198] A candidate point is selected from the candidate point list as a target shooting point for establishing a connection relationship with the first shooting point, so as to establish a directed connection relationship between the first shooting point and the target shooting point.

[0199] A judgment module 42 is configured to replace the second candidate node with the parent node corresponding to the second candidate point if the parent node corresponding to the second candidate point is assigned a maximum value;

[0200] If the parent node corresponding to the second candidate point is not assigned a maximum value, calculating a third evaluation score corresponding to the parent node corresponding to the second candidate point; and,

[0201] If the second evaluation score is greater than the third evaluation score, the second candidate node is replaced by the parent node corresponding to the second candidate point.

[0202] Figure 5 The structure diagram of an electronic device provided by an exemplary embodiment of the present application is shown in FIG. Figure 5 As shown, it includes: a memory 50a and a processor 50b; wherein the memory 50a is used to store a computer program; the processor 50b is coupled to the memory 50a, and is used to execute the computer program to perform the following steps:

[0203] Acquire the first shooting point information when shooting an image in the house at the first shooting point, display a directed graph containing the information of each shooting point in the panoramic image of the house, and establish a directed connection between each point in the directed graph;

[0204] In response to a manual movement instruction of the user to the first shooting point, determining an evaluation score of a directed connection between the first shooting point and surrounding shooting points;

[0205] Based on the evaluation scores, a directed connection relationship is established between the first shooting point and other shooting points in the surrounding shooting points.

[0206] The processor 50b is used to determine the first point ID corresponding to the first shooting point according to the shooting order of the shooting points;

[0207] According to the first point ID, the point IDs corresponding to the surrounding shooting points are grouped to obtain a first point group and a second point group; wherein each point ID in the first point group is smaller than the first point ID, and each point ID in the second point group is larger than the first point ID.

[0208] The processor 50b is used to establish a candidate point list based on the first point grouping; the candidate point list includes point IDs;

[0209] In response to a manual movement instruction of the user for the first shooting point, selecting a first candidate point from the candidate point list;

[0210] Acquire the position information corresponding to the first candidate point, and calculate a first distance value between the first candidate point and the first shooting point; and,

[0211] Calculating a first degree of overlap using point cloud information of the first candidate point and the first shooting point;

[0212] Calculate the first evaluation score corresponding to each of the candidate points in the candidate point list corresponding to the first point group by using the first distance value and the first overlap degree;

[0213] A candidate point is selected from the candidate point list as a target shooting point for establishing a connection relationship with the first shooting point, so as to establish a directed connection relationship between the first shooting point and the target shooting point.

[0214] The processor 50b is used to obtain first point cloud information photographed at the first candidate point;

[0215] Acquire second point cloud information obtained by photographing at the first photographing point;

[0216] Counting the number of point clouds that appear in the first point cloud information and the second point cloud information;

[0217] The ratio of the number of the point cloud and the number of the point cloud appearing in the second point cloud is calculated, and the ratio is used as the first overlap degree.

[0218] The processor 50b is used to obtain the first coordinates of the first candidate point, and obtain the second coordinates of the first shooting point;

[0219] Calculate the first distance value using the first coordinate and the second coordinate;

[0220] The first distance value and the first overlap degree are input into a first calculation formula:

[0221] score = 1 / (dist+1e -6 )+overlap;

[0222] Among them, score is the first evaluation score, distb represents the first distance value, and overlap represents the first repetition degree.

[0223] The processor 50b is used to determine whether the first distance value is greater than a distance threshold, and determine whether the first overlap degree is greater than an overlap degree threshold;

[0224] If the judgment result is that the first distance value is greater than the distance threshold, and / or the first overlap is less than the overlap threshold, a maximum value is assigned to the target shooting point.

[0225] The processor 50b is used to establish a candidate point list based on the second point grouping; the candidate point list includes point IDs;

[0226] In response to a user's manual movement instruction for the first shooting point, selecting a second candidate point from the candidate point list;

[0227] Acquire the posture information corresponding to the second candidate point, and calculate the second distance value between the second candidate point and the first shooting point; and,

[0228] Calculating a second degree of overlap using point cloud information of the second candidate point and the first shooting point;

[0229] Calculate the second evaluation score corresponding to each of the candidate points in the candidate point list corresponding to the first point group by using the second distance value and the second overlap degree;

[0230] A candidate point is selected from the candidate point list as a target shooting point for establishing a connection relationship with the first shooting point, so as to establish a directed connection relationship between the first shooting point and the target shooting point.

[0231] The processor 50b is configured to replace the second candidate node with the parent node corresponding to the second candidate point if the parent node corresponding to the second candidate point is assigned a maximum value;

[0232] If the parent node corresponding to the second candidate point is not assigned a maximum value, calculating a third evaluation score corresponding to the parent node corresponding to the second candidate point; and,

[0233] If the second evaluation score is greater than the third evaluation score, the second candidate node is replaced by the parent node corresponding to the second candidate point.

[0234] It should be noted that the specific implementation of the above modules or units provided in this embodiment can refer to the above Figure 1 The relevant description of the illustrated method embodiment will not be repeated here.

[0235] An exemplary embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, causes one or more processors to execute the steps in the above-mentioned shooting point prediction method. The detailed implementation process of each step can be found in the relevant description of the above-mentioned embodiments, and will not be repeated here.

[0236] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0237] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0238] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0239] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0240] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0241] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0242] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0243] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the existence of other identical elements in the process, method, commodity or device including the element.

[0244] The above is only the embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.< / pcl::pointxyz>

Claims

1. A method for connecting shooting points based on a directed graph, characterized in that: The method comprises: Acquire the first shooting point information when shooting an image in the house at the first shooting point, display a directed graph containing the information of each shooting point in the panoramic image of the house, and establish a directed connection between each point in the directed graph; In response to a manual movement instruction of the user to the first shooting point, determining an evaluation score of a directed connection between the first shooting point and surrounding shooting points; Based on the evaluation scores, a directed connection relationship is established between the first shooting point and other shooting points in the surrounding shooting points.

2. The method according to claim 1, characterized in that After obtaining the first shooting point information when shooting the image of the information inside the house at the first shooting point, the method includes: Determine, according to the shooting order of the shooting points, a first point ID corresponding to the first shooting point; According to the first point ID, the point IDs corresponding to the surrounding shooting points are grouped to obtain a first point group and a second point group; wherein each point ID in the first point group is smaller than the first point ID, and each point ID in the second point group is larger than the first point ID.

3. The method according to claim 1, characterized in that: In response to a manual movement instruction of the first shooting point by a user, determining a positional relationship between the first shooting point and surrounding shooting points; Based on the positional relationship, establishing a directed connection relationship between the first shooting point and other points in the surrounding shooting points includes: Based on the first point grouping, a candidate point list is established; the candidate point list includes point IDs; In response to a manual movement instruction of the user for the first shooting point, selecting a first candidate point from the candidate point list; Acquire the position information corresponding to the first candidate point, and calculate a first distance value between the first candidate point and the first shooting point; and, Calculating a first degree of overlap using point cloud information of the first candidate point and the first shooting point; Calculate the first evaluation score corresponding to each of the candidate points in the candidate point list corresponding to the first point group by using the first distance value and the first overlap degree; A candidate point is selected from the candidate point list as a target shooting point for establishing a connection relationship with the first shooting point, so as to establish a directed connection relationship between the first shooting point and the target shooting point.

4. The method according to claim 3, characterized in that The calculating the first overlap degree by using the point cloud information of the first candidate point and the first shooting point includes: Acquire first point cloud information captured at the first candidate point; Acquire second point cloud information obtained by photographing at the first photographing point; Counting the number of point clouds that appear in the first point cloud information and the second point cloud information; The ratio of the number of the point cloud and the number of the point cloud appearing in the second point cloud is calculated, and the ratio is used as the first overlap degree.

5. The method according to claim 1, characterized in that The step of calculating the evaluation scores corresponding to the respective candidate points in the candidate point list corresponding to the first point group by using the first distance value and the first overlap degree comprises: Acquire a first coordinate of the first candidate point, and acquire a second coordinate of the first shooting point; Calculate the first distance value using the first coordinate and the second coordinate; The first distance value and the first overlap degree are input into a first calculation formula: score=1 / (dist+1e -6 )+overlap; Among them, score is the first evaluation score, distb represents the first distance value, and overlap represents the first repetition degree.

6. The method according to claim 5, characterized in that Before inputting the first distance value and the first overlap degree into the first calculation formula, the method further includes: Determining whether the first distance value is greater than a distance threshold, and determining whether the first overlap degree is greater than an overlap degree threshold; If the judgment result is that the first distance value is greater than the distance threshold, and / or the first overlap is less than the overlap threshold, a maximum value is assigned to the target shooting point.

7. The method according to claim 2 or 5, characterized in that: In response to a manual movement instruction of the first shooting point by a user, determining a positional relationship between the first shooting point and surrounding shooting points; Based on the positional relationship, establishing a directed connection relationship between the first shooting point and other points in the surrounding shooting points includes: Based on the second point grouping, a candidate point list is established; the candidate point list includes point IDs; In response to a manual movement instruction of the user for the first shooting point, selecting a second candidate point from the candidate point list; Acquire the position information corresponding to the second candidate point, and calculate the second distance value between the second candidate point and the first shooting point; and, Calculating a second degree of overlap using point cloud information of the second candidate point and the first shooting point; Calculate the second evaluation score corresponding to each of the candidate points in the candidate point list corresponding to the first point group by using the second distance value and the second overlap degree; A candidate point is selected from the candidate point list as a target shooting point for establishing a connection relationship with the first shooting point, so as to establish a directed connection relationship between the first shooting point and the target shooting point.

8. The method according to claim 7, characterized in that Also includes: If the parent node corresponding to the second candidate point is assigned a maximum value, the parent node corresponding to the second candidate point replaces the second candidate node; If the parent node corresponding to the second candidate point is not assigned a maximum value, calculating a third evaluation score corresponding to the parent node corresponding to the second candidate point; and, If the second evaluation score is greater than the third evaluation score, the second candidate node is replaced by the parent node corresponding to the second candidate point.

9. A shooting point connection device based on a directed graph, characterized in that: The device comprises: An acquisition module, used to acquire first shooting point information when shooting an image in a house at the first shooting point, display a directed graph containing information of each shooting point in a panoramic image of the house, and establish directed connections between each point in the directed graph; a determination module, configured to determine, in response to a user's manual movement instruction for the first shooting point, an evaluation score of a directed connection between the first shooting point and surrounding shooting points; A connection establishing module is used to establish a directed connection relationship between the first shooting point and other points in the surrounding shooting points based on the size of the evaluation score.

10. An electronic device, characterized in that: include: A memory and a processor; wherein the memory is used to store a computer program; and the processor is coupled to the memory and is used to execute the computer program to execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the method according to any one of claims 1 to 8.

Images