Space shooting missing detection method and device, electronic equipment and storage medium

Abstract

The invention provides a space shooting missing detection method and device, electronic equipment and a storage medium, and the method comprises the steps: determining a first set comprising a target prediction point location based on a shot point location and a prediction point location of a target space object, the prediction point locations which belong to the door point location and the open space point location in the prediction point locations which do not participate in shooting are target prediction point locations; for each target prediction point location, identifying a target shooting point location of which the position relationship with the target prediction point location meets a preset condition from the shot point locations, and determining a second set; traversing a target shooting point location in a second set corresponding to the target prediction point location, and identifying whether the target prediction point location is associated with a missing space or not according to the intersection condition of a target line segment corresponding to the target shooting point location and the closed space; and determining a space missed shooting detection result according to the missed space identification condition associated with each target prediction point location in the first set. According to the invention, real shooting data can be provided through missed shooting detection, and the shooting quality is ensured.

Description

Technical Field

[0001] The present application relates to the field of computer data processing technology, and in particular to a spatial missed shot detection method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of VR (Virtual Reality) technology, panoramas are widely used in house rental, hotel homestay, home decoration, various exhibitions and other fields. Compared with ordinary single-perspective images, panoramas can provide broader field of view information, allowing browsing users to restore the scene to a certain extent. Therefore, panorama previews are becoming more and more popular and sought after by the public.

[0003] In the fields of 3D reconstruction and VR, it is common to use panoramic data for subsequent task construction. However, in the process of selecting shooting points, different photographers have different ideas, which may lead to some rooms being missed or forgotten. In the subsequent reconstruction, it will be found that the virtual construction is different from the actual space, which loses authenticity and may not be accepted by the majority of users. This not only wastes manpower and material resources, but also fails to meet user needs.

[0004] Based on this, in order to ensure authenticity, it is necessary to perform missed shot detection after shooting is completed to ensure that real data is provided and thus meet the browsing needs of users. Summary of the invention

[0005] The embodiments of the present application provide a spatial missed shot detection method, device, electronic device and storage medium to solve the problem in the prior art that data loses authenticity and cannot meet user browsing needs due to spatial missed shots.

[0006] In a first aspect, an embodiment of the present application provides a spatial missed shot detection method, comprising:

[0007] In the case of acquiring the photographed points and predicted points of the target space object, determining a first set including at least one target predicted point based on the photographed points and the predicted points, wherein the predicted points belonging to the door points and the open space points among the predicted points not involved in the photographing are all target predicted points;

[0008] For each target prediction point, identify a target shooting point whose positional relationship with the target prediction point satisfies a preset condition among the shot points of the target space object, and determine a second set corresponding to the target prediction point;

[0009] Traversing the target shooting points in the second set corresponding to the target prediction point, and identifying whether the target prediction point is associated with a missing space in the target space object according to the intersection of the target line segment corresponding to the target shooting point and the closed space corresponding to the target shooting point;

[0010] Determine a spatial missed shot detection result according to the missed space identification status associated with each target prediction point in the first set;

[0011] The target line segment corresponding to the target shooting point is determined based on the target shooting point and the target predicted point, and the closed space corresponding to the target shooting point is constructed based on point cloud data corresponding to the picture content captured at the target shooting point.

[0012] In a second aspect, an embodiment of the present application provides a spatial missed shot detection device, comprising:

[0013] A first determination module is used to determine, when the photographed points and predicted points of the target space object are acquired, a first set including at least one target predicted point based on the photographed points and the predicted points, wherein the predicted points belonging to the door points and the open space points among the predicted points not involved in the photographing are all target predicted points;

[0014] an identification and determination module, configured to identify, for each target prediction point, a target shooting point whose positional relationship with the target prediction point satisfies a preset condition among the shot points of the target space object, and determine a second set corresponding to the target prediction point;

[0015] a processing module, configured to traverse the target shooting points in the second set corresponding to the target prediction point, and identify whether the target prediction point is associated with a missing space in the target space object according to the intersection of the target line segment corresponding to the target shooting point and the closed space corresponding to the target shooting point;

[0016] A second determination module is used to determine a spatial missed shot detection result according to the missed space recognition status associated with each target prediction point in the first set;

[0017] The target line segment corresponding to the target shooting point is determined based on the target shooting point and the target predicted point, and the closed space corresponding to the target shooting point is constructed based on point cloud data corresponding to the picture content captured at the target shooting point.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the spatial missed beat detection method as described in the first aspect above.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the spatial missed beat detection method described in the first aspect above are implemented.

[0020] The technical solution of the embodiment of the present application, when obtaining the photographed points and predicted points of the target space object, determines a first set including at least one target predicted point based on the photographed points and the predicted points, determines a second set including associated target shooting points for each target predicted point in the first set, traverses the target shooting points in the second set corresponding to the target predicted points, and identifies whether the target predicted point is associated with a missing space in the target space object based on the intersection of the target line segment corresponding to each target shooting point and the closed space corresponding to the target shooting point, determines a space missed shot detection result based on the missed space identification associated with each target prediction point, so as to detect whether the space is missed in the process of shooting the target space object, and can provide real and effective shooting data through missed shot detection, ensure shooting quality, reduce unnecessary waste as much as possible, and improve user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram showing a spatial missed shot detection method provided in an embodiment of the present application;

[0022] Figure 2 A specific implementation flow chart of the spatial missed shot detection method provided in an embodiment of the present application is shown;

[0023] Figure 3 A schematic diagram showing a spatial missed shot detection device provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. The multiple in the embodiments of the present application can include two and more than two.

[0027] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0028] The embodiments of the present application provide a solution to the problem that the authenticity of the data is damaged and the user needs cannot be met due to missing spaces during panoramic shooting. After the user has shot all the points, the processing result of whether there are any missed rooms is obtained based on the prior information of the shot points (such as predicted points, door points, open spaces, point cloud data, etc.), and the locations of the missed rooms are indicated, which can ensure the shooting quality, minimize unnecessary waste, and improve user satisfaction.

[0029] The following is an introduction to the space missed shot detection method provided in the embodiment of the present application. Missed shot space refers to a space that should be photographed but was not photographed, which is an independent space or an open space. In a panoramic shooting scenario, after the user selects a point to shoot, one or more predicted points associated with the shooting point are determined by calculation, and the user can continue to shoot at the predicted point. After shooting at the predicted point, the predicted point is used as the photographed point, and at least one predicted point associated with the photographed point is determined by calculation. The user continues to select a predicted point to shoot, and so on, to complete the panoramic shooting of the entire space to be shot (such as the house to be shot). It should be noted that a certain shooting point may not have a corresponding predicted point, such as the predicted point of the shooting point cannot be calculated.

[0030] Among them, the predicted points are usually at the door points, and there are also predicted points inside and outside the open space. Since the missed space is an independent space or an open space, missed detection can be performed based on the door points and the open space. During the detection, the information of the predicted points and the information of the photographed points need to be used.

[0031] like Figure 1 As shown, the spatial missed shot detection method provided in the embodiment of the present application includes the following steps:

[0032] Step 101: When obtaining photographed points and predicted points of the target space object, determine a first set including at least one target predicted point based on the photographed points and the predicted points, wherein the predicted points belonging to door points and open space points among the predicted points not involved in the photographing are all target predicted points.

[0033] After the user completes shooting for the target space object (such as the target house object), the photographed point corresponding to the target space object is obtained. Since the associated predicted point can be calculated for each shooting point, when the photographed point of the target space object is determined, the predicted point of the target space object is a known point. After obtaining the photographed points and predicted points of the target space object, a first set including at least one target predicted point is determined based on the comparison of the photographed points and the predicted points. In this embodiment, the target predicted point is determined among the missed predicted points that did not participate in the shooting, and the predicted points belonging to the door points and the open space points among the predicted points that did not participate in the shooting are all target predicted points. The door point can be understood as a predicted point caused by the door point, and the open space point can be understood as a predicted point caused by the open space.

[0034] By determining a first set including at least one target prediction point, the gate points and open space points in the missed prediction points that are not involved in the shooting can be aggregated, so as to facilitate missed space detection using the points in the first set.

[0035] Step 102: for each target prediction point, identify target shooting points whose positional relationship with the target prediction point satisfies a preset condition from among the shot points of the target space object, and determine a second set corresponding to the target prediction point.

[0036] For each target prediction point in the first set, traverse the already photographed points corresponding to the target space object, identify the photographing points whose positional relationship with the current target prediction point meets the preset conditions among the already photographed points corresponding to the target space object, determine the identified photographing points as the target photographing points that match the current target prediction point, and determine the second set corresponding to the current target prediction point based on the target photographing points that match the current target prediction point.

[0037] For each target prediction point in the first set, there may be a corresponding second set, which is used to store target shooting points whose positional relationship with the target prediction point satisfies the preset conditions; if there is no target shooting point whose positional relationship with the target prediction point satisfies the preset conditions, the second set is an empty set.

[0038] Step 103, traverse the target shooting points in the second set corresponding to the target prediction point, and identify whether the target prediction point is associated with the missing space in the target space object according to the intersection of the target line segment corresponding to the target shooting point and the closed space corresponding to the target shooting point; the target line segment corresponding to the target shooting point is determined based on the target shooting point and the target prediction point, and the closed space corresponding to the target shooting point is constructed based on the point cloud data corresponding to the picture content captured at the target shooting point.

[0039] Since each target prediction point in the first set corresponds to a second set, for each target prediction point, the target shooting points in the corresponding second set are traversed, and the intersection between the target line segment corresponding to the target shooting point and the closed space corresponding to the target shooting point is detected, and based on the intersection, it is identified whether the target prediction point is associated with the missing space in the target space object.

[0040] Among them, the target line segment corresponding to the target shooting point is determined based on the target shooting point and the associated target prediction point, and the connecting line between the target shooting point and the associated target prediction point is the target line segment corresponding to the target shooting point; the closed space corresponding to the target shooting point is constructed based on the point cloud data corresponding to the picture content captured at the target shooting point.

[0041] If the target line segment corresponding to the target shooting point intersects with the closed space corresponding to the target shooting point, it indicates that the target prediction point is outside the closed space formed; if the target line segment corresponding to the target shooting point does not intersect with the closed space corresponding to the target shooting point, it indicates that the target prediction point is inside the closed space formed. After traversing the target shooting points in the second set corresponding to the target prediction point and detecting the intersection of the target line segment corresponding to the target shooting point and the closed space corresponding to the target shooting point, it is identified whether the target prediction point is associated with the missing space in the target space object according to the intersection corresponding to each detected target shooting point, so as to determine the association between the current target prediction point and the missing space.

[0042] Step 104: Determine the spatial missed shot detection result according to the missed space identification status associated with each target prediction point in the first set.

[0043] For each target prediction point in the first set, after determining the missing space recognition status associated with each target prediction point, the space missed detection result is determined according to the recognition status to detect whether the space is missed in the process of shooting the target space object.

[0044] The above implementation scheme of the present application, when obtaining the photographed points and predicted points of the target space object, determines a first set including at least one target predicted point based on the photographed points and the predicted points, determines a second set including associated target shooting points for each target predicted point in the first set, traverses the target shooting points in the second set corresponding to the target predicted points, and identifies whether the target predicted point is associated with a missing space in the target space object based on the intersection of the target line segment corresponding to each target shooting point and the closed space corresponding to the target shooting point, determines a space missed shot detection result based on the missed space identification associated with each target prediction point, so as to detect whether the space is missed in the process of shooting the target space object, and can provide real and effective shooting data through missed shot detection, ensure shooting quality, reduce unnecessary waste as much as possible, and improve user satisfaction.

[0045] The process of determining the first set is introduced below. When determining the first set including at least one target predicted point based on the photographed points and the predicted points, it includes:

[0046] The photographed points are compared with the predicted points corresponding to the target space objects to identify the predicted points that are not involved in the shooting; among the predicted points that are not involved in the shooting, the predicted points that belong to the door points and the predicted points that belong to the open space are identified, and the first set is determined based on the identified predicted points.

[0047] Since the predicted points of the target space object are determined when the photographed points of the target space object are determined, the photographed points and the predicted points corresponding to the target space object can be compared to identify the predicted points that are not involved in the shooting. For example, after the target space object is photographed at shooting point 1, the predicted points A and B associated with shooting point 1 are calculated, and predicted point A is used as shooting point 2. After the target space object is photographed at shooting point 2, the predicted point C associated with shooting point 2 is calculated, and predicted point C is used as shooting point 3. After the target space object is photographed at shooting point 3, the shooting is completed. Shooting point 1, shooting point 2, and shooting point 3 are compared with predicted point A, predicted point B, and predicted point C, and the unoccupied predicted point B can be identified, which belongs to the predicted point that is not involved in the shooting.

[0048] The identified predicted points that did not participate in the shooting may include predicted points belonging to door points, predicted points belonging to open space points, and other predicted points. Since the missed space is an independent space or an open space, it is necessary to identify the predicted points belonging to door points and the predicted points belonging to open space points among the predicted points that did not participate in the shooting, and then determine the first set based on the identified predicted points, and then determine the predicted point set for missed space detection.

[0049] During the above implementation process, the predicted points that did not participate in the shooting are identified based on the comparison between the photographed points and the predicted points, and the predicted points belonging to the gate points and the predicted points belonging to the open space points are further screened out from the identified predicted points. The gate points and the open space points are retained to obtain the predicted points for missed space detection.

[0050] In an optional embodiment, for each target prediction point in the first set, a target shooting point whose positional relationship with the target prediction point satisfies a preset condition is identified from the shot points of the target space object, and the second set corresponding to the target prediction point is determined, including:

[0051] For each target prediction point in the first set, a target shooting point whose distance to the target prediction point is less than a preset threshold is identified among the shot points of the target space object; in response to identifying at least one target shooting point, a second set corresponding to the target prediction point and including at least one target shooting point is constructed; wherein the distance between the shot point and the target prediction point is determined based on the world coordinate information of the point.

[0052] The world coordinate information corresponding to the shooting point and the predicted point is obtained based on the local coordinate information of the point. For the shooting point, the coordinate information corresponding to its own coordinate system belongs to the local coordinate information. For the predicted point, the coordinate information in the coordinate system corresponding to the associated shooting point belongs to the local coordinate information. Each shooting point corresponds to its own coordinate system (also called the local coordinate system). After the world coordinate information of the shooting point and the predicted point is obtained based on the coordinate conversion, the distance between the points can be calculated based on the world coordinate information.

[0053] Traverse the target prediction points in the first set, and for each target prediction point, select the shooting points whose distance from the current target prediction point is less than a preset threshold (such as an empirical value of 2.5m) from the already shot points of the target space object, and use the selected shooting points as the target shooting points. If at least one target shooting point associated with the current target prediction point is selected, determine the second set corresponding to the current target prediction point based on the at least one selected target shooting point, so as to aggregate the already shot points whose distance from the current target prediction point is less than the preset threshold. After determining the second set corresponding to the target prediction point, the association between the target prediction point and the missing space in the target space object can be identified based on the target prediction point and the target shooting points in the second set, and then determine the spatial missed shooting detection result for the target space object according to the identification situation corresponding to each target prediction point.

[0054] Among them, for any target prediction point, in response to the target shooting point whose distance to the target prediction point is less than the preset threshold not being identified, the missing space in the target space object associated with the target prediction point is determined. That is, if the target shooting point whose distance to the current target prediction point is less than the preset threshold is not identified, it can be regarded that the distance between the current target prediction point and each shot point is large, the second set is an empty set, and the current target prediction point belongs to an isolated point, which corresponds to an independent space or an open space, and then the missing space in the target space object associated with the current target prediction point is determined.

[0055] During the above implementation process, for each target prediction point in the first set, an associated target shooting point is identified among the photographed points. If an associated target shooting point is identified, a second set corresponding to the target prediction point is constructed to aggregate the photographed points whose distance to the target prediction point is less than a preset threshold, so as to provide a basis for identifying missed spaces. If an associated target shooting point is not identified, it is determined that the target prediction point is an isolated point, thereby determining the missed space in the target space object associated with the target prediction point.

[0056] In another embodiment of the present application, the method further includes:

[0057] Identify the door body objects in the target space object, and construct corresponding door body point cloud data based on the door point information of each door body object, wherein the door body objects include predicted door objects and real door objects;

[0058] For each target shooting point in the second set corresponding to the target prediction point, the target line segment corresponding to the target shooting point is determined based on the line connecting the target shooting point and the target prediction point, and the closed space corresponding to the target shooting point is constructed based on the point cloud data corresponding to the picture content captured at the target shooting point and the matched door body point cloud data.

[0059] In this embodiment, for the target space object, it is necessary to identify the door body object included therein, and the door body object includes a predicted door object and a real door object. The predicted door object is, for example, a boundary line object in an open space. For example, the restaurant space and the living room space in the target house object constitute an open space, and the boundary line between the restaurant space and the living room space corresponds to a predicted door object (also called a virtual door object). For the identified door body object, the door body point cloud data corresponding to the door body object is constructed based on the door point information of each door body object as the data of the closed wall. For example, a plane perpendicular to the ground is constructed based on the two points at the door point, and the height is similar to that of the door.

[0060] For each target prediction point, in the corresponding second set, the target line segment corresponding to each target shooting point is determined based on the line connecting each target shooting point and the target prediction point; and for each target shooting point, after determining the door body object matched by the current target shooting point, the point cloud data corresponding to the image content captured at the target shooting point and the door body point cloud data matched by the target shooting point are merged to construct the closed space corresponding to the target shooting point. Then, the intersection of the target line segment corresponding to the target shooting point and the closed space corresponding to the target shooting point can be detected.

[0061] Wherein, for the target prediction point, traversing the target shooting points in the second set, and identifying whether the target prediction point is associated with the missing space in the target space object according to the intersection of the target line segment corresponding to the target shooting point and the closed space corresponding to the target shooting point, including:

[0062] sequentially detecting each target shooting point in the second set corresponding to the target prediction point, and identifying whether the target line segment corresponding to the target shooting point intersects with the closed space corresponding to the target shooting point;

[0063] When detecting a target shooting point position where the target line segment does not intersect with the corresponding closed space, determining that the target prediction point position is not associated with the missing space in the target space object;

[0064] When it is detected that the target line segment corresponding to each target shooting point in the second set intersects with the corresponding closed space, it is determined that the target prediction point is associated with the missing space in the target space object.

[0065] For any target prediction point, traverse the target shooting points in the corresponding second set, and detect whether the target line segment corresponding to the line connecting the target shooting point and the target prediction point intersects with the closed space corresponding to the target shooting point. If the target line segment corresponding to the target shooting point intersects with the closed space corresponding to the target shooting point, it indicates that the target prediction point is outside the formed closed space; if the target line segment corresponding to the target shooting point does not intersect with the closed space corresponding to the target shooting point, it indicates that the target prediction point is inside the formed closed space.

[0066] When a target shooting point is detected where the target line segment does not intersect with the corresponding closed space, the target prediction point is determined to be inside the closed space corresponding to the target shooting point, and then the target prediction point is determined to be not associated with the missing space in the target space object. If after completing the traversal of the target shooting points in the second set, it is detected that the target line segment corresponding to each target shooting point intersects with the corresponding closed space, it indicates that the target prediction point is outside the closed space corresponding to each target shooting point, and then the target prediction point is determined to be associated with the missing space in the target space object.

[0067] During the above implementation process, after detecting the door body object in the target space object, the door body point cloud data of the door body object is constructed, and the point cloud data corresponding to the target shooting point and the matched door body point cloud data are merged to construct the closed space corresponding to the target shooting point. Then, based on the intersection of the target line segment corresponding to the target shooting point and the closed space, it is identified whether the target prediction point is associated with the missing space. The relationship between the line segment and the closed space contour can be identified based on the intersection of the point line segment and the point cloud data, thereby determining the association between the target prediction point and the missing space.

[0068] In an optional implementation, after identifying the association between the target prediction point and the missing space for each target prediction point in the first set, determining the spatial missed shot detection result according to the missing space identification associated with each target prediction point in the first set includes:

[0069] Recording point information corresponding to the target prediction point of the missing space in the associated target space object in the first set, where the point information at least includes position information and subordinate shooting points;

[0070] The spatial missed shot detection result corresponding to the target spatial object is determined according to the recording result and output, and the spatial missed shot detection result is used to indicate the spatial missed shot situation of the target spatial object.

[0071] After identifying the target prediction point of the missing space in the associated target space object in the first set, the point information corresponding to the identified target prediction point is recorded to store the position information of the identified target prediction point and the shooting point to which it belongs. Based on the recorded position information of the target prediction point and the shooting point to which it belongs, a spatial missed shot detection result corresponding to the target space object is generated and output. The output spatial missed shot detection result is used to indicate the spatial missed shot situation of the target space object, so that the user can determine at which prediction point the shooting is missed, and the shooting point to which the prediction point belongs, based on the spatial missed shot detection result, and then make up for the missed shot to provide real shooting data of the target space object.

[0072] The following is an introduction to the spatial missed shot detection method provided by the embodiment of the present application through an overall implementation process. Figure 2 As shown, the method comprises the following steps:

[0073] Step 201: Obtain the photographed points and predicted points of the target space object.

[0074] Step 202: compare the photographed points with the predicted points, identify the predicted points that are not involved in the shooting, among the predicted points that are not involved in the shooting, identify the predicted points that are door points and open space points, and determine a first set including at least one target predicted point based on the identified predicted points.

[0075] Step 203: Identify door body objects in the target space object, and construct corresponding door body point cloud data based on the door point information of each door body object.

[0076] Step 204 : for the target predicted points in the first set, identify the target photographed points whose distances to the target predicted points are less than a preset threshold from the photographed points, and then execute step 205 or step 206 .

[0077] Step 205 , in response to identifying at least one target shooting point associated with the target prediction point, construct a second set corresponding to the target prediction point and including at least one target shooting point, and execute step 207 after step 205 .

[0078] Step 206 , in response to not identifying a target shooting point whose distance to the target prediction point is less than a preset threshold, determining a missing space in the target space object associated with the target prediction point, and then returning to step 204 .

[0079] Step 207: for each target shooting point in the second set corresponding to the target prediction point, determine the target line segment corresponding to the target shooting point based on the line connecting the target shooting point and the target prediction point, and construct a closed space corresponding to the target shooting point based on the point cloud data corresponding to the image content captured at the target shooting point and the matched door body point cloud data.

[0080] Step 208: sequentially detect each target shooting point in the second set corresponding to the target prediction point, and identify whether the target line segment corresponding to the target shooting point intersects with the closed space corresponding to the target shooting point.

[0081] Step 209: When it is detected that the target shooting point does not intersect with the corresponding closed space, it is determined that the target prediction point is not associated with the missing space in the target space object; when it is detected that the target line segment corresponding to each target shooting point in the second set intersects with the corresponding closed space, it is determined that the target prediction point is associated with the missing space in the target space object.

[0082] Step 210: record the point information corresponding to the target prediction point of the missing space in the associated target space object in the first set, generate and output the space missed shot detection result corresponding to the target space object.

[0083] In the above implementation process, after determining the first set based on the comparison between the photographed points and the predicted points, the corresponding second set including the target shooting points is determined for each target predicted point in the first set, and the target shooting points in the second set are traversed. Based on the intersection of the target line segment corresponding to the target shooting point and the closed space corresponding to the target shooting point, it is identified whether the target predicted point is associated with the missed space in the target space object, and then it can be detected whether the space is missed in the process of shooting the target space object, so as to provide real and effective shooting data through missed detection, ensure the shooting quality, reduce unnecessary waste as much as possible, and improve user satisfaction.

[0084] The present application embodiment provides a spatial missed beat detection device, such as Figure 3 As shown, the device comprises:

[0085] A first determination module 301 is used to determine, when the photographed points and predicted points of the target space object are acquired, a first set including at least one target predicted point based on the photographed points and the predicted points, wherein the predicted points belonging to the door points and the open space points among the predicted points not involved in the photographing are all target predicted points;

[0086] The identification and determination module 302 is used to identify, for each target prediction point, a target shooting point whose positional relationship with the target prediction point satisfies a preset condition among the shot points of the target space object, and determine a second set corresponding to the target prediction point;

[0087] The processing module 303 is used to traverse the target shooting points in the second set corresponding to the target prediction point, and identify whether the target prediction point is associated with the missing space in the target space object according to the intersection of the target line segment corresponding to the target shooting point and the closed space corresponding to the target shooting point;

[0088] A second determination module 304 is used to determine a spatial missed shot detection result according to the missed space recognition status associated with each target prediction point in the first set;

[0089] The target line segment corresponding to the target shooting point is determined based on the target shooting point and the target predicted point, and the closed space corresponding to the target shooting point is constructed based on point cloud data corresponding to the picture content captured at the target shooting point.

[0090] Optionally, the first determining module includes:

[0091] A comparison and identification submodule, used to compare the photographed points with the predicted points corresponding to the target space object, and identify the predicted points that are not involved in the photographing;

[0092] The identification and determination submodule is used to identify the predicted points belonging to door points and open space points among the predicted points that are not involved in the shooting, and determine the first set based on the identified predicted points.

[0093] Optionally, the identification and determination module includes:

[0094] an identification submodule, configured to identify, for each target prediction point in the first set, a target shooting point whose distance to the target prediction point is less than a preset threshold among the shot points of the target space object;

[0095] A construction submodule, configured to construct, in response to identifying at least one target shooting point, a second set corresponding to the target prediction point and including at least one target shooting point;

[0096] The distance between the photographed point and the target predicted point is determined based on the world coordinate information of the point.

[0097] Optionally, the identification and determination module is further used to:

[0098] In response to not identifying a target shooting point whose distance from the target prediction point is less than a preset threshold, determining that the target prediction point is associated with a missing space in the target space object.

[0099] Optionally, the device further comprises:

[0100] An identification and construction module, used for identifying door objects in the target space object, and constructing corresponding door point cloud data based on door point information of each door object, wherein the door objects include predicted door objects and real door objects;

[0101] A construction module is determined, which is used to determine, for each target shooting point in the second set corresponding to the target prediction point, a target line segment corresponding to the target shooting point based on a line connecting the target shooting point and the target prediction point, and to construct a closed space corresponding to the target shooting point based on point cloud data corresponding to the image content captured at the target shooting point and matched door body point cloud data.

[0102] Optionally, the processing module includes:

[0103] A detection and identification submodule, used to detect each target shooting point in the second set corresponding to the target prediction point in turn, and identify whether the target line segment corresponding to the target shooting point intersects with the closed space corresponding to the target shooting point;

[0104] A first determination submodule is used to determine that the target prediction point is not associated with the missing space in the target space object when a target shooting point that does not intersect the target line segment and the corresponding closed space is detected;

[0105] The second determination submodule is used to determine the missing space in the target space object associated with the target prediction point when it is detected that the target line segment corresponding to each target shooting point in the second set intersects with the corresponding closed space.

[0106] Optionally, the second determining module is further configured to:

[0107] Recording point information corresponding to the target predicted point associated with the missing space in the target space object in the first set, wherein the point information at least includes position information and subordinate shooting points;

[0108] The spatial missed shot detection result corresponding to the target spatial object is determined according to the recording result and output, and the spatial missed shot detection result is used to indicate the spatial missed shot situation of the target spatial object.

[0109] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0110] An embodiment of the present application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned spatial missed shot detection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0111] For example, Figure 4 FIG. 1 shows a schematic diagram of the physical structure of an electronic device. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430, and the processor 410 is used to execute each process of the spatial missed shot detection method of the embodiment of the present application, which will not be described one by one here.

[0112] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0113] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned spatial missed shot detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0114] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0115] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0116] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0117] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0118] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0119] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0122] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0123] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A spatial missed shot detection method, characterized in that: include: In the case of acquiring the photographed points and predicted points of the target space object, determining a first set including at least one target predicted point based on the photographed points and the predicted points, wherein the predicted points belonging to the door points and the open space points among the predicted points not involved in the photographing are all target predicted points; For each target prediction point, identify a target shooting point whose positional relationship with the target prediction point satisfies a preset condition among the shot points of the target space object, and determine a second set corresponding to the target prediction point; Traversing the target shooting points in the second set corresponding to the target prediction point, and identifying whether the target prediction point is associated with a missing space in the target space object according to the intersection of the target line segment corresponding to the target shooting point and the closed space corresponding to the target shooting point; Determine a spatial missed shot detection result according to the missed space identification status associated with each target prediction point in the first set; The target line segment corresponding to the target shooting point is determined based on the target shooting point and the target predicted point, and the closed space corresponding to the target shooting point is constructed based on point cloud data corresponding to the picture content captured at the target shooting point.

2. The method according to claim 1, characterized in that The determining, based on the photographed points and the predicted points, a first set including at least one target predicted point, comprises: Comparing the photographed points with the predicted points corresponding to the target space object, and identifying the predicted points that are not involved in the photographing; Among the predicted points that are not involved in the shooting, predicted points belonging to door points and open space points are identified, and the first set is determined based on the identified predicted points.

3. The method according to claim 1, characterized in that For each target prediction point, identifying a target shooting point whose positional relationship with the target prediction point satisfies a preset condition from among the shot points of the target space object, and determining a second set corresponding to the target prediction point, including: For each target prediction point in the first set, identifying a target shooting point whose distance to the target prediction point is less than a preset threshold from the shot points of the target space object; In response to identifying at least one target shooting point, constructing a second set corresponding to the target prediction point and including at least one target shooting point; The distance between the photographed point and the target predicted point is determined based on the world coordinate information of the point.

4. The method according to claim 3, characterized in that: The method further comprises: In response to not identifying a target shooting point whose distance from the target prediction point is less than a preset threshold, determining that the target prediction point is associated with a missing space in the target space object.

5. The method according to claim 1 or 3, characterized in that: The method further comprises: Identify door objects in the target space object, and construct corresponding door point cloud data based on door point information of each door object, wherein the door objects include predicted door objects and real door objects; For each target shooting point in the second set corresponding to the target prediction point, the target line segment corresponding to the target shooting point is determined based on the line connecting the target shooting point and the target prediction point, and the closed space corresponding to the target shooting point is constructed based on the point cloud data corresponding to the picture content captured at the target shooting point and the matched door body point cloud data.

6. The method according to claim 5, characterized in that The traversing the target shooting points in the second set corresponding to the target prediction point, and identifying whether the target prediction point is associated with a missing space in the target space object according to an intersection between a target line segment corresponding to the target shooting point and a closed space corresponding to the target shooting point, includes: sequentially detecting each target shooting point in the second set corresponding to the target prediction point, and identifying whether the target line segment corresponding to the target shooting point intersects with the closed space corresponding to the target shooting point; When detecting a target shooting point position where the target line segment does not intersect with the corresponding closed space, determining that the target prediction point position is not associated with a missing space in the target space object; When it is detected that the target line segment corresponding to each target shooting point in the second set intersects with the corresponding closed space, it is determined that the target prediction point is associated with the missing space in the target space object.

7. The method according to claim 1 or 4, characterized in that: The determining of the spatial missed shot detection result according to the missed space recognition status associated with each target prediction point in the first set includes: Recording point information corresponding to the target predicted point associated with the missing space in the target space object in the first set, wherein the point information at least includes position information and subordinate shooting points; The spatial missed shot detection result corresponding to the target spatial object is determined according to the recording result and output, and the spatial missed shot detection result is used to indicate the spatial missed shot situation of the target spatial object.

8. A spatial missed beat detection device, characterized in that: include: A first determination module is used to determine, when the photographed points and predicted points of the target space object are acquired, a first set including at least one target predicted point based on the photographed points and the predicted points, wherein the predicted points belonging to the door points and the open space points among the predicted points not involved in the photographing are all target predicted points; an identification and determination module, configured to identify, for each target prediction point, a target shooting point whose positional relationship with the target prediction point satisfies a preset condition among the shot points of the target space object, and determine a second set corresponding to the target prediction point; a processing module, configured to traverse the target shooting points in the second set corresponding to the target prediction point, and identify whether the target prediction point is associated with a missing space in the target space object according to the intersection of the target line segment corresponding to the target shooting point and the closed space corresponding to the target shooting point; A second determination module is used to determine a spatial missed shot detection result according to the missed space recognition status associated with each target prediction point in the first set; The target line segment corresponding to the target shooting point is determined based on the target shooting point and the target predicted point, and the closed space corresponding to the target shooting point is constructed based on point cloud data corresponding to the picture content captured at the target shooting point.

9. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the spatial missed beat detection method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the spatial missed beat detection method according to any one of claims 1 to 7 are implemented.

Images