A virtual-real scene difference identification method, system and device

Abstract

This invention discloses a method, system, and device for identifying differences between virtual and real scenes, relating to the fields of digital twin and 3D point cloud processing technology. The method involves aligning real-world point clouds with a digital twin model and dividing them into identical spatial units. Digital objects present in the digital twin model but whose occupied spatial units lack real-world point clouds are then selected. Lines are drawn from the point cloud acquisition position to the spatial units occupied by each selected digital object, checking whether the lines pass through spatial units containing real-world point clouds before reaching the corresponding spatial unit. Based on the number of lines passing through and not passing through spatial units containing real-world point clouds, and the consecutive adjacency of corresponding spatial units in front of the digital object, occluded digital objects are identified. The point cloud acquisition position is changed, and new lines are drawn. If the new lines do not pass through spatial units containing real-world point clouds and the occupied spatial unit still lacks real-world point clouds, the digital object is modified to be a missing digital object.

Description

Technical Field

[0001] This invention relates to the field of digital twin and 3D point cloud processing technology, and more specifically, to a method, system and device for identifying differences between virtual and real scenes. Background Technology

[0002] Existing methods for identifying differences between virtual and real scenes typically begin by acquiring real-world point cloud data of the actual scene using laser scanning equipment, depth cameras, or mobile point cloud acquisition devices. This real-world point cloud data is then spatially registered with the digital twin model. Based on the positional, distance, or spatial occupancy relationships between the point cloud and the model surface, it is determined whether objects in the digital twin model exist in the real scene. Some technologies further divide the target scene into grids or voxels, identifying potential new, missing, or repositioned objects between the digital twin model and the real scene based on the presence of real-world point cloud data within each grid or voxel. For scenes where multiple objects occlude each other, visibility analysis, model ray detection, or repeated point cloud acquisition from multiple acquisition locations are typically used to supplement the judgment of model surfaces for which real-world point cloud data has not been obtained.

[0003] The existing technology has the following shortcomings: The absence of a real-world point cloud within the spatial region corresponding to a digital object could be due to two reasons: the digital object may be missing from the real scene, or it may still exist but be occluded by other real-world objects in front of it. Both situations manifest as a lack of corresponding point clouds at the model location in a single point cloud acquisition. Existing technologies, relying solely on the distance between the point cloud and the model or the presence of point clouds within a spatial unit, struggle to distinguish between the absence of a digital object and occlusion of the observation path. A small number of noisy points, discrete point clouds, small object point clouds, or insufficient point cloud acquisition range can also lead to digital objects without real-world point clouds within their occupied spatial units being incorrectly identified as occluded or missing. Even with verification using multiple point cloud acquisition locations, without determining the continuous occlusion range based on the observation path between the point cloud acquisition location before changing the acquisition location and the digital object, it's difficult to selectively choose new point cloud acquisition locations. Furthermore, there's a lack of processing mechanisms to correlate new observation results with occluded digital objects for modification.

[0004] To address the above problems, this invention proposes a solution. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method, system, and device for identifying differences between virtual and real scenes, in order to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A method for recognizing differences between virtual and real scenes, including the following steps; Align the real-world point cloud with the digital twin model and divide it into the same spatial units, then filter out digital objects that exist in the digital twin model but whose spatial units do not contain the real-world point cloud. Determine the point cloud acquisition location when acquiring real point cloud, draw lines from the point cloud acquisition location to the spatial unit occupied by each selected digital object, and check whether the spatial unit containing real point cloud is passed before reaching the corresponding spatial unit along each line. When the number of connections passing through spatial units containing real-world point clouds is greater than the number of connections not passing through spatial units containing real-world point clouds, and the corresponding spatial units are consecutively adjacent in front of the digital object and reach a preset number, the digital object is identified as an occluded digital object. Change the point cloud acquisition location, and draw a line from the new point cloud acquisition location to the spatial unit occupied by the occupied digital object. When the new line does not pass through the spatial unit containing real point cloud, and there is still no real point cloud in the spatial unit occupied by the digital object, modify it to the missing digital object.

[0007] In a preferred embodiment, aligning the physical point cloud with the digital twin model includes: Fixed scene structures that correspond to each other and have not changed are selected from the real point cloud and the digital twin model respectively. The moving distance and rotation direction of the real point cloud are determined according to the positional differences between the fixed scene structures. The real point cloud is adjusted according to the moving distance and rotation direction, and the point cloud acquisition position is adjusted in the same position and direction to make the real point cloud, the point cloud acquisition position and the digital twin model in the same coordinate system.

[0008] In a preferred embodiment, dividing the real-world point cloud and the digital twin model into identical spatial units and filtering digital objects includes: Determine the three-dimensional spatial range jointly covered by the real-world point cloud and the digital twin model, divide the three-dimensional spatial range according to the uniform spatial unit side length, and write the points in the real-world point cloud into the spatial unit in which they are located. Based on the point cloud acquisition location, determine the observable surface of the digital object facing the point cloud acquisition location, determine the spatial unit corresponding to the observable surface as the spatial unit occupied by the digital object, and filter digital objects whose occupied spatial units are within the effective acquisition range and do not have real point clouds.

[0009] In a preferred embodiment, lines are drawn from the point cloud acquisition location to the spatial units occupied by each selected digital object, including: For each spatial unit occupied by a selected digital object that does not contain a real point cloud, the center of the surface of the spatial unit facing the point cloud acquisition location is determined. The line segment between the point cloud acquisition location and the surface center is taken as the line connecting the spatial unit. Starting from the point cloud acquisition location, the spatial units that the line passes through are determined sequentially along the line until the spatial unit is reached.

[0010] In a preferred embodiment, checking along each connecting line whether a spatial unit containing a real-world cloud has been passed before reaching the spatial unit connected by the connecting line includes: Read whether each spatial unit traversed by the connection contains a real point cloud; When a line passes through a spatial unit containing a real point cloud before reaching the spatial unit it connects to, the line is identified as a line that passes through a spatial unit containing a real point cloud, and the first spatial unit containing a real point cloud that the line passes through is recorded as an occluding spatial unit. When a line passes through spatial units that do not contain real point clouds before reaching the spatial unit it connects, the line is determined to be a line that does not pass through spatial units containing real point clouds.

[0011] In a preferred embodiment, determining that occlusion space units are consecutively adjacent in front of the digital object includes: The occlusion space units on the surface of the shared space unit are defined as continuous adjacent. Based on the continuous adjacent relationship, the occlusion space units are divided into at least one group of continuous adjacent space units. The group of continuous adjacent space units containing the most space units is defined as the occlusion space units that are continuously adjacent in front of the digital object.

[0012] In a preferred embodiment, the preset quantity is determined based on the number of space units occupied by the digital object and a preset continuity ratio. The preset continuity ratio is determined based on the coverage relationship between the consecutive adjacent occupying space units in front of the confirmed occupied digital object and the space units occupied by the digital object.

[0013] In a preferred embodiment, changing the point cloud acquisition location includes: Based on the consecutive adjacent occlusion space units in front of the occluded digital object, the candidate point cloud acquisition location is determined in the acquisition area where the point cloud acquisition device is allowed to enter and the space unit occupied by the occluded digital object is within the effective acquisition range. Predicted connections were made from each candidate point cloud acquisition location to the spatial unit occupied by the occluded digital object. The candidate point cloud acquisition location whose predicted connection passed through the fewest occluded spatial units was selected as the new point cloud acquisition location. Obtain real-world point cloud data from new point cloud acquisition locations, use the same spatial unit division and real-world point cloud existence judgment method, reconnect and check occluded digital objects.

[0014] A virtual-real scene difference recognition system includes: an object filtering module, a connection checking module, an occluded object determination module, and a missing object modification module, with signal connections between the modules; The object filtering module is used to align the real point cloud with the digital twin model and divide it into the same spatial units, and filter digital objects that exist in the digital twin model but do not exist in the real point cloud within the spatial unit they occupy. The connection check module is used to determine the point cloud acquisition location when acquiring real point cloud. It draws a line from the point cloud acquisition location to the spatial unit occupied by each selected digital object, and checks whether the connection line passes through a spatial unit containing real point cloud before reaching the corresponding spatial unit. The occlusion object determination module is used to determine a digital object as an occluded digital object when the number of lines passing through spatial units containing real point clouds is greater than the number of lines not passing through spatial units containing real point clouds, and the corresponding spatial units are consecutively adjacent in front of the digital object and reach a preset number. The missing object modification module is used to change the point cloud acquisition position. A line is drawn from the new point cloud acquisition position to the spatial unit occupied by the occupied digital object. When the new line does not pass through the spatial unit containing real point cloud data, and there is still no real point cloud data in the spatial unit occupied by the digital object, it is modified to be a missing digital object.

[0015] A virtual-real scene difference recognition device includes a point cloud acquisition device and a difference recognition processing device. The point cloud acquisition device is used to acquire real-world point clouds of a target scene from the point cloud acquisition location. The difference recognition processing device is used to align the real-world point clouds with a digital twin model and divide them into the same spatial units. Based on the connection check results from the point cloud acquisition location to the spatial unit occupied by the digital object, the device determines the occluded digital object. After changing the point cloud acquisition location, if the new connection does not pass through the spatial unit containing the real-world point clouds and there are still no real-world point clouds in the spatial unit occupied by the digital object, the device modifies it to the missing digital object.

[0016] The technical effects and advantages of the present invention's method for recognizing differences between virtual and real scenes: By aligning real-world point cloud data with the digital twin model and dividing them into identical spatial units, the spatial distribution of real-world point cloud data and the model positions of digital objects can be compared and contrasted at a unified spatial scale. Digital objects present in the digital twin model but lacking real-world point cloud data within their respective spatial units are selected, forming a range of digital objects requiring further evaluation. By drawing lines from the point cloud acquisition location to the spatial units occupied by the digital objects, it is checked whether the lines pass through spatial units containing real-world point cloud data before reaching the corresponding spatial unit. The number of lines passing through and not passing through spatial units containing real-world point cloud data, along with the continuous adjacency of the corresponding spatial units in front of the digital object, distinguishes between occluded and missing digital objects. By changing the point cloud acquisition location and re-checking the lines, if the new lines do not pass through spatial units containing real-world point cloud data but the spatial unit occupied by the digital object still lacks real-world point cloud data, the occluded digital object is modified to a missing digital object. This establishes a verification relationship based on different point cloud acquisition locations and outputs the virtual-real scene difference recognition results corresponding to occluded and missing digital objects. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of a virtual-real scene difference recognition method according to the present invention.

[0018] Figure 2 This is a schematic diagram illustrating the alignment of the real-world location cloud with the digital twin model and the division of spatial units.

[0019] Figure 3 A schematic diagram for filtering and checking digital objects. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example

[0021] Please see Figures 1-3 As shown, this invention discloses a method for recognizing differences between virtual and real scenes, including the following steps: Step S1: Align the real-world point cloud with the digital twin model and divide it into the same spatial units, then filter out digital objects that exist in the digital twin model but whose spatial units do not contain the real-world point cloud. Step S2: Determine the point cloud acquisition location when acquiring real point cloud data. Draw lines from the point cloud acquisition location to the spatial units occupied by each selected digital object. Check whether the connection line passes through a spatial unit containing real point cloud data before reaching the corresponding spatial unit. Step S3: When the number of lines passing through spatial units containing real-world point clouds is greater than the number of lines not passing through spatial units containing real-world point clouds, and the corresponding spatial units are consecutively adjacent in front of the digital object and reach a preset number, the digital object is identified as an occluded digital object. Step S4: Change the point cloud acquisition position and draw a line from the new point cloud acquisition position to the spatial unit occupied by the occupied digital object. When the new line does not pass through the spatial unit containing the real point cloud and there is still no real point cloud in the spatial unit occupied by the digital object, modify it to the missing digital object.

[0022] In step one, the real-world point cloud and the digital twin model are aligned and divided into identical spatial units. Digital objects that exist in the digital twin model but whose occupied spatial units do not contain the real-world point cloud are then filtered out. Specific steps include: Align the real point cloud with the digital twin model and divide it into the same spatial units. Based on the point cloud acquisition position, determine the observable surface of each digital object in the digital twin model facing the point cloud acquisition position. Determine the spatial unit corresponding to the observable surface as the spatial unit occupied by the digital object, and filter out digital objects that exist in the digital twin model but whose occupied spatial units do not contain real point clouds.

[0023] It should be noted that real-world point cloud is a set of three-dimensional points obtained by scanning the target scene with a point cloud acquisition device, and each point includes positions in at least three coordinate directions; A digital twin model includes the geometric surfaces and spatial locations of each digital object within the target scene, as well as object identifiers used to distinguish different digital objects. Digital objects can be building components, production equipment, cabinets, shelves, doors and windows, pipes, or other objects with independent geometric extents in a digital twin model; This implementation method does not use the specific category of the digital object as the basis for judgment, but rather processes it based on the spatial relationship between the surface of the digital object and the real-world point cloud.

[0024] It should be noted that the point cloud of the real-world points is processed before aligning with the digital twin model. Point cloud processing includes deleting points that are outside the target scene range, deleting isolated points whose distance from surrounding points is significantly greater than the normal distance between points, and unifying the length unit between real-world point clouds and digital twin models; The distance between normal points can be determined by finding the neighboring points of each point in the real-world point cloud, sorting the distances of multiple neighboring points, and obtaining the distance at the median position. Points whose distance from surrounding points is consistently greater than a preset multiple of the normal point-to-point distance, and for which there are no other real-world points within the predetermined surrounding area, can be deleted as isolated points.

[0025] Furthermore, alignment refers to adjusting the position and orientation of the real-world scene cloud relative to the digital twin model, so that the fixed scene structure in the real-world scene cloud and the corresponding structure in the digital twin model are at the same coordinate position. During alignment, select wall intersections, corners, fixed columns, or fixed structures that have been confirmed to remain unchanged from the real-world point cloud and the digital twin model, respectively, as the corresponding structures. Determine the distance the real-world point cloud needs to move and the direction it needs to rotate based on the positional differences between the corresponding structures, and adjust the real-world point cloud according to the determined distance and direction.

[0026] Furthermore, after completing one position and orientation adjustment, the distance between the positions of multiple fixed structures in the real-world point cloud and their positions in the digital twin model is calculated, and the alignment error after this adjustment is obtained based on the distance. When the alignment error is greater than the preset alignment error, the movement distance and rotation direction of the real-world point cloud are redefined; when the alignment error is not greater than the preset alignment error, the alignment is completed. The preset alignment error is determined based on the side length of the spatial unit. The preset alignment error should be less than the side length of the spatial unit to avoid the same object surface being written into adjacent but non-corresponding spatial units due to alignment error. For example, half the side length of the spatial unit can be used as the preset alignment error, or the preset alignment error can be determined based on the sum of the repeated measurement error of the point cloud acquisition device at normal distance and the geometric error of the model.

[0027] It should be noted that the fixed structure used for alignment does not include digital objects that need to be occluded or missing in this case; Using digital objects that have moved, are missing, or may be occluded as alignment criteria may cause the entire real-world point cloud to move in the direction of the digital object's change. Therefore, walls, floors, load-bearing columns, or equipment foundations should be used as the primary means of alignment.

[0028] Furthermore, when adjusting the position and orientation of the real-world point cloud, the same position and orientation adjustment is made to the point cloud acquisition position when acquiring the real-world point cloud, so as to obtain the point cloud acquisition position in the same coordinate system as the digital twin model. Therefore, the lines subsequently drawn from the point cloud acquisition location to the digital object can have a consistent spatial relationship with the aligned real point cloud and digital twin model.

[0029] After alignment is completed, the three-dimensional spatial range jointly covered by the real-world point cloud and the digital twin model is determined, and the three-dimensional spatial range is divided into multiple cubic spatial units according to the uniform spatial unit side length. Spatial units can be divided using a fixed grid or by using an octree to divide them layer by layer; When using an octree partitioning method, starting from the root space that surrounds the target scene, the spatial region containing the real scene cloud or the surface of the digital object is divided into eight subspaces until the side length of the subspace reaches the side length of the spatial unit, and the last-level subspace is taken as the spatial unit.

[0030] It should be noted that the side length of the spatial unit is determined based on the distance between points in the real-world point cloud and the minimum surface size of the digital object to be identified. The side length of the spatial unit is not less than the distance between normal points, so as to avoid excessive dispersion of real points on the surface of normal objects; the side length of the spatial unit is less than the minimum surface width of the digital object to be identified, so that the surface of a digital object can correspond to multiple spatial units. Once the side length of the spatial unit is determined, it remains unchanged in steps one through four.

[0031] For example, first obtain the distance between multiple points in the real-world point cloud and their nearest neighbor, and take the distance in the middle position after sorting as the normal distance between points; then obtain the minimum surface width of the digital object to be identified in the digital twin model. The side length of the spatial unit can be selected between the normal distance between points and the minimum surface width, so that the main surface of the digital object facing the point cloud acquisition position can be divided into multiple spatial units in at least two coordinate directions.

[0032] Furthermore, each point in the real-world point cloud is written into the corresponding spatial unit according to its coordinates, and the number of real-world points in each spatial unit is counted. When the number of real points of interest within a spatial unit reaches the threshold for the number of real points of interest, the spatial unit is defined as a spatial unit containing a cloud of real points of interest; when the number of real points of interest does not reach the threshold for the number of real points of interest, the spatial unit is defined as a spatial unit not containing a cloud of real points of interest.

[0033] It should be noted that the threshold for the number of real points is used to exclude the influence of a single noise point or a small number of discrete points on the state of the spatial unit. The threshold for the number of real points can be determined based on the number of real points within the spatial unit corresponding to the confirmed fixed structural surface. Specifically, the number of real points in multiple fixed structure surface spatial units is counted, and the number of real points at the lower position is taken as the threshold for the number of real points. This allows most spatial units on the surface of a normal object to be identified as containing a cloud of real points, while preventing spatial units containing only a small number of discrete noise points from being identified as containing a cloud of real points. If outlier removal has been completed before data collection and each retained point can represent a valid measurement result, then a single real point can be used as the threshold for the number of real points.

[0034] Furthermore, based on the effective acquisition distance and acquisition direction of the point cloud acquisition device, the effective acquisition range corresponding to the current point cloud acquisition location is determined; When the distance between the point cloud acquisition device and the spatial unit does not exceed the effective acquisition distance of the device, and the spatial unit is located within the acquisition direction range of the device, the spatial unit is determined to be within the effective acquisition range; The effective acquisition distance and acquisition direction of a point cloud acquisition device can be read from the device parameters or obtained by testing a fixed surface at a known distance.

[0035] It should be noted that only spatial units within the effective acquisition range participate in subsequent occlusion and missing data determination; For spatial units that are outside the effective acquisition distance or located outside the acquisition direction, even if there are no real point clouds in them, it cannot be determined that digital objects are missing.

[0036] Furthermore, for each digital object in the digital twin model, the observable surface of the digital object facing the point cloud acquisition location is determined based on the point cloud acquisition location; Specifically, model lines are drawn from the point cloud acquisition location to each surface location of the digital object. If the model line does not pass through other surfaces of the same digital object before reaching the corresponding surface location, the surface location is determined as an observable surface. If the model line passes through other surfaces of the same digital object first, it means that the corresponding surface location is occluded by the digital object itself and is not considered as an observable surface.

[0037] It should be noted that by first identifying observable surfaces, we can avoid using the internal space of digital objects or surfaces facing away from the point cloud acquisition location as evidence of missing points in the cloud. Point cloud acquisition devices typically only obtain surface point clouds of digital objects facing the acquisition location, and do not obtain point clouds inside the entity. Therefore, this implementation only compares the spatial units corresponding to the observable surface.

[0038] Furthermore, the spatial units that can be entered or passed through on the observable surface are defined as the spatial units occupied by the digital object, and a correspondence is established between the object identifier of the digital object and each spatial unit occupied by the digital object. For a digital object represented by a triangular mesh, the triangular face is determined to intersect the spatial cell when the vertex of the triangular face is located inside the spatial cell, the edge of the triangular face passes through the spatial cell, or the triangular face passes through the spatial cell.

[0039] Furthermore, the total number of spatial units occupied by each digital object, and the number of spatial units in which no real point cloud exists; The number of spatial units that do not exist in the real-world cloud is compared with the total number of spatial units occupied by the digital object to determine whether to filter the digital object.

[0040] It should be noted that in a strict screening method, a digital object is screened out when there are no real point clouds within all the spatial units occupied by the digital object. In another approach applicable to partial point cloud defects, a threshold can be set for the number of spatial units in which no real point cloud exists. When the number of spatial units in which no real point cloud exists in the space occupied by a digital object reaches this threshold, the digital object is filtered out. The threshold for the number of spatial units where no real-world points of view are present is determined based on the total number of spatial units occupied by digital objects. This threshold can be obtained by multiplying the total number of spatial units occupied by digital objects by a preset proportion of no real-world points of view, and then rounding up.

[0041] It should be noted that the preset percentage of point clouds without real objects is determined based on the historical point cloud coverage of normally existing digital objects; Multiple samples of similar digital objects that have been confirmed to exist normally and are not obscured are obtained. The proportion of spatial units in the space occupied by the digital objects in each sample that do not contain real-world cloud is calculated. The larger value of the proportion is obtained, and an allowable range to accommodate measurement fluctuations is added to the larger value to obtain the preset proportion of non-real-world cloud. Therefore, minor point cloud gaps caused by normal acquisition fluctuations will not directly trigger the screening process, while digital objects exceeding the normal fluctuation range will be included in the subsequent connection check.

[0042] For example, when the proportion of non-real-world cloud in the space units occupied by confirmed normal digital objects does not exceed a certain range, the proportion higher than that range can be determined as the preset proportion of non-real-world cloud. For scenarios without historical samples, the preset proportion of point cloud locations that do not exist can be determined based on the minimum normal coverage of the point cloud acquisition device on a known fixed surface.

[0043] After completing step one, we obtain the filtered digital objects, the object identifier of each digital object, the spatial unit occupied by the digital object, the spatial unit that does not contain real point cloud, the spatial unit that contains real point cloud, the point cloud acquisition location, the effective acquisition range, and the basis for determining the preset quantity used in the subsequent step three. Step one only filters digital objects that require further judgment, without directly determining whether the digital objects are occluded or missing.

[0044] In step two, the point cloud acquisition location is determined when acquiring real-world point clouds. Lines are drawn from the point cloud acquisition location to the spatial units occupied by each selected digital object. It is then checked whether the line passes through a spatial unit containing real-world point clouds before reaching the corresponding spatial unit. Specific steps include: Determine the point cloud acquisition location when acquiring real-world point clouds. Draw lines from the point cloud acquisition location to each spatial unit without real-world point clouds occupied by each digital object selected in step one. Check whether each line passes through a spatial unit containing real-world point clouds before reaching the corresponding spatial unit. Record the spatial unit containing real-world point clouds that each line passes through first, and obtain the line check results corresponding to each digital object.

[0045] It should be noted that the point cloud acquisition location is the spatial location used by the point cloud acquisition device in step one when it acquires the real-world point cloud data. For fixed laser scanning equipment, the point cloud acquisition location is the scanning center; for depth cameras, the point cloud acquisition location is the camera optical center; for mobile point cloud acquisition equipment, the real point cloud is divided into multiple acquisition segments according to the equipment's operating time, and the equipment position when the corresponding acquisition segment's real point cloud is generated is taken as the point cloud acquisition location.

[0046] It should be noted that when a real-world point cloud is obtained by merging data from multiple point cloud collection locations, the correspondence between each real-world point and the point cloud collection location should be preserved. For a digital object, connection checks can be performed separately based on each point cloud acquisition location, without directly merging multiple observation directions of unknown origin into a single point cloud acquisition location.

[0047] Further, read the digital objects selected in step one and the spatial units of the non-existent real-world point clouds they occupy; For each spatial unit occupied by a digital object that does not contain a real point cloud, determine the surface center of the spatial unit on the side facing the point cloud acquisition location, and draw a line from the point cloud acquisition location to the surface center. A line connects a non-physical point cloud to a spatial unit occupied by a digital object, so that the inspection result of each line can correspond to a surface part of the digital object.

[0048] It should be noted that by using the center of the surface of the spatial unit occupied by the digital object that does not contain real point cloud on the side facing the point cloud acquisition position as the endpoint of the connection, the connection can be stopped when it reaches the surface of the digital object, avoiding the connection from continuing to pass through the interior or rear of the digital object. If the intersection point between the digital object surface and the spatial unit can be obtained directly, the intersection point between the digital object surface and the direction of the connection can also be used as the endpoint of the connection.

[0049] Furthermore, starting from the point cloud acquisition location, the spatial units traversed by the connecting line are determined sequentially from near to far until the corresponding spatial unit is reached; When determining the spatial unit through which the connection passes, calculate the order in which the connection reaches the boundary of the next spatial unit in the three coordinate directions. Enter the adjacent spatial unit according to the boundary reached first, and repeat the process to obtain the spatial units through which the connection passes in sequence.

[0050] It should be noted that when a line passes through the common edge of two spatial units or the common vertex of multiple spatial units, all spatial units located in the direction of the line's movement and adjacent to the common edge or common vertex are considered as spatial units passed through by the line, in order to avoid missed detections due to coordinate rounding. For spatial units that are directly adjacent to the point cloud acquisition location and are used to place the acquisition equipment, they can be excluded from the occlusion judgment to avoid the point cloud of the equipment itself affecting the connection inspection.

[0051] Furthermore, the spatial cell state obtained in step one is read for each spatial cell traversed by the connection line; When a line passes through a spatial unit containing a real point cloud before reaching the corresponding spatial unit, the line is determined to be a line that passes through a spatial unit containing a real point cloud; when a line passes through none of the spatial units containing a real point cloud before reaching the corresponding spatial unit, the line is determined to be a line that does not pass through any spatial unit containing a real point cloud.

[0052] Furthermore, for a line that passes through a spatial unit containing a real point cloud, starting from the point cloud acquisition location, the first spatial unit containing a real point cloud that appears on the line is identified and recorded as the first spatial unit containing a real point cloud that the line passes through. Compared to recording all spatial cells containing real point cloud data that the connection passes through, recording the first spatial cell passed through can represent the real surface that first blocks the measurement path when observed from the point cloud acquisition location.

[0053] It should be noted that only when the distance between the first spatial unit containing real point cloud and the point cloud acquisition location is less than the distance between the corresponding spatial unit and the point cloud acquisition location, it is considered as the corresponding spatial unit in front of the digital object. Since this step checks the distance relationship before reaching the spatial unit occupied by the digital object that does not have a real point cloud along the connecting line, the distance relationship can be directly determined by the order in which the spatial units are passed, or it can be verified by calculating the distance from the center of each spatial unit to the point cloud acquisition location.

[0054] Furthermore, when the connection passes through a spatial unit containing real-world point clouds, but the spatial unit is an observable surface that the target digital object itself has already manifested, it is not used as a frontal obstruction formed by other objects. Specifically, based on the object identifier of the digital object and the correspondence between the spatial unit and the surface of the digital object, it is determined whether the spatial unit containing the real point cloud also belongs to the surface of the target digital object. Spatial units belonging to the surface of the target digital object itself are excluded from the front occlusion check.

[0055] It should be noted that in step one, digital objects that do not have real point cloud in the entire space unit occupied by the digital object are selected. Therefore, the real point cloud on the surface of the target digital object itself will usually not appear in front of the line. However, when using a preset screening method that does not have real point clouds, or when only part of the space unit occupied by the digital object does not have real point clouds, the above exclusion process needs to be performed to avoid the surface of the target digital object that already has real point clouds being mistakenly identified as an obstruction in front.

[0056] Furthermore, for each selected digital object, the connection check results are saved one by one according to the spatial unit of the non-existent real point cloud it occupies; The results of each connection check should include at least the object identifier of the digital object, the location of the corresponding spatial unit, whether the connection passes through a spatial unit containing a real point cloud, and the location of the first spatial unit containing a real point cloud that it passes through.

[0057] It should be noted that this step does not require prior identification of which real object the clouds in front of you belong to; As long as the spatial unit containing the real point cloud is located between the point cloud acquisition location and the digital object, and is not the surface of the target digital object itself, it can be indicated that the corresponding observation path is blocked by the real surface in front. The actual surface in front can be cargo, vehicles, temporary equipment, other fixed components, or other objects entering the observation path.

[0058] For example, there is no physical point cloud within the multiple space units occupied by the cabinet in the digital twin model; Of the lines drawn from the point cloud acquisition location to the space units occupied by the rack, some lines first pass through the space unit containing the real point cloud on the surface of the toolbox, while other lines pass through the edge of the toolbox without passing through the space unit containing the real point cloud. This step records the two types of connections separately and saves the toolbox surface space unit that each blocked connection first encounters.

[0059] Furthermore, for the spatial units occupied by digital objects at the edge of the effective acquisition range, the distance between them and the upper limit of the effective acquisition distance of the point cloud acquisition device can be checked. When the difference between the spatial unit occupied by the digital object and the upper limit of the effective acquisition distance is less than the allowable error of the device distance measurement, the corresponding connection inspection result is marked as an edge result. Step 3 can avoid using the edge result to judge occlusion or missing information, thus avoiding misjudgment caused by the effective distance boundary of the device.

[0060] After completing step two, we obtain the lines that pass through spatial units containing real-world points of view, the lines that do not pass through spatial units containing real-world points of view, and the first spatial unit containing real-world points of view encountered by each line. The above connection check results serve as the data source for step three: counting the number of connections, determining whether corresponding spatial units are consecutive and adjacent, and identifying occluded or missing digital objects.

[0061] In step three, when the number of lines passing through spatial units containing real-world point clouds is greater than the number of lines not passing through spatial units containing real-world point clouds, and the corresponding spatial units are consecutively adjacent in front of the digital object and reach a preset number, the digital object is determined to be an occluded digital object. Specific details include: Based on the connection check results obtained in step two, the number of connections that pass through and do not pass through spatial units containing real-world point clouds are counted respectively. The spatial unit containing real-world point clouds that each passing connection first passes through is taken as the corresponding spatial unit. It is determined whether the corresponding spatial unit is consecutively adjacent in front of the digital object. When the number of connections that pass through spatial units containing real-world point clouds is greater than the number of connections that do not pass through, and the consecutively adjacent corresponding spatial units reach a preset number, the digital object is identified as an occluded digital object; otherwise, it is identified as a missing digital object.

[0062] It should be noted that each digital object is counted separately, and the lines corresponding to a digital object are only associated with that digital object. Lines from different digital objects are not combined for statistics. A connection in step two can only be counted as either a connection that passes through a spatial unit containing a real point cloud or a connection that does not pass through a spatial unit containing a real point cloud.

[0063] Furthermore, the number of lines corresponding to a digital object that pass through spatial units containing real-world point clouds is counted to obtain the number of lines that pass through; the number of lines that do not pass through spatial units containing real-world point clouds is counted to obtain the number of lines that do not pass through. If the number of connected points is greater than the number of unconnected points, it means that when observing more than half of the spatial units occupied by the digital object from the current point cloud acquisition position, the observation path has encountered other real point clouds before reaching the digital object.

[0064] It should be noted that using the number of connected lines being greater than the number of unconnected lines as the first criterion is to confirm that the cloud cover of the real-world point in front covers the main observation range of the digital object, rather than just obscuring a small portion of the digital object's edge. For scenarios with higher requirements for occlusion coverage, the required ratio of the number of connected lines to the total number of valid connections can be determined based on the confirmed occlusion samples.

[0065] Furthermore, when using the proportion judgment, multiple samples of the same type of digital objects that have been confirmed to be occluded and multiple samples of the same type of digital objects that have been confirmed to be missing are obtained, and the proportion of the number of connected lines of each sample to the total number of valid connections is calculated. Choose the proportion that can separate the majority of occluded samples from the majority of missing samples as the threshold for the proportion after the connection. When the proportion of connected lines reaches this threshold, the first judgment condition is met; If historical samples are lacking, the first criterion is that the number of connected lines is greater than the number of unconnected lines, i.e., the proportion of connected lines exceeds half.

[0066] Furthermore, the spatial units containing real-world cloud features that are first passed through by each connecting line are obtained, and it is determined whether the spatial units are consecutive and adjacent. When two spatial cells share a single spatial cell surface, the two spatial cells are considered to be continuously adjacent; when two spatial cells only share an edge or a vertex, they are not necessarily considered to be continuously adjacent. By using the adjacency relationship of shared surfaces, it is possible to eliminate the false continuous occlusion area formed by a small number of obliquely dispersed real points.

[0067] Furthermore, starting from any spatial unit containing real-world cloud that is first passed, search for other spatial units that are continuously adjacent to it, and continue to search for spatial units that are continuously adjacent to the found spatial units until there are no new continuously adjacent spatial units, thus obtaining a group of continuously adjacent spatial units. Repeat the above search for other spatial units that have not been added to the group to obtain multiple consecutive adjacent spatial unit groups.

[0068] Furthermore, the number of spatial units in each consecutive adjacent spatial unit group is counted, and the spatial units in the consecutive adjacent spatial unit group with the largest number are determined as the corresponding consecutive adjacent spatial units in front of the digital object.

[0069] It should be noted that the number of consecutive adjacent spatial units in front of the digital object is used to determine whether the real-world point cloud in front forms a continuous object surface. If there are many connecting lines, but the spatial units encountered by each connecting line are scattered, it may be caused by suspended pipelines, point clouds of a small number of moving people, or discrete noise, which is insufficient to prove that the digital object is occluded by continuous entities.

[0070] Furthermore, a preset quantity is determined based on the number of spatial units occupied by the digital object; Specifically, the number of spatial units occupied by the digital object is multiplied by a preset continuous ratio, and the result is rounded up to obtain the preset quantity corresponding to the digital object. By using a preset number that is related to the number of space units occupied by the digital object, it is possible to avoid discrepancies in judgment caused by using the same fixed number for digital objects of different sizes.

[0071] It should be noted that the preset continuous ratio is determined based on the coverage relationship between the corresponding spatial units that are consecutively adjacent to the digital object in front of the confirmed occupied digital object and the spatial unit occupied by the digital object. In a sample of multiple confirmed occluded digital objects, the proportion of consecutive adjacent spatial units in front of the digital object to the total number of spatial units occupied by the digital object is statistically analyzed, and the lower proportion that can stably represent consecutive occlusion is taken as the preset consecutive proportion. The preset continuous ratio should be higher than the maximum continuous ratio formed by normal scattered point clouds in front of digital objects, so that the scattered point clouds cannot reach the preset number.

[0072] For example, first collect point clouds of multiple digital objects when they are not occluded, and count the number of the largest consecutive adjacent spatial units formed by noise or other small objects in front of them; then collect point clouds of multiple digital objects when they are occluded by real objects, and count the number of consecutive adjacent spatial units formed by the actual occlusion. The preset number is defined as a value that is greater than the maximum number of unoccluded samples and not higher than the number of most occluded samples. For digital objects of different sizes, the corresponding preset quantity can be obtained by converting them according to a preset continuous ratio.

[0073] Furthermore, when the number of lines passing through spatial units containing real-world point clouds is greater than the number of lines not passing through spatial units containing real-world point clouds, and the number of consecutive adjacent corresponding spatial units in front of the digital object reaches a preset number, the digital object is identified as an occluded digital object.

[0074] Furthermore, when the number of lines passing through spatial units containing real point clouds is no more than the number of lines not passing through spatial units containing real point clouds, it means that most of the observation path of the digital object is not blocked by real point clouds before reaching the digital object; when the number of lines passed through is more than the number of lines not passed through, but the number of consecutive adjacent corresponding spatial units in front of the digital object does not reach the preset number, it means that the real point clouds in front have not formed a continuous range that meets the occlusion requirements. For digital objects that do not meet the occlusion judgment conditions, they are identified as missing digital objects.

[0075] It should be noted that before identifying a digital object as a missing digital object, it should be confirmed that the spatial unit occupied by the digital object is within the effective acquisition range obtained in step one. If the main spatial unit occupied by a digital object is outside the effective acquisition range, it will not be identified as a missing digital object and can be left as an unresolved result, waiting to be checked from other point cloud acquisition locations.

[0076] For example, a digital object has one hundred spatial units that are within the effective acquisition range and do not have physical point clouds, corresponding to one hundred connecting lines; Seventy of the connections pass through spatial units containing real-world cloud features, while thirty of the connections do not pass through spatial units containing real-world cloud features. Of the seventy connecting lines, the largest group of consecutive adjacent spatial units contains sixty spatial units. If the preset quantity of the digital object is fifty, then the digital object is identified as an occluded digital object if it simultaneously satisfies the conditions of having a large number of connected lines and having a preset number of consecutive adjacent spatial units.

[0077] For example, if the number of connected lines corresponding to another digital object is also greater than the number of unconnected lines, but the first spatial units formed by the object form multiple smaller scattered areas, and the largest consecutive adjacent spatial unit group does not reach the preset number, then the real point cloud cannot indicate that there is continuous occlusion in front of the digital object, and the digital object is identified as a missing digital object.

[0078] Furthermore, when generating the judgment result, a corresponding relationship is established between the object identifier of the digital object, the point cloud collection location, the number of lines passing through the spatial unit containing the real point cloud, the number of lines not passing through the spatial unit containing the real point cloud, the number of consecutive adjacent corresponding spatial units in front of the digital object, the preset number, and the occluded or missing digital objects. The results of the digital object determination include occluded digital objects and missing digital objects.

[0079] It should be noted that the determination of the occluded digital object in step three is based on the initial judgment result obtained from the point cloud acquisition position before the point cloud acquisition position was changed. Since continuous real-world point cloud data in front may coexist with missing digital objects, for example, if a digital object has been removed but other objects are placed in front of its original position, step four selects a new point cloud acquisition location based on the corresponding spatial units that are consecutively adjacent in front of the digital object, and verifies the occluded digital object.

[0080] After completing step three, we obtain the occluded digital object, the missing digital object, and the corresponding spatial units that are consecutively adjacent to the occluded digital object in front of it. The consecutive adjacent spatial units in front of the occluded digital object are used in step four to determine the new point cloud acquisition location.

[0081] In step S4, the point cloud acquisition position is changed, and a line is drawn from the new point cloud acquisition position to the spatial unit occupied by the occupied digital object. When the new line does not pass through the spatial unit containing the real point cloud, and there is still no real point cloud in the spatial unit occupied by the digital object, it is modified to be a missing digital object. The specific content includes: Based on the consecutive adjacent spatial units in front of the occluded digital object in step three, a new point cloud acquisition position is selected so that the predicted connection pointing to the digital object does not pass through the corresponding spatial unit as much as possible. The real point cloud is obtained from the new point cloud acquisition position, and a new connection is made to the spatial unit occupied by the occluded digital object. When the new connection does not pass through the spatial unit containing the real point cloud, and there is still no real point cloud in the spatial unit occupied by the digital object, the occluded digital object in step three is modified to a missing digital object.

[0082] It should be noted that step four modifies the judgment result of the digital object made in step three, but does not modify the geometric information of the digital object in the digital twin model, nor does it change the side length of the spatial unit used in step one. The modified digital object is kept in correspondence with the occluded digital object in step three through the object identifier.

[0083] Furthermore, the candidate point cloud acquisition location is determined based on the corresponding spatial units that are consecutively adjacent in front of the occluded digital object obtained in step three. The candidate point cloud acquisition location is located within the acquisition area that the point cloud acquisition device is allowed to enter, and the space unit occupied by the occluded digital object is within the effective acquisition distance and acquisition direction of the device; The data collection area can be determined based on the scene channel, the movable range of the device, or a pre-set set of data collection locations.

[0084] Furthermore, for each candidate point cloud acquisition location, a predicted connection is made to the spatial unit occupied by the occluded digital object, and the number of consecutive adjacent spatial units in front of the digital object through which the predicted connection passes is counted. Select the candidate point cloud acquisition location with the fewest consecutive adjacent corresponding spatial units that the predicted connection passes through in front of the digital object as the new point cloud acquisition location.

[0085] It should be noted that when selecting a new point cloud acquisition location, it is not required that the new point cloud acquisition location be at a fixed distance from the point cloud acquisition location before the change. Instead, it is required that the observation direction from the new point cloud acquisition location to the digital object can avoid consecutive adjacent corresponding spatial units in front of the digital object as much as possible. If multiple candidate point cloud acquisition locations can avoid consecutive adjacent corresponding spatial units in front of the digital object, a location that is closer to the digital object and in the middle of the normal measurement distance range of the device can be selected to reduce the impact of sparse point clouds at a distance on the verification results.

[0086] For example, when consecutive adjacent corresponding spatial units in front of a digital object are located directly in front of the digital object, candidate point cloud acquisition positions can be determined in the collectable area on the left or right side of the digital object, and the side through which the predicted connection to the digital object passes less through the corresponding spatial units can be selected. If consecutive adjacent spatial units in front of a digital object cover one side of the digital object, then the candidate point cloud acquisition location on the opposite side is selected.

[0087] Furthermore, at the new point cloud acquisition location, a real-world point cloud is acquired from the new point cloud acquisition location. The real-world point cloud acquired from the new point cloud acquisition location is processed in the same way as in step one, and the real-world point cloud acquired from the new point cloud acquisition location is aligned with the digital twin model. During alignment, the unchanged fixed structure is used, and the same position and orientation adjustments are made to the new point cloud acquisition location so that the new point cloud acquisition location, the real point cloud acquired from the new point cloud acquisition location, and the digital twin model are in the same coordinate system.

[0088] Furthermore, the real point cloud obtained from the new point cloud acquisition location is written into the same spatial cell according to the side length of the spatial cell determined in step one, and the spatial cell containing the real point cloud is re-determined according to the same real point number threshold. Step four does not change the spatial unit side length and the threshold for the number of real points to avoid the loss of comparability of the collection results before and after due to different judgment standards for spatial division or point cloud.

[0089] Furthermore, based on the new point cloud acquisition location, the observable surface of the occluded digital object facing the new point cloud acquisition location is re-determined, and the spatial unit corresponding to the observable surface is determined as the spatial unit occupied by the digital object. The observable surface needs to be redefined because the surface area of ​​the digital object facing the acquisition device may change after the point cloud acquisition location is changed.

[0090] It should be noted that the spatial unit occupied by the digital object should still be within the effective acquisition range corresponding to the new point cloud acquisition location; If a digital object is outside the effective acquisition distance or outside the acquisition direction at the new point cloud acquisition location, the real-world point cloud acquired this time cannot be used to modify the occluded digital object into a missing digital object, and other candidate point cloud acquisition locations should be selected again.

[0091] Furthermore, new connections are made from the new point cloud acquisition location to the spatial units occupied by the digital objects. Following the same method as in step two, it is checked whether the object passes through a spatial unit containing real point clouds before reaching the corresponding spatial unit.

[0092] Furthermore, based on the new connection check results, each new connection is confirmed to have passed through a spatial unit containing real-world point clouds before reaching the corresponding spatial unit.

[0093] It should be noted that in claim 1, "the new connection does not pass through the spatial unit containing the real point cloud" means that the new connection made from the new point cloud acquisition position to each spatial unit occupied by the occupied digital object does not pass through the spatial unit containing the real point cloud before reaching the corresponding spatial unit; if any new connection passes through the spatial unit containing the real point cloud, then this condition is not satisfied.

[0094] When the above conditions are met, the observation path between the new point cloud acquisition location and the occluded digital object is determined to be free from real point cloud occlusion.

[0095] Furthermore, check whether the occluded digital object appears as a real point cloud within the spatial unit occupied by the new point cloud acquisition location. When the number of real points within a spatial unit occupied by a digital object reaches the threshold for the number of real points in step one, it is determined that a cloud of real points exists within that spatial unit.

[0096] When the new connection does not pass through the spatial unit containing the real point cloud, and there is still no real point cloud in the spatial unit occupied by the digital object, it means that there is no obstruction on the observation path from the new point cloud acquisition position to the digital object, but the position where the digital object should have appeared has not yet obtained the real point cloud. In this case, the occluded digital object in step three is modified to a missing digital object.

[0097] It should be noted that the occluded digital object will only be modified to a missing digital object if the spatial unit occupied by the digital object is within the effective acquisition range of the new point cloud acquisition location, the new connection does not pass through the spatial unit containing the real point cloud, and there is still no real point cloud in the spatial unit occupied by the digital object. The above three conditions are used to exclude erroneous modifications caused by insufficient equipment distance, the observation direction not covering the target, or the new observation path still being obstructed.

[0098] When a new connection does not pass through a spatial unit containing a real point cloud, and a real point cloud appears in the spatial unit occupied by the digital object, it indicates that the digital object actually exists. The absence of a real point cloud in step three is caused by the occlusion of the real point cloud in front of the point cloud acquisition position before the point cloud acquisition position was changed. The occlusion judgment is maintained, and the appearance of the real point cloud is recorded as a confirmation of the judgment result in step three.

[0099] When any new connection still passes through a spatial unit containing real point cloud, and there is still no real point cloud in the spatial unit occupied by the digital object, it means that the new point cloud acquisition position cannot effectively observe the digital object. The determination of the occluded digital object in step three is maintained, and another new point cloud acquisition position is selected for verification based on the spatial unit containing real point cloud that the new connection first passes through.

[0100] For example, in step three, a server rack is identified as a digital object that is being obscured, and it is determined that there are consecutive adjacent corresponding space units in front of the server rack that reach a preset number. Step 4: Select the side of the server rack as the new point cloud acquisition location; If new connections made from the side do not pass through spatial units containing real point clouds, but there are still no real point clouds in the spatial unit occupied by the cabinet, then the cabinet is changed from an obscured digital object to a missing digital object.

[0101] For example, if a real point cloud appears in the space unit occupied by the cabinet after the data is collected from the side of the cabinet, it means that the cabinet still exists. The occlusion determination is maintained, and the point cloud collection position before the point cloud collection position is changed, the corresponding space units that are consecutively adjacent in front of the digital object, the new point cloud collection position, and the location where the real point cloud appears are recorded.

[0102] Furthermore, establish a record of digital object judgment results; The digital object judgment result record includes at least the object identifier, the occluded digital object determined in step three, the new point cloud acquisition location, the new connection check result in step four, and the missing digital object or the occlusion judgment determined in step four. When modifying an occluded digital object to a missing digital object, the corresponding digital object is found based on the object identifier, and no other digital object object is generated.

[0103] It should be noted that step four does not update the preset quantity in step three; The preset quantity is used to determine whether the corresponding spatial unit is consecutively adjacent in front of the digital object and reaches the preset quantity. Step four obtains a new real point cloud by changing the point cloud acquisition position, and uses the new real point cloud to determine whether the occluded digital object is modified into a missing digital object. Therefore, step four forms a verification of the occluded digital objects, rather than repeatedly adjusting the preset number in the same recognition process.

[0104] Through the above steps, Step 1 filters digital objects that exist in the digital twin model but whose occupied space unit does not contain real point clouds in the same spatial unit; Step 2 draws lines from the point cloud acquisition position to the spatial unit occupied by the digital object and obtains the spatial unit containing real point clouds that each line first passes through; Step 3 determines the occluded digital object based on the relationship between the number of lines that pass through and do not pass through spatial units containing real point clouds, and the case where the corresponding spatial units are consecutively adjacent in front of the digital object and reach a preset number; Step 4 changes the point cloud acquisition position, and if the new line does not pass through a spatial unit containing real point clouds and there are still no real point clouds in the spatial unit occupied by the digital object, it is modified to be the missing digital object.

[0105] This invention discloses a virtual-real scene difference recognition system, including an object filtering module, a connection checking module, an occluded object determination module, and a missing object modification module, with signal connections between the modules; The object filtering module is used to align the real point cloud with the digital twin model and divide it into the same spatial units, and filter digital objects that exist in the digital twin model but do not exist in the real point cloud within the spatial unit they occupy. The connection check module is used to determine the point cloud acquisition location when acquiring real point cloud. It draws a line from the point cloud acquisition location to the spatial unit occupied by each selected digital object, and checks whether the connection line passes through a spatial unit containing real point cloud before reaching the corresponding spatial unit. The occlusion object determination module is used to determine a digital object as an occluded digital object when the number of lines passing through spatial units containing real point clouds is greater than the number of lines not passing through spatial units containing real point clouds, and the corresponding spatial units are consecutively adjacent in front of the digital object and reach a preset number. The missing object modification module is used to change the point cloud acquisition position. A line is drawn from the new point cloud acquisition position to the spatial unit occupied by the occupied digital object. When the new line does not pass through the spatial unit containing real point cloud data, and there is still no real point cloud data in the spatial unit occupied by the digital object, it is modified to be a missing digital object.

[0106] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0107] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and inventive constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0108] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0109] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0110] In conclusion, the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for recognizing differences between virtual and real scenes, characterized in that, Includes steps; Align the real-world point cloud with the digital twin model and divide it into the same spatial units, then filter out digital objects that exist in the digital twin model but whose spatial units do not contain the real-world point cloud. Determine the point cloud acquisition location when acquiring real point cloud, draw lines from the point cloud acquisition location to the spatial unit occupied by each selected digital object, and check whether the spatial unit containing real point cloud is passed before reaching the corresponding spatial unit along each line. When the number of connections passing through spatial units containing real-world point clouds is greater than the number of connections not passing through spatial units containing real-world point clouds, and the corresponding spatial units are consecutively adjacent in front of the digital object and reach a preset number, the digital object is identified as an occluded digital object. Change the point cloud acquisition location, and draw a line from the new point cloud acquisition location to the spatial unit occupied by the occupied digital object. When the new line does not pass through the spatial unit containing real point cloud, and there is still no real point cloud in the spatial unit occupied by the digital object, modify it to the missing digital object.

2. The method for recognizing differences between virtual and real scenes according to claim 1, characterized in that, Aligning physical point-of-sight clouds with digital twin models, including: Fixed scene structures that correspond to each other and have not changed are selected from the real point cloud and the digital twin model respectively. The moving distance and rotation direction of the real point cloud are determined according to the positional differences between the fixed scene structures. The real point cloud is adjusted according to the moving distance and rotation direction, and the point cloud acquisition position is adjusted in the same position and direction to make the real point cloud, the point cloud acquisition position and the digital twin model in the same coordinate system.

3. The method for recognizing differences between virtual and real scenes according to claim 2, characterized in that, The real-world point cloud and digital twin model are divided into the same spatial units, and digital objects are selected, including: Determine the three-dimensional spatial range jointly covered by the real-world point cloud and the digital twin model, divide the three-dimensional spatial range according to the uniform spatial unit side length, and write the points in the real-world point cloud into the spatial unit in which they are located. Based on the point cloud acquisition location, determine the observable surface of the digital object facing the point cloud acquisition location, determine the spatial unit corresponding to the observable surface as the spatial unit occupied by the digital object, and filter digital objects whose occupied spatial units are within the effective acquisition range and do not have real point clouds.

4. The method for recognizing differences between virtual and real scenes according to claim 1, characterized in that, Draw lines from the point cloud acquisition location to the spatial units occupied by each selected digital object, including: For each spatial unit occupied by a selected digital object that does not contain a real point cloud, the center of the surface of the spatial unit facing the point cloud acquisition location is determined. The line segment between the point cloud acquisition location and the surface center is taken as the line connecting the spatial unit. Starting from the point cloud acquisition location, the spatial units that the line passes through are determined sequentially along the line until the spatial unit is reached.

5. The method for recognizing differences between virtual and real scenes according to claim 4, characterized in that, Check along each connecting line whether a spatial unit containing a real point cloud is passed before reaching the spatial unit connected by that line, including: Read whether each spatial unit traversed by the connection contains a real point cloud; When a line passes through a spatial unit containing a real point cloud before reaching the spatial unit it connects to, the line is identified as a line that passes through a spatial unit containing a real point cloud, and the first spatial unit containing a real point cloud that the line passes through is recorded as an occluding spatial unit. When a line passes through spatial units that do not contain real point clouds before reaching the spatial unit it connects, the line is determined to be a line that does not pass through spatial units containing real point clouds.

6. The method for recognizing differences between virtual and real scenes according to claim 1, characterized in that, Determining that occlusion spatial units are consecutively adjacent in front of the digital object includes: The occlusion space units on the surface of the shared space unit are defined as continuous adjacent. Based on the continuous adjacent relationship, the occlusion space units are divided into at least one group of continuous adjacent space units. The group of continuous adjacent space units containing the most space units is defined as the occlusion space units that are continuously adjacent in front of the digital object.

7. The method for recognizing differences between virtual and real scenes according to claim 6, characterized in that, The preset quantity is determined based on the number of space units occupied by the digital object and the preset continuous ratio. The preset continuous ratio is determined based on the coverage relationship between the consecutive adjacent occupying space units in front of the confirmed occupied digital object and the space units occupied by the digital object.

8. The method for recognizing differences between virtual and real scenes according to claim 1, characterized in that, Changing the point cloud acquisition location includes: Based on the consecutive adjacent occlusion space units in front of the occluded digital object, the candidate point cloud acquisition location is determined in the acquisition area where the point cloud acquisition device is allowed to enter and the space unit occupied by the occluded digital object is within the effective acquisition range. Predicted connections were made from each candidate point cloud acquisition location to the spatial unit occupied by the occluded digital object. The candidate point cloud acquisition location whose predicted connection passed through the fewest occluded spatial units was selected as the new point cloud acquisition location. Obtain real-world point cloud data from new point cloud acquisition locations, use the same spatial unit division and real-world point cloud existence judgment method, reconnect and check occluded digital objects.

9. A virtual-real scene difference recognition system, used to implement the virtual-real scene difference recognition method according to any one of claims 1-8, characterized in that, include: The object filtering module is used to align the real point cloud with the digital twin model and divide it into the same spatial units, and filter digital objects that exist in the digital twin model but do not exist in the real point cloud within the spatial unit they occupy. The connection check module is used to determine the point cloud acquisition location when acquiring real point cloud. It draws a line from the point cloud acquisition location to the spatial unit occupied by each selected digital object, and checks whether the connection line passes through a spatial unit containing real point cloud before reaching the corresponding spatial unit. The occlusion object determination module is used to determine a digital object as an occluded digital object when the number of lines passing through spatial units containing real point clouds is greater than the number of lines not passing through spatial units containing real point clouds, and the corresponding spatial units are consecutively adjacent in front of the digital object and reach a preset number. The missing object modification module is used to change the point cloud acquisition position. A line is drawn from the new point cloud acquisition position to the spatial unit occupied by the occupied digital object. When the new line does not pass through the spatial unit containing real point cloud data, and there is still no real point cloud data in the spatial unit occupied by the digital object, it is modified to be a missing digital object.

10. A virtual-real scene difference recognition device, used to implement the virtual-real scene difference recognition method according to any one of claims 1 to 8, characterized in that, It includes a point cloud acquisition device and a difference recognition and processing device; the point cloud acquisition device is used to acquire real-world point clouds of the target scene from the point cloud acquisition location; the difference recognition and processing device is used to align the real-world point clouds with the digital twin model and divide them into the same spatial units, determine the occluded digital objects based on the connection check results from the point cloud acquisition location to the spatial unit occupied by the digital object, and after changing the point cloud acquisition location, if the new connection does not pass through the spatial unit containing the real-world point clouds and there are still no real-world point clouds in the spatial unit occupied by the digital object, modify it to the missing digital object.

Images