Construction method of VR scene analysis based on three-dimensional digital visualization

Abstract

The application discloses a VR scene analysis construction method based on three-dimensional digital visualization, relates to the field of VR scene analysis construction, and comprises the following steps: profile boundary features of a target object are acquired, and an approaching condition judgment is performed on the profile boundary features in combination with a motion direction vector of a control end, so that the approaching judgment is applied to the geometric boundary of the surface of the target object; a transient speed change rate of the control end is analyzed, a space coordinate of an expected touch end point is calculated according to the transient speed change rate, and an interactive judgment tolerance region is dynamically adjusted according to the expected touch end point; a smooth compensation is performed on a jitter offset component in a continuous motion track, actual trigger coordinates after correction are generated, and the adjusted interactive judgment tolerance region is used as a touch judgment condition; the problems of false triggering or touch positioning deviation caused by the jitter of the control end in VR interaction are solved, and the stability of object grabbing operation in a virtual scene is improved.

Description

Technical Field

[0001] This invention belongs to the field of VR scene analysis and construction technology, and relates to a VR scene analysis and construction method based on three-dimensional digital visualization. Background Technology

[0002] VR technology creates immersive 3D digital scenes, enabling users to interact with virtual objects using natural hand movements. In VR interaction, users move in 3D space using control devices such as controllers or gesture sensors. The system collects the position trajectory of the control device in real time and determines whether it touches or grasps the target object, thus achieving an immersive interactive experience.

[0003] Several solutions for VR scene interaction control already exist in the existing technology. For example, the invention patent with authorization announcement number CN119536526B proposes an LBE-enhanced multimodal perception interactive VR display control method and system. This method collects the user's multimodal perception data through a VR headset and hand controller, uses a Kalman filter algorithm to fuse position data and motion data to calculate the user's continuous motion trajectory, and dynamically adjusts the range of motion based on the continuous motion trajectory.

[0004] Although the scheme demonstrates advantages in motion trajectory smoothing through multimodal data fusion and Kalman filtering, it still has the following shortcomings: First, the dynamic range adjustment of the existing technology is aimed at the user's range of activity and solves the problem of the user exceeding the physical space boundary, but it cannot solve the problem of the accuracy of touch judgment between the hand and the target object.

[0005] Secondly, existing technologies use Kalman filtering for data fusion and smoothing, which is a probability-based filtering method. Its smoothing effect depends on the prior statistical characteristics of system noise and measurement noise. It is slow to respond to drastic changes in hand movement during high-speed approach and has difficulty effectively capturing instantaneous speed changes and endpoint position shifts in the hand movement trajectory. When the user approaches a small target object at high speed, the fixed tolerance threshold cannot be adaptively adjusted with the movement speed. If the tolerance is set too high, it is easy to cause false triggering by adjacent objects. If the tolerance is set too low, it is easy to miss detection during high-speed approach.

[0006] Furthermore, when dealing with gesture sensors that have inherent noise, existing technologies may cause the target object to be mistakenly bounced away or cause high-frequency vibrations due to the instantaneous recognition delay or positional deviation at the moment of contact, thus disrupting the immersive experience of VR interaction. Summary of the Invention

[0007] In view of this, in order to solve the problems mentioned in the background art, the present invention provides a VR scene analysis and construction method based on three-dimensional digital visualization.

[0008] The objective of this invention can be achieved through the following technical solution: a VR scene analysis and construction method based on three-dimensional digital visualization, comprising: acquiring the spatial pose data and contour boundary features of the target object to be analyzed in the VR scene;

[0009] Real-time acquisition of the continuous motion trajectory of the control terminal in three-dimensional space, and analysis of the current motion direction vector and instantaneous velocity change rate;

[0010] The approach condition is determined by the interaction between the motion direction vector and the contour boundary features. If the approach condition is determined, the spatial coordinates of the expected touch point are calculated based on the instantaneous velocity change rate.

[0011] Smoothing compensation is performed on the jitter offset component caused by the jitter of the control end in the continuous motion trajectory to generate the corrected actual trigger coordinates, and the interaction judgment tolerance area in the scene construction is dynamically adjusted according to the expected touch endpoint.

[0012] When the corrected actual trigger coordinates fall into the adjusted interaction judgment tolerance area, a grab and lock operation is performed on the target object, and the approach parameters of this touch process are associated with the corrected parameters and stored in the scene construction attribute set, and the dynamic interaction attributes of the target object are updated.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention obtains the contour boundary features of the target object and combines the motion direction vector of the control end with the contour boundary features to make interactive approach condition judgment, so that the approach judgment acts on the geometric boundary of the target object surface, overcomes the defect of the prior art that only adjusts the safety boundary and cannot solve the problem of the accuracy of the touch judgment between the hand and the target object, and avoids false triggering caused by insufficient judgment basis in multi-target complex scenes.

[0014] (2) This invention analyzes the instantaneous velocity change rate of the control terminal, calculates the spatial coordinates of the expected touch endpoint based on the instantaneous velocity change rate, and dynamically adjusts the interactive judgment tolerance area according to the expected touch endpoint. This overcomes the shortcomings of existing Kalman filters, such as lag in response when approaching at high speed and the inability of fixed tolerance to adapt to different motion speeds. At high speed, the tolerance is adaptively expanded to cover the endpoint prediction deviation, and at low speed, the tolerance is adaptively reduced to avoid false triggering by adjacent objects, thus achieving adaptive matching between the judgment tolerance and the motion state of the control terminal.

[0015] (3) This invention generates corrected actual trigger coordinates by performing smooth compensation on the jitter offset component caused by the jitter of the control terminal in the continuous motion trajectory, and uses the adjusted interaction judgment tolerance area as the access condition for touch judgment. This overcomes the defect that the target object is mistakenly bounced away or high-frequency vibration is caused by the judgment delay or position deviation at the moment of contact. The corrected actual trigger coordinates are close to the theoretical touch position of the control terminal under the condition of no jitter noise, avoiding interaction failure caused by noise interference or judgment deviation, and improving the immersiveness of VR scene interaction. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a diagram illustrating the implementation steps of the method of the present invention;

[0018] Figure 2 This is a flowchart of the VR scene interaction proximity condition judgment process of the present invention;

[0019] Figure 3 This is a flowchart illustrating the dynamic adjustment process of the interactive determination tolerance area in this invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figure 1 As shown, the present invention provides a VR scene analysis and construction method based on three-dimensional digital visualization, including: S1, acquiring the spatial pose data and contour boundary features of the target object to be analyzed in the VR scene.

[0022] Considering that the interaction between the control terminal and the target object depends on the actual geometric intersection between the control terminal's motion trajectory and the target object's outer surface in three-dimensional space, rather than an approximation of a center point or bounding box, it is necessary to pre-extract the target object's spatial pose data and contour boundary features to establish an interaction judgment basis that conforms to geometric constraints. Based on this, the aforementioned target object's spatial pose data and contour boundary features include: S1-1, acquiring a discrete set of points on the target object's outer surface in space from multiple orientations;

[0023] S1-2. Separate the point groups belonging to the target object from the discrete point set according to their spatial location, and take the outer surface points of these point groups as the object point cloud.

[0024] S1-3. Identify the locations where depth values ​​change along each scanning direction in the object point cloud, connect the change locations to form a boundary line, and simultaneously calculate the local normal vector change of each point in the object point cloud. Enclose the contour line formed by the points whose normal vector change exceeds the average normal vector change of the object point cloud, and together with the boundary line, constitute the contour boundary feature of the target object.

[0025] S1-4. Unify the object point cloud to the VR scene reference coordinate system, extract the geometric center of the object point cloud as the position of the target object, and extract the main extension direction of the object point cloud as the orientation of the target object to form spatial pose data.

[0026] In step S1-1, since the back face or self-occluded area of ​​the target object in the VR scene cannot be observed from a single viewpoint, according to the viewpoint complementarity principle of spatial geometry, data must be fused from multiple directions to eliminate blind spots and reconstruct the complete outer surface geometry.

[0027] The depth acquisition device employs a lidar scanner, capable of projecting pulsed laser beams onto the target object and calculating the distance between the device and various points on the target object's outer surface by receiving reflected signals, generating a 3D point cloud with the device's coordinate system as a reference. The scanning orientation is configured as follows: the target object is placed on a scanning turntable, which is then rotated in equal angular increments. Each preset angle increment triggers the depth acquisition device to perform a scan; preferably, the turntable rotation angle increment is 15°–30°.

[0028] The single scan execution process is as follows: at each acquisition point, the depth acquisition device is activated to project and emit laser pulses toward the target object, and the echo signal reflected by the outer surface of the target object is received by the receiver. Based on the time difference between emission and reception, the depth value of each sampling point on the outer surface of the target object in the device coordinate system is calculated, and local point cloud data under the location is generated.

[0029] In steps S1-2, the discrete point set acquired by the depth acquisition device during the scanning process inevitably includes background environment points, support platform points, and stray noise points caused by multiple reflections, in addition to sampling points on the outer surface of the target object. Therefore, a clustering segmentation algorithm based on Euclidean distance is used to process the original discrete point set. Specifically, a spatial clustering radius threshold r is set, and each point in the original discrete point set is traversed. A spherical neighborhood is constructed with the current point as the center and r as the radius. Points with a spatial distance less than r from the current point are grouped into the same cluster. The above operation is recursively performed until all points are clustered, resulting in multiple clusters that are spatially separated from each other. Preferably, the clustering radius threshold r is 2 to 4 times the average point spacing, which is obtained by calculating the arithmetic mean of the Euclidean distances between each point in the original discrete point set and its nearest neighbor.

[0030] For discrete point sets with sparse spatial distribution or uneven density, density-based noise spatial clustering is used instead of Euclidean distance clustering. Points with density connections are assigned to the same cluster, and isolated points that cannot be assigned to any cluster are marked as noise points and removed.

[0031] After obtaining the complete point group corresponding to the target object through clustering, all points in the point group are used as the object's point cloud. If the discrete point set contains stray points located inside the target object due to multiple reflections or transmissions, they are removed as follows: the local neighborhood point density of each point in the point group is calculated, and points whose local neighborhood point density is lower than a preset multiple of the average neighborhood point density are marked as internal stray points and removed. Preferably, the preset multiple is 0.5 times, that is, points whose local neighborhood point density is lower than half of the average neighborhood point density are determined to be internal stray points.

[0032] The calculation method for the aforementioned local neighborhood point density is as follows: For each point in the point group, the number of neighboring points within a preset radius is obtained, and this number is used as the local neighborhood point density of each point. This preset radius is the same as the clustering radius threshold r.

[0033] The location where the depth value jump occurs in S1-3 above is identified as follows: the depth value of each sampling point is read sequentially along each scanning direction, the depth difference between adjacent points is calculated, and the position of the point pair where the depth difference between adjacent points is greater than the depth threshold is marked as the depth jump position. The depth threshold is preferably 3 to 5 times the average depth difference of all adjacent points in the scanning direction in the object point cloud, and false jumps caused by slight surface undulations are filtered out.

[0034] The depth transition boundary line and the high curvature contour line are spatially superimposed in the VR scene reference coordinate system, and the geometric union of the two is taken as the contour boundary feature of the target object.

[0035] The calculation process for the change in local normal vector of each point in the object point cloud is as follows: obtain the neighborhood point set of each point in the object point cloud within the clustering radius threshold range, perform principal component analysis on the neighborhood point set, and take the eigenvector corresponding to the smallest eigenvalue as the local normal vector of each point; traverse all points in the object point cloud in sequence to obtain the local normal vector sequence of each point.

[0036] The steps in S1-4 above to unify the object point cloud to the VR scene reference coordinate system are as follows: At each acquisition point of the depth acquisition device, the rotation angle of the scanning turntable and the spatial pose of the depth acquisition device in the VR scene reference coordinate system are recorded simultaneously; based on the rotation angle and spatial pose, a rigid body transformation matrix from the device coordinate system to the VR scene reference coordinate system at each acquisition point is constructed; the local point cloud data generated at each acquisition point is multiplied by the corresponding rigid body transformation matrix to obtain the global point cloud data unified to the VR scene reference coordinate system.

[0037] The steps for determining the principal extension direction of the object point cloud are as follows: Calculate the geometric center of all points in the object point cloud, subtract the coordinates of the geometric center from the coordinates of each point to obtain a decentralized point cloud, construct the covariance matrix of the decentralized point cloud, and perform eigenvalue decomposition on the covariance matrix to obtain three eigenvalues ​​and their corresponding three eigenvectors. The direction of the eigenvector corresponding to the largest eigenvalue is taken as the principal extension direction of the object point cloud.

[0038] S2. Real-time acquisition of the continuous motion trajectory of the control terminal in three-dimensional space, and analysis of the current motion direction vector and instantaneous velocity change rate.

[0039] Since the continuous motion of the control terminal in three-dimensional space has inertia, the current direction of motion determines the target direction of the interaction approach, while the instantaneous rate of change of velocity reflects the acceleration and deceleration trend of the control terminal. These two together constitute the kinematic basis for predicting the spatial position of the control terminal at the next moment.

[0040] Therefore, the above analysis of the current motion direction vector and instantaneous velocity change rate is as follows: the position coordinates of the control terminal in three-dimensional space are continuously acquired at a fixed time step, forming a position sequence arranged in time order.

[0041] The control terminal is a VR interactive controller or a user hand tracking device.

[0042] The fixed time step is set based on the sampling performance of the spatial positioning system. Different types of positioning systems have different sampling frequencies, and the corresponding sampling period is the fixed time step. The sampling frequency of optical positioning systems is usually 60Hz to 120Hz, corresponding to a time step of 8.33ms to 16.67ms; the sampling frequency of inertial positioning systems is usually 100Hz to 200Hz, corresponding to a time step of 5ms to 10ms.

[0043] The displacement vector within each time step is obtained by subtracting the vectors of two adjacent positions in the position sequence, and the direction of the displacement vector at the current time step is taken as the current motion direction vector.

[0044] Displacement vectors for multiple time steps are extracted consecutively. Vector difference is performed on the displacement vectors of adjacent time steps and divided by the time step to obtain the instantaneous rate of change of velocity.

[0045] Among them, the number of continuously extracted displacement vectors shall not be less than 3, preferably 5 to 10.

[0046] The first-order difference of the displacement vector with respect to time is the velocity vector. Taking the first-order difference of the velocity vector yields the acceleration vector, which is the instantaneous rate of change of velocity, representing the speed and direction of the velocity change at the current moment. A larger instantaneous rate of change of velocity indicates a more drastic change in the motion state of the control unit, and a greater prediction deviation in the subsequent contact with the endpoint.

[0047] S3. Based on the motion direction vector and contour boundary features, perform interactive approach condition judgment. If it is determined to be an approach state, calculate the expected spatial coordinates of the endpoint based on the instantaneous velocity change rate.

[0048] Given that the direction of movement of the control end is pointing towards the target area and the spatial distance between it and the target surface is shortening, if the direction is pointing but the distance is not shortening, it is considered that the direction is accidentally aligned; if the distance is shortening but the direction is not pointing towards the target, it is considered that the object is brushed past.

[0049] refer to Figure 2 As shown, the above-mentioned interactive approach condition judgment based on the motion direction vector and contour boundary features is implemented as follows: calculate the angle between the current motion direction vector of the control terminal and the center direction of the interactive perception area of ​​each target object.

[0050] The interactive sensing area refers to a spherical space constructed with the geometric center of the target object as its center and a preset sensing distance as its radius. The preset sensing distance is no less than half the diagonal length of the target object's bounding box, ensuring that the control terminal is included in the proximity determination when it enters the effective interactive range around the target object.

[0051] The center direction of the interactive sensing area refers to the vector constructed from the geometric center of each target object to the current position of the control terminal. The direction of this vector is the center direction of the interactive sensing area of ​​that target object.

[0052] The specific calculation method for the included angle is as follows: the current motion direction vector of the control terminal and the center direction vector of the interactive perception area of ​​each target object are normalized respectively. Specifically, the cosine value of the included angle is calculated by the vector dot product formula, and then the included angle value is obtained by the inverse cosine function.

[0053] Compare the changing trends of the shortest spatial distance from the control terminal to the contour boundary of each target object over multiple consecutive sampling periods.

[0054] The target object with the smallest included angle and whose shortest spatial distance shows a continuously decreasing trend is identified as the current interaction approach object.

[0055] The aforementioned continuous decreasing trend is manifested as follows: The shortest spatial distance from the control terminal to the target object's outline boundary is calculated for the current sampling period and several consecutive sampling periods prior to it. If the distance value in the current period is less than the distance value in the previous period, and this decreasing relationship holds true for several consecutive periods, then it is determined to be a continuous decreasing trend. Preferably, the number of consecutive periods is 3 to 5.

[0056] The smallest directional angle indicates that the direction of movement of the control terminal is most directly aligned with the target object among all target objects. The continuously decreasing shortest spatial distance indicates that the control terminal is approaching the surface of the target object, rather than just being directionally aligned but actually moving away. At this point, both the directional and distance dimensions converge, thus accurately locating the target object that the user is actually interacting with in complex multi-object scenarios.

[0057] Because the control end does not move at a constant speed during the approach process, the existence of acceleration causes the touch-off point calculated based solely on the current speed to lag behind the actual landing point during acceleration and to advance ahead of the actual landing point during deceleration.

[0058] Therefore, the spatial coordinates of the expected endpoint are calculated based on the instantaneous rate of change of velocity as follows: the current spatial position of the control terminal is taken as the starting point, and the current motion direction vector is taken as the direction of contact propulsion.

[0059] The sampling interval between two consecutive time steps is obtained. The instantaneous velocity change rate is multiplied by the sampling interval to obtain the velocity compensation amount, which is added to the current motion velocity vector to obtain the predicted velocity vector. The current motion velocity vector is obtained by dividing the displacement vector of the current time step by a fixed time step.

[0060] A prediction ray is generated along the direction of the prediction velocity vector from the starting point. The spatial intersection of the prediction ray and the approaching action surface is calculated. If there are multiple intersections, the intersection closest to the touch start reference point is selected as the target intersection point. The coordinates of the target intersection point are used as the spatial coordinates of the expected touch endpoint.

[0061] The approaching action surface is taken from the outer surface area of ​​the target object contour boundary feature where the normal vector points to the control end. Based on the current position of the control end, the angle between the normal vector of each facet in the object point cloud and the direction vector from the facet to the control end is compared, and the facet with the angle less than 90° is defined as the approaching action surface.

[0062] The predicted ray has multiple intersection points with the approaching surface, meaning the ray penetrates the uneven surface or passes through multiple facets successively.

[0063] If the predicted ray does not intersect with the approaching action surface, determine whether this is the first time entering the approaching state. If so, take the projection point of the current spatial position of the control terminal along the current motion direction vector on the approaching action surface as the initial value of the expected touch endpoint and continue to execute the subsequent steps. If not, determine that a valid expected touch endpoint cannot be calculated in the current approaching state, keep the expected touch endpoint of the previous moment unchanged, and wait for the recalculation of the next time step.

[0064] The first intersection of the predicted ray and the approaching surface is the target surface position that the control end will first touch along the current predicted trajectory. This target surface position is the reference point for subsequent dynamic tolerance area adjustments. Selecting the intersection point closest to the touch start reference point is to lock the actual first surface position that the control end touches when multiple intersection points are generated after the ray penetrates the surface, rather than the back or internal intersection point after penetration, ensuring that the touch endpoint conforms to the geometry of the first arrival.

[0065] If the expected endpoint cannot be calculated for 3 to 5 consecutive time steps, the current approach state is terminated, and the expected endpoint is reset to the projection point of the current spatial position of the control terminal on the approach action surface.

[0066] S4. Perform smoothing compensation on the jitter offset component caused by the jitter of the control end in the continuous motion trajectory, generate the corrected actual trigger coordinates, and dynamically adjust the interaction judgment tolerance area in the scene construction according to the expected touch endpoint.

[0067] Considering the physiological tremors of the human body and the jitter offset component caused by the measurement noise of the positioning system, if the original coordinates are used directly for subsequent judgment, the tolerance area will fluctuate repeatedly or even fail to be captured.

[0068] Based on this, the generation of the corrected actual trigger coordinates is implemented as follows: extract the most recent number of sampling points before the current moment from the continuous motion trajectory to form a trajectory sequence to be smoothed, identify the spatial position of each sampling point in the trajectory sequence to be smoothed, and mark the sampling points whose spatial position deviates from the fitted trend line by more than a preset threshold and whose change direction is inconsistent with the current motion direction as jitter offset points.

[0069] The most recent number of sampling points is preferably 10 to 20 sampling points, covering at least one complete jitter cycle and not exceeding the time window of the effective motion trend.

[0070] The fitting trend line is constructed as follows: for all sampling points in the trajectory sequence to be smoothed, the least squares method is used to perform cubic polynomial fitting along the time axis to obtain a parameterized fitting curve with time as the independent variable and spatial three-dimensional coordinates as the dependent variable. The fitting trend line is the sequence of tangent directions of the parameterized curve within the current time window.

[0071] The preset threshold is determined as follows: for each sampling point in the trajectory sequence to be smoothed, calculate its vertical distance to the fitted trend line, calculate the arithmetic mean of all vertical distances, and use a multiple of this average value as the preset threshold, preferably 2 to 3 times.

[0072] The jitter offset point is separated from the trajectory sequence to be smoothed, and linear interpolation is performed based on the position coordinates of the adjacent normal sampling points before and after the jitter offset point to obtain a smooth trajectory sequence.

[0073] For a sampling point marked as a jitter offset point, obtain its position coordinates and timestamps of the preceding and following normal sampling points in the trajectory sequence to be smoothed. Based on the time ratio between the timestamp of the jitter offset point and the timestamps of the preceding and following normal sampling points, take values ​​proportionally between the corresponding coordinate values ​​of the preceding and following normal sampling points along each spatial coordinate axis. Combine the values ​​taken in each coordinate axis direction to form the alternative position coordinates of the jitter offset point.

[0074] If there are multiple consecutive jitter offset points, then for each jitter offset point, take its preceding and following normal sampling points, and calculate the replacement position coordinates independently according to its timestamp; if there are no normal sampling points before the start of the trajectory sequence to be smoothed, or no normal sampling points after the end of the sequence, then discard the jitter offset point directly without interpolation.

[0075] Calculate the displacement vector of adjacent sampling points in the first smooth trajectory sequence. If the angle between the displacement vector direction and the current motion direction vector exceeds the allowable range, mark the sampling point corresponding to the displacement vector as a residual jitter point, and perform linear interpolation replacement again to obtain the second smooth trajectory sequence.

[0076] After one smoothing process, the jitter points that deviate from the fitted trend line in spatial location have been replaced. However, the displacement vector direction of some residual jitter points with small spatial location deviations may still deviate from the current main motion direction. The angle between the displacement vector direction and the current motion direction vector reflects the degree of consistency between the sampling point's motion direction and the overall trend direction. An angle exceeding the allowable range indicates that the sampling point's motion direction deviates from the main motion direction, which is consistent with the randomness characteristic of jitter noise in direction. Therefore, residual jitter points are identified.

[0077] The allowable range is determined as follows: for each point in the trajectory sequence to be smoothed that has been identified as a normal sampling point, calculate the angle between its displacement vector and the current motion direction vector, calculate the arithmetic mean of all angle values, and set the allowable range based on the average value. The boundary value of the allowable range is 2 to 3 times the average value, and the upper limit is no more than 90°.

[0078] Extract the position coordinates of the sampling point closest to the current time in the quadratic smooth trajectory sequence, and use them as the corrected actual trigger coordinates.

[0079] The sampling point closest to the current moment in the secondary smoothed trajectory sequence is the latest position estimate after jitter removal and interpolation correction. Among all available data points, it is the closest to the current moment in time and best represents the current position of the control terminal.

[0080] The time coefficient represents the length of the time window from the current moment to the expected moment of touch. Preferably, the value of the time coefficient is related to the sampling period of the spatial positioning system and is set to 5 to 30 times the sampling period, and is not less than 1 / 10 of the shortest time required for the control terminal to go from entering the interactive sensing area to touching the target surface.

[0081] refer to Figure 3 As shown, the specific steps for dynamically adjusting the interaction judgment tolerance area in the scene construction according to the expected touch endpoint are as follows: taking the spatial coordinates of the expected touch endpoint on the approaching action surface as the reference, the dynamic expansion radius is set by multiplying the absolute value of the current instantaneous velocity change rate by the square of the time coefficient, and expanding outward along the action surface direction to obtain the initial interaction judgment tolerance area.

[0082] The larger the absolute value of the instantaneous velocity change rate, the more drastic the change in the motion state of the control end. The greater the possibility that the actual touch position deviates from the expected touch endpoint within the prediction time window, the more the tolerance area needs to be expanded accordingly to cover the possible touch deviation, so that the tolerance area can be adaptively adjusted according to the intensity of the motion of the control end.

[0083] Take the current and previous correction trigger coordinates from the quadratic smooth trajectory sequence, and calculate the real-time deviation vector between the current correction trigger coordinates and the expected endpoint.

[0084] The center of the initial interactive judgment tolerance region is shifted equally along the direction of the real-time deviation vector.

[0085] If the translated region exceeds the geometric boundary line of the approaching surface, it is checked whether the region exceeds the boundary line and is cut off along the normal direction of the approaching surface. If the area of ​​the region after cutting is less than the minimum area threshold, the initial interaction judgment tolerance area is retained as the interaction judgment tolerance area.

[0086] The tolerance area must always be within the outer surface of the target object. The part that exceeds the boundary line is in the void area. Touching the void area does not actually touch the object surface, so the part that exceeds the boundary needs to be trimmed to prevent grabbing in the air. At the same time, if the trimmed area is smaller than the minimum area threshold, it is considered that the effective area is too small and the user cannot trigger it naturally. Therefore, the current area is maintained to ensure the operability of the interaction.

[0087] The process for detecting whether the translated region exceeds the geometric boundary line of the approaching surface is as follows: Draw a ray horizontally from the point to be determined, and calculate the number of intersections between this ray and the geometric boundary line. If the number of intersections is odd, the point is located inside the boundary line; if it is even, the point is located outside the boundary line. If the point to be determined is on the boundary line, it is directly marked as a boundary point. If any discrete point is found to be outside the boundary line during the detection process, a clipping operation is triggered.

[0088] Preferably, the minimum area threshold is set based on the projected area of ​​the target object on the approach surface, taking 15% to 30% of the projected area, and not lower than the lower limit of the area corresponding to the positioning error tolerance range of the spatial positioning system.

[0089] If the area of ​​the initial interaction judgment tolerance region is less than the minimum area threshold, then a new circular area with the minimum area threshold as the radius and the expected touch endpoint as the center is used as the interaction judgment tolerance region.

[0090] S5. When the corrected actual trigger coordinates fall into the adjusted interaction judgment tolerance area, perform a grab and lock operation on the target object, and store the approach parameters and correction parameters of this touch process in the scene construction attribute set, and update the dynamic interaction attributes of the target object.

[0091] The above-mentioned capture and locking operation on the target object is implemented as follows: if the corrected actual trigger coordinates are located inside or on the boundary of the interactive judgment tolerance area, then it falls into the interactive judgment tolerance area.

[0092] After confirming the landing, the spatial pose data of the target object in the local coordinate system is obtained. The current movement direction of the control end is determined as the grasping force direction, and the spatial coordinates of the expected touch point are determined as the grasping force point.

[0093] The direction of the grasping force determines the orientation constraint direction of the target object after it is grasped. The current movement direction of the control terminal is the direction of the force applied by the hand at the moment of contact. Using this direction as the grasping force direction conforms to the mechanical logic of the hand approaching an object and applying force in the physical world. The grasping force point determines the specific anchor point position of the control terminal and the target object's orientation binding. The expected touch endpoint is the position where the control terminal first contacts the target surface as calculated by the system. Using this as the grasping force point ensures that the binding constraint acts on the actual contact position rather than an arbitrary point, ensuring that the spatial position of the grasping lock is consistent with the physical touch point.

[0094] The motion degrees of freedom of the control end and the pose of the target object are bound at the point of force application. Rotation and translation are locked and constrained. When the subsequent motion direction of the control end is opposite to the direction of force application, the rotation and translation locking constraints are released.

[0095] The direction of the grasping force is the direction of movement of the control end when it approaches the target object, which physically corresponds to the direction of the hand pressing or gripping the object; when the subsequent movement direction of the control end is opposite to this direction, it indicates that the hand is retracting from the surface of the target object, at which point the grasping lock is released to allow the target object to be freed from the constraint of the control end.

[0096] The above-mentioned update of the dynamic interaction attributes of the target object is implemented as follows: create an interaction record entry for the target object, and write the approaching action surface identifier of the touch, the expected touch endpoint coordinates, the adjusted tolerance area boundary, and the corrected actual trigger coordinates according to the timestamp.

[0097] Using the expected endpoint coordinates as an index, the deviation vector between the expected endpoint coordinates and the corrected actual trigger coordinates is calibrated as a compensation parameter, and the capture lock status and timestamp are marked at the same time.

[0098] Considering the repetitive nature of deviations caused by system positioning errors and smoothing algorithm lag, and their relatively fixed direction within a short period, the inherent repetitive deviation direction of the system can be identified by statistically analyzing the deviation direction with the highest frequency of occurrence of each coordinate axis deviation component in the historical compensation parameters. The initial tolerance area for the next interaction is then pre-shifted along this direction to compensate for the deviation before it occurs. Therefore, after updating the dynamic interaction attributes of the target object, the following steps are also included: based on the compensation parameters of the most recent preset number of touches in the historical record, the mode of the deviation components in each coordinate axis direction is statistically analyzed as the main deviation direction, and the initial interaction judgment tolerance area for the next interaction is pre-shifted and adjusted.

[0099] If the preset number of times is too few, the statistical results are easily affected by single random deviations; if the number of times is too many, outdated historical information will be introduced and the current system status will not be reflected. The optimal number of times is 10 to 20.

[0100] The mode of each coordinate axis direction is obtained by statistically analyzing the deviation components of each coordinate axis direction and taking the deviation value with the highest frequency in each coordinate axis direction as the mode of that coordinate axis direction.

[0101] The principal deviation direction is the direction in which the moduli of the X, Y, and Z coordinate axes are combined into a three-dimensional deviation vector.

[0102] The specific steps of the above pre-offset adjustment are as follows: After the initial interaction judgment tolerance area is generated for the next interaction, the center of the initial interaction judgment tolerance area is translated as a whole along the main deviation direction. The absolute value of the mode of each coordinate axis direction is measured during the translation to obtain the initial interaction judgment tolerance area after pre-offset adjustment, and then handed over to the clipping and judgment process.

[0103] After releasing the rotation and translation lock constraints, clear the current interaction judgment tolerance area and the expected touch endpoint, and use the current position of the control terminal as the starting reference point again, waiting for the judgment of the next interaction approach condition.

[0104] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0105] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0106] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0107] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0108] Finally, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A VR scene analysis and construction method based on 3D digital visualization, characterized by: include: Acquire spatial pose data and contour boundary features of the target object to be analyzed in a VR scene; Real-time acquisition of the continuous motion trajectory of the control terminal in three-dimensional space, and analysis of the current motion direction vector and instantaneous velocity change rate; The approach condition is determined by the interaction between the motion direction vector and the contour boundary features. If the approach state is determined, the spatial coordinates of the expected touch point are calculated based on the instantaneous velocity change rate. Smoothing compensation is performed on the jitter offset component caused by the jitter of the control end in the continuous motion trajectory to generate the corrected actual trigger coordinates, and the interaction judgment tolerance area in the scene construction is dynamically adjusted according to the expected touch endpoint. When the corrected actual trigger coordinates fall into the adjusted interaction judgment tolerance area, the grab and lock operation on the target object is performed, and the approach parameters of this touch process are associated with the corrected parameters and stored in the scene construction attribute set, and the dynamic interaction attributes of the target object are updated. The step of determining the interactive convergence condition based on the motion direction vector and contour boundary features includes: Calculate the angle between the current motion direction vector of the control terminal and the center direction of the interactive perception area of ​​each target object; Compare the changing trends of the shortest spatial distance from the control terminal to the contour boundary of each target object over multiple consecutive sampling periods; The target object with the smallest included angle and whose shortest spatial distance shows a continuously decreasing trend is identified as the current interaction approach object; The calculation of the expected spatial coordinates of the endpoint based on the instantaneous rate of change of velocity includes: Starting from the current spatial position of the control terminal, the current direction of motion vector is used as the direction of contact and propulsion; The sampling interval between two consecutive time steps is obtained. The instantaneous velocity change rate is multiplied by the sampling interval to obtain the velocity compensation amount, which is then added to the current motion velocity vector to obtain the predicted velocity vector. A prediction ray is generated along the direction of the prediction velocity vector from the starting point. The spatial intersection of the prediction ray and the approaching action surface is calculated. If there are multiple intersections, the intersection closest to the touch start reference point is selected as the target intersection point. The coordinates of the target intersection point are used as the spatial coordinates of the expected touch endpoint. The generation of the corrected actual trigger coordinates is implemented as follows: Extract the most recent number of sampling points before the current moment from the continuous motion trajectory to form a trajectory sequence to be smoothed. Identify the spatial position of each sampling point in the trajectory sequence to be smoothed. Mark the sampling points whose spatial position deviates from the fitted trend line by more than a preset threshold and whose change direction is inconsistent with the current motion direction as jitter offset points. Separate the jitter offset points from the trajectory sequence to be smoothed, and perform linear interpolation based on the position coordinates of the adjacent normal sampling points before and after the jitter offset points to obtain a smoothed trajectory sequence. Calculate the displacement vector of adjacent sampling points in the first smooth trajectory sequence. If the angle between the displacement vector direction and the current motion direction vector exceeds the allowable range, mark the sampling point corresponding to the displacement vector as a residual jitter point, and perform linear interpolation replacement again to obtain the second smooth trajectory sequence. Extract the position coordinates of the sampling point closest to the current time in the quadratic smooth trajectory sequence, and use them as the corrected actual trigger coordinates; The dynamic adjustment of the interaction judgment tolerance area in the scene construction based on the expected touch endpoint includes: Based on the spatial coordinates of the expected endpoint on the approaching action surface, the dynamic expansion radius is set by multiplying the absolute value of the current instantaneous velocity change rate by the square of the time coefficient, and then expanding outward along the action surface to obtain the initial interaction judgment tolerance area. Take the current and previous correction trigger coordinates from the quadratic smooth trajectory sequence, and calculate the real-time deviation vector between the current correction trigger coordinates and the expected endpoint. The center of the initial interactive judgment tolerance region is shifted equally along the direction of the real-time deviation vector. If the translated region exceeds the geometric boundary line of the approaching surface, the excess part is cut off along the normal direction of the surface using the boundary line as the cutting edge. If the area of ​​the cut region is less than the minimum area threshold, the initial interaction judgment tolerance region is retained as the interaction judgment tolerance region. If the area of ​​the initial interaction judgment tolerance region is less than the minimum area threshold, then a new circular area with the minimum area threshold as the radius and the expected touch endpoint as the center is used as the interaction judgment tolerance region.

2. The VR scene analysis and construction method based on three-dimensional digital visualization according to claim 1, characterized in that: The spatial pose data and contour boundary features of the target object include: Collect a set of discrete points in space on the outer surface of the target object from multiple directions; In the discrete point set, the point groups belonging to the target object are separated according to their spatial location, and the outer surface points of these point groups are taken as the object point cloud. In the object point cloud, the locations where the depth value changes abruptly along each scanning direction are identified, and the abrupt locations are connected to form a boundary line. At the same time, the local normal vector change of each point in the object point cloud is calculated, and the contour line formed by enclosing the points whose normal vector change exceeds the average normal vector change of the object point cloud, together with the boundary line, constitutes the contour boundary feature of the target object. The object point cloud is unified under the VR scene reference coordinate system. The geometric center of the object point cloud is extracted as the position of the target object, and the main extension direction of the object point cloud is extracted as the orientation of the target object, thus forming spatial pose data.

3. The VR scene analysis and construction method based on three-dimensional digital visualization according to claim 1, characterized in that: The analysis of the current motion direction vector and instantaneous velocity change rate includes: The position coordinates of the control terminal in three-dimensional space are continuously acquired at fixed time steps to form a position sequence arranged in chronological order; The vector difference between two adjacent position coordinates in the position sequence is used to obtain the displacement vector within each time step, and the direction of the displacement vector at the current time step is used as the current motion direction vector. Displacement vectors for multiple time steps are extracted consecutively. Vector difference is performed on the displacement vectors of adjacent time steps and divided by the time step to obtain the instantaneous rate of change of velocity.

4. The VR scene analysis and construction method based on three-dimensional digital visualization according to claim 1, characterized in that: The process of performing the grabbing and locking operation on the target object includes: If the corrected actual trigger coordinates are located inside or on the boundary of the interactive judgment tolerance area, then they fall into the interactive judgment tolerance area. After confirming the landing, obtain the spatial pose data of the target object in the local coordinate system, determine the current movement direction of the control end as the grasping force direction, and determine the spatial coordinates of the expected touch point as the grasping force point. The motion degrees of freedom of the control end and the pose of the target object are bound at the point of force application. Rotation and translation are locked and constrained. When the subsequent motion direction of the control end is opposite to the direction of force application, the rotation and translation locking constraints are released.

5. The VR scene analysis and construction method based on three-dimensional digital visualization according to claim 1, characterized in that: The dynamic interactive properties of the target object are updated as follows: Create an interaction record entry for the target object, and write the approaching action surface identifier of the touch, the expected touch endpoint coordinates, the adjusted tolerance area boundary, and the corrected actual trigger coordinates according to the timestamp; Using the expected endpoint coordinates as an index, the deviation vector between the expected endpoint coordinates and the corrected actual trigger coordinates is calibrated as a compensation parameter, and the capture lock status and timestamp are marked at the same time.

6. The VR scene analysis and construction method based on three-dimensional digital visualization according to claim 5, characterized in that: After updating the dynamic interactive properties of the target object, the method further includes: Based on the compensation parameters of the most recent preset number of touches in the historical records, the mode of the deviation components in each coordinate axis direction is used as the main deviation direction, and the initial interaction judgment tolerance area is pre-offset adjusted for the next interaction.

Images