Collision point detection method, device, readable storage medium and program product

Abstract

A collision point detection method, device, readable storage medium and program product are provided herein. The method comprises: obtaining a depth texture, the depth texture having a two-dimensional texture space, the two-dimensional texture space comprising a plurality of texels, each texel being associated with a corresponding depth value; projecting a first ray to the two-dimensional texture space and determining a detection path according to a projection result; detecting, on the detection path, whether the first ray passes through an object surface according to a preset step, the object surface being represented by the depth values of the corresponding texels; in response to detecting the passing, obtaining a first coordinate and a second coordinate, the first coordinate being a coordinate of a first step point in the two-dimensional texture space, the second coordinate being a coordinate of a second step point in the two-dimensional texture space, the first step point and the second step point being two adjacent step points before and after the passing on the detection path; interpolating the first coordinate and the second coordinate, and determining a third coordinate according to an interpolation result, the third coordinate being a corresponding coordinate of a collision point in the two-dimensional texture space.

Description

Technical Field

[0001] This article relates to the field of computer technology, and in particular to a collision point detection method, device, readable storage medium, and program product. Background Technology

[0002] With the rapid development of Mixed Reality (MR) and spatial computing technologies, Extended Reality (XR) devices (such as Augmented Reality (AR) headsets, Virtual Reality (VR) devices, and Metaverse interactive terminals) have become core carriers of human-computer interaction. In mixed reality scenarios, extended reality devices need to acquire environmental depth textures in real time through depth sensors and perform environment raycasting based on these textures to enable interaction between virtual objects and the real environment. For example, in mixed reality applications, users need to accurately place virtual objects (such as furniture, tools, or interface elements) in the real environment, requiring raycasting to determine whether the virtual objects collide with physical surfaces such as the ground and walls; in gesture interaction, the device needs to identify the contact points of the user's gestures through raycasting and provide tactile feedback, etc.

[0003] However, the current collision point detection results have the problems of low accuracy and large error. Summary of the Invention

[0004] In some cases, a collision point detection method, apparatus, readable storage medium, and program product are provided.

[0005] In a first aspect, a collision point detection method is provided, comprising: acquiring a depth texture, the depth texture having a two-dimensional texture space, the two-dimensional texture space including a plurality of texels, each of the texels being associated with a corresponding depth value; projecting a first ray onto the two-dimensional texture space, and determining a detection path based on the projection result; detecting whether the first ray traverses an object surface at a preset step along the detection path, the object surface being characterized by the depth values ​​of a plurality of corresponding texels; in response to detecting traversal, acquiring a first coordinate and a second coordinate, the first coordinate being the coordinate of a first stepping point in the two-dimensional texture space, the second coordinate being the coordinate of a second stepping point in the two-dimensional texture space, the first stepping point and the second stepping point being two adjacent stepping points before and after traversal on the detection path; interpolating the first coordinate and the second coordinate, and determining a third coordinate based on the interpolation result, the third coordinate being the corresponding coordinate of the collision point in the two-dimensional texture space.

[0006] Secondly, a collision point detection device is provided, comprising: a first acquisition unit for acquiring a depth texture, the depth texture having a two-dimensional texture space, the two-dimensional texture space including a plurality of texels, each texel being associated with a corresponding depth value; a first projection unit for projecting a first ray onto the two-dimensional texture space and determining a detection path based on the projection result; a step detection unit for detecting whether the first ray crosses an object surface on the detection path according to a preset step, the object surface being characterized by the depth values ​​of a plurality of corresponding texels; a second acquisition unit for acquiring a first coordinate and a second coordinate in response to the detection of a crossing, the first coordinate being the coordinate of a first step point in the two-dimensional texture space, the second coordinate being the coordinate of a second step point in the two-dimensional texture space, the first step point and the second step point being two adjacent step points before and after the crossing on the detection path; and an interpolation unit for interpolating the first coordinate and the second coordinate and determining a third coordinate based on the interpolation result, the third coordinate being the corresponding coordinate of the collision point in the two-dimensional texture space.

[0007] Thirdly, an electronic device is provided, comprising: a processor and a memory; the memory storing computer-executable instructions; the processor executing the computer-executable instructions stored in the memory, causing the at least one processor to perform the method described in the first aspect above and various possible designs of the first aspect.

[0008] Fourthly, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, and when a processor executes the computer-executable instructions, the method described in the first aspect and various possible designs of the first aspect is implemented.

[0009] Fifthly, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described in the first aspect and various possible designs of the first aspect.

[0010] In some cases, traversal detection is performed by projecting rays onto the 2D texture space of the depth texture and performing discrete step-by-step detection along the detection path. Between two adjacent step points where a traversal occurs, the texture coordinates of the two points are interpolated to obtain the collision point coordinates in the texture space. This eliminates the need to continuously reduce the step interval, improving the detection accuracy of collision points and reducing errors while controlling computational load. The coordinates of the collision point in the texture space are continuous floating-point numbers, which helps to improve jitter or jagged edges that occur when the user's position or movement changes in low-resolution depth textures. Furthermore, traversal judgment of the object surface is achieved based on the depth values ​​of the depth texture texels, eliminating the need for original 3D model geometric data and reducing scene data loading and storage overhead. Additionally, the above scheme can obtain the third coordinate of the traversal point in the 2D texture space by interpolating the first and second coordinates, eliminating the need for iterative binary search and multiple texture sampling, thus reducing computational overhead. Attached Figure Description

[0011] To more clearly illustrate certain situations or technical solutions in the prior art, the following will briefly introduce the drawings required in some situations or in the description of the prior art. Obviously, the drawings described below are for some situations, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram illustrating the application scenarios of the collision point detection method provided in some situations;

[0013] Figure 2 This is a flowchart illustrating the collision point detection methods provided in some situations;

[0014] Figure 3 A flowchart illustrating collision point detection methods for other scenarios;

[0015] Figure 4 Provide interpolation diagrams for certain situations;

[0016] Figure 5 This is a schematic diagram illustrating the scenario-based collision point detection process for certain situations.

[0017] Figure 6 A structural block diagram of a collision point detection device provided in some situations;

[0018] Figure 7 This is a schematic diagram of the structure of an electronic device provided in some situations. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages clearer, the technical solutions in some scenarios will be clearly and completely described below with reference to the accompanying drawings. Obviously, the scenarios described are only a part, not all. Based on the content described in some scenarios, all other scenarios obtained by a person skilled in the art without inventive effort are within the scope of protection of this document.

[0020] In XR devices, sensors such as depth cameras and Time-of-Flight (ToF) ranging sensors can be used to scan the surrounding physical environment and generate an environmental depth map that digitally represents the real world. To achieve precise interaction between virtual objects and the real world, a virtual ray is typically emitted from the starting point of the ray (such as the user's hand or eye position), and the first object that intersects with the real environment, as well as the precise position and distance of that object, are detected.

[0021] In some cases, sampling points are selected at fixed intervals along a virtual ray in a virtual 3D space. Each sampling point is projected onto the environmental depth map, and the depth value corresponding to the projection position is read and compared with the depth of the sampling point itself on the ray. This approach cannot adaptively adjust the step size based on the distance from the sampling point to the ray's starting point: oversampling is prone to occur when the sampling point is close to the ray's starting point, resulting in wasted computational power; undersampling is prone to occur when the sampling point is far from the ray's starting point, posing a risk of missed object detection. Coarse-grained stepping can only determine the interval where the intersection point is located, requiring the introduction of binary search iteration to solve for the precise intersection point, incurring additional computational overhead. Furthermore, due to the nonlinear characteristics of perspective projection, the fixed 3D step size, after being mapped to the depth buffer, will produce interval distortion, making the hit detection accuracy unstable at different distances.

[0022] In some cases, multi-frame depth maps are fused to construct a 3D voxel model or a truncated signed distance function (TSDF) model, and ray intersection is performed based on the 3D voxel model. This method has high detection accuracy, but the computational latency is large, it relies on the pre-scanning process, and it is difficult to adapt to real-time interactive XR scenarios.

[0023] In some cases, a 3D virtual ray can be projected onto the 2D depth space of a depth texture. The ray then propagates along its path in this 2D depth space, moving pixel by pixel. Each step jumps to the next adjacent texel, sampling the depth value at the center of that texel and comparing it to the ray's depth at that location to determine if the ray has passed through the object's surface. This method ensures that each texel the ray passes through on the image is accessed only once, offering high computational efficiency and predictable computational overhead. It has been widely used in graphics rendering effects such as screen-space reflections.

[0024] However, the aforementioned scheme, by only detecting ray crossing events across the environment's depth map at integer texel boundaries, suffers from hit point accuracy limited by the depth texture resolution, failing to meet the precise fitting requirements of virtual objects to the real environment in XR scenes. Furthermore, due to this accuracy limitation, in scenes with low depth map resolution, the output is discrete texel numbers. While the ray path appears smooth and continuous on the image with slight user head or hand movements, crossing a pixel boundary instantly jumps the returned texel number to an adjacent texel, causing a sudden change in the corresponding world coordinates and resulting in visible jitter. When a ray sweeps obliquely across a tilted real surface, because the depth value recorded at each texel center is discrete, the midpoints of the output collision points are arranged in a stepped pattern along the pixel grid, failing to fit the originally smooth slope and exhibiting a noticeable jagged effect. In other words, with low depth texture resolution, jitter or jaggedness can occur when the user's position or movement changes.

[0025] To address the aforementioned issues, some depth image-based hit detection methods, when detecting a ray crossing an object's surface along its propagation path, interpolate the coordinates of two adjacent 2D depth texels before and after the crossing, and determine the collision point's coordinates based on the interpolation result. This improves collision point detection accuracy while reducing computational load. It also helps mitigate jitter or jagged edges that occur when the user's position or movement changes in low-resolution depth textures.

[0026] Figure 1 The diagram illustrates scenarios where collision point detection methods are applied in certain situations, such as... Figure 1 As shown, a user wears a head-mounted extended reality device 101, through which they can see virtual content superimposed on the real environment, forming a three-dimensional space 102. An application can run within the extended reality device 101. When the user interacts with objects in the real environment using their hand or a controller, the application generates a virtual ray, the first ray, starting from the position of the user's hand or controller in the three-dimensional space and extending along the user's pointing direction. This first ray is used to detect whether there is an object surface in the real environment in the user's pointing direction. The application determines the location of the collision point based on the interaction result between the first ray and the depth texture, and uses this collision point location as the basis for interaction. For example, the application can place a virtual object on the surface of a real object based on the collision point location, or trigger corresponding interactive feedback based on the collision point location. The application outputs the coordinates of the collision point in three-dimensional space for use by upper-layer applications. It is understood that the head-mounted extended reality device is only one example, and other types of extended reality devices may also be included. Figure 1 The head-mounted augmented reality device does not constitute a limitation on the augmented reality device to which collision point detection methods can be applied in some situations.

[0027] Please refer to Figure 2 , Figure 2 Here is a flowchart illustrating collision point detection methods for certain scenarios, such as... Figure 2 The method includes the following steps:

[0028] S201: Obtain the depth texture. The depth texture has a two-dimensional texture space, which includes multiple texels, each of which is associated with a corresponding depth value.

[0029] In some cases, the execution entity of the collision point detection method can be an application running in the extended reality device. In other cases, the execution entity can also be an application running in other electronic devices that can communicate with the extended reality device. The following description uses an application running in the extended reality device as the execution entity.

[0030] In some cases, prior to step S201 above, the method further includes the following steps:

[0031] A first coordinate system is constructed based on multiple environmental images, which are acquired by an image sensor located in the extended reality device.

[0032] A first environment image and depth texture are acquired, and a three-dimensional space is constructed in a first coordinate system based on the first environment image and depth texture; the depth texture is obtained from data detected by a depth sensor in an extended reality device.

[0033] In some contexts, an image sensor refers to a sensor mounted on an augmented reality device that is capable of capturing images of the environment. Image sensors can include devices such as cameras or webcams that capture images of visible or infrared light. Multi-frame environmental images refer to multiple environmental images captured by an image sensor at different times. These images reflect different perspectives of the same scene observed by the image sensor from different positions or orientations. In some cases, multi-frame environmental images can be a sequence of video frames continuously captured as the augmented reality device moves.

[0034] The first coordinate system refers to a three-dimensional spatial coordinate system constructed based on multiple frames of environmental images, used to describe the relative positional relationships between objects in the real environment. The first coordinate system has a fixed origin, direction axes, and scale units. In some cases, the first coordinate system can be a world coordinate system constructed by the executing entity using Visual Simultaneous Localization and Mapping (V-SLAM) technology. The origin of this coordinate system can be set to the initial position when the device starts up, and the direction axes are determined by the direction of gravity and the initial orientation of the device. In some cases, depth texture can be a depth map output by a depth sensor (such as a time-of-flight camera or a structured light depth camera). Depth texture can be depth data stored in a texture format and can include multiple texels. A texel is a basic element in two-dimensional texture space, short for texture pixel. Each texel has a unique texture coordinate in two-dimensional texture space. Each texel is associated with a corresponding depth value, which represents the distance from the physical point in the real world corresponding to that texel to a reference point (such as the optical center of the depth sensor, the user's viewpoint, etc.). In some cases, a texel can correspond to one or more points on a tabletop in a real-world environment, and the depth value associated with that texel represents the distance of that point or group of points to the depth sensor. In other cases, a texel can correspond to one or more points on a wall in a real-world environment. Each texel corresponds to a depth value that characterizes the distance between the point or group of points on the wall and the depth sensor.

[0035] Three-dimensional space refers to a 3D scene representation constructed in a first coordinate system by fusing visual information from multiple frames of environmental images and distance information from depth textures. This 3D space contains the geometric shape, position, and distance information of object surfaces in the environment, and can be used to describe the three-dimensional structure of the real environment. In some cases, 3D space can be a scene model composed of 3D point clouds. In other cases, 3D space can be a scene surface model composed of triangular meshes.

[0036] In some scenarios, when a user enters a real-world environment wearing an augmented reality (AR) device, the image sensors on the AR device begin acquiring environmental images based on received acquisition instructions. As the user's head moves or the AR device moves in space, the image sensors continuously acquire images of the real environment from different positions and postures, forming multiple frames of environmental images. The executing entity can then construct a first coordinate system based on these multiple frames of environmental images. In some cases, the executing entity can employ a Visual Simultaneous Localization and Mapping (V-SLAM) scheme to construct the first coordinate system. For example, the executing entity can extract feature points (such as corner points, edge points, or textured pixels) from each frame of the multi-frame environmental images and perform feature point matching between different frames. Through triangulation algorithms, the executing entity can calculate the three-dimensional spatial coordinates of the same feature point in the first coordinate system based on the differences in image position of the same feature point from different viewpoints. As the number of processed image frames increases, more and more feature points are mapped into the first coordinate system, forming a sparse three-dimensional feature point map. Simultaneously, during the construction of the first coordinate system, the aforementioned execution entity can also determine the spatial pose of the image sensor when acquiring each frame of image (i.e., the position and orientation of the image sensor in the first coordinate system). This spatial pose information is used to accurately map the depth values ​​in the depth texture to the first coordinate system subsequently.

[0037] In some cases, the construction of the first coordinate system can also incorporate data from the Inertial Measurement Unit (IMU) to improve positioning accuracy and robustness during large-scale motion or rapid rotation. The IMU can provide angular velocity and acceleration information, helping extended reality devices maintain stable tracking of the first coordinate system in scenarios with insufficient image features (such as when facing a white wall or rotating rapidly).

[0038] After establishing the first coordinate system, the executing entity can acquire a first environmental image and a depth texture. The first environmental image can be a frame of environmental image captured by an image sensor at a certain moment. The depth texture can be depth data captured by a depth sensor on an extended reality device at the same moment. The executing entity can construct a three-dimensional space in the first coordinate system based on the received first environmental image and depth texture. The three-dimensional space is used to describe the geometry, position, and distance information of the surface of objects in the real environment. For example, firstly, using the first environmental image, the spatial pose of the image sensor in the first coordinate system when acquiring the first environmental image is determined. This spatial pose can be determined by the device tracking information obtained when constructing the first coordinate system. Then, each pixel in the depth texture is mapped from the two-dimensional image coordinate system of the depth sensor to the first coordinate system. Specifically, for each texel in the depth texture, the executing entity can acquire its texel coordinates in the depth texture and the corresponding depth value of the texel. Based on the texel coordinates, the depth value, the intrinsic parameters of the depth sensor, and the spatial pose of the image sensor in the first coordinate system, the texel is transformed from the depth sensor coordinate system to the first coordinate system, thereby obtaining the three-dimensional spatial point corresponding to the texel. By performing the above mapping operation on multiple texels in the depth texture, the executing entity can generate a three-dimensional point cloud or a three-dimensional mesh, which constitutes a three-dimensional space.

[0039] In some cases, the executing entity can pre-construct and maintain a three-dimensional space, and then, after acquiring the first environment image and depth texture, incrementally update the constructed three-dimensional space using the environment image and depth texture data of the current frame. For example, it can supplement the point cloud data of newly observed areas or update the spatial information of existing areas.

[0040] By acquiring multiple frames of environmental images from an image sensor and constructing a first coordinate system based on these images, a global three-dimensional spatial reference base can be established. This first coordinate system provides a unified spatial positioning basis for subsequent three-dimensional space construction and collision point detection, thereby improving the consistency and reliability of spatial positioning to a certain extent.

[0041] The acquisition of depth texture in step S201 above is performed during collision point detection, involving the acquisition of multiple depth textures. In some cases, the execution entity can receive the current frame depth texture from the depth sensor. In other cases, the raw depth data output by the depth sensor may be non-linear. The execution entity can convert the raw depth data into linear physical depth values ​​according to the depth buffer parameters calibrated by the extended reality device, and then store the converted linear physical depth values ​​in texture format to form a depth texture. In yet another case, the resolution of the raw depth data output by the depth sensor may be high (e.g., 640×480). The execution entity can downsample this high-resolution depth data to a preset resolution (e.g., 128×128), and then store the downsampled depth data in texture format to form a depth texture. Downsampling can reduce the computational load of subsequent steps. In still other cases, the raw depth data output by the depth sensor may contain noise. The execution entity can filter the depth data (e.g., median filtering, bilateral filtering) to suppress noise, and then store the filtered depth data in texture format to form a depth texture.

[0042] In some cases, the depth texture space comprises multiple texels, each associated with a depth value representing the distance from the corresponding physical point in the real world to the depth sensor. For example, a texel with a depth value of 2.0 meters indicates that the physical point corresponding to that texel is 2.0 meters away from the depth sensor. The execution entity can locate the corresponding texel in the depth texture using texture coordinates and read the depth value associated with that texel. The position of a texel in the depth texture is uniquely identified by its texture coordinates, which are typically represented by two-dimensional coordinates (u, v), where u corresponds to the horizontal position and v corresponds to the vertical position. For example, in a 128×128 resolution depth texture, the texture coordinates u and v both range from 0 to 127. The texture coordinates (64, 64) point to the texel in the 64th column and 64th row of that depth texture.

[0043] In some cases, when an augmented reality device includes two depth sensors, one for the left eye and one for the right eye, the depth texture can include both left-eye and right-eye depth textures.

[0044] S202: Project the first ray into the two-dimensional texture space, and determine the detection path based on the projection result.

[0045] The first ray here is a virtual ray defined in three-dimensional space, used to detect its intersection with the object's surface. The first ray is defined by a starting point (hereinafter referred to as the second starting point) and a direction vector. In some cases, the starting point of the first ray can be the eye position of the wearer of the extended reality device, and the direction can be the wearer's line of sight. In other cases, the starting point of the first ray can be the position of the controller held by the user, and the direction can be the pointing direction of the controller.

[0046] In some cases, the executing entity can first determine the starting point (i.e., the second starting point hereinafter referred to as the second starting point) and the ending point (i.e., the second ending point hereinafter referred to as the second ending point) of the first ray in three-dimensional space. The second starting point can be, for example, the eye position of the user of the extended reality device, and the direction can be the user's line of sight. The second ending point can be obtained by extending the starting point along the direction by a preset maximum distance. Then, the executing entity can project the second starting point and the second ending point from the three-dimensional world space to the two-dimensional texture space, respectively, to obtain the first projection position of the second starting point in the two-dimensional texture space (i.e., the first starting point hereinafter referred to as the first starting point) and the second projection position of the second ending point in the two-dimensional texture space (i.e., the first ending point).

[0047] In some cases, when transforming a point in 3D space (e.g., a second starting point or a second ending point) from 3D world space to 2D texture space, the executing entity first transforms the point to homogeneous clip space using a view projection matrix to obtain homogeneous coordinates. Then, the executing entity performs perspective division on these homogeneous coordinates, dividing each component by a perspective factor to obtain normalized device coordinates. Subsequently, the executing entity maps these normalized device coordinates to texture coordinates in the range of 0 to 1, and then multiplies the texture coordinates by the resolution of the depth texture to obtain the floating-point position of the point in texture space. Finally, the executing entity rounds down this floating-point position to obtain the texel index of the point. Therefore, p0 and p1 are integer texture coordinates obtained by the executing entity after the above coordinate transformation process, pointing to the specific texel positions corresponding to the starting and ending points of the first ray, respectively.

[0048] The execution entity can determine the detection path based on the projection results. The detection path is the path formed by multiple texels traversed by the first ray in the two-dimensional texture space, including multiple texels from the first projection position to the second projection position. Assuming the starting point of the first ray's projection in the depth texture is texture coordinate (0,0) and the ending point is texture coordinate (10,5), then the detection path is the sequence of texels traversed from (0,0) to (10,5). The sequence of texels on the detection path determines the range and direction of the step detection. The detection range includes the first starting point and the first ending point of the detection path. The first starting point is the first texel position visited on the detection path; the first ending point is the last texel position visited on the detection path. The direction of the detection path is the approximate direction from the starting point to the ending point.

[0049] S203: On the detection path, the first ray is detected to see if it crosses the object surface according to the preset step. The object surface is represented by the depth values ​​of multiple corresponding pixels.

[0050] An object surface refers to the surface of an object in the aforementioned three-dimensional space, which is represented by the depth values ​​of multiple corresponding texels in the depth texture space. In some cases, the object surface can be a desktop or wall surface in a real environment. This surface information is encoded in the depth values.

[0051] The executing entity can start from the first starting point and advance step by step according to the preset steps. At each step point, the executing entity can perform a collision detection operation.

[0052] In some cases, step S203 above, which involves detecting whether the first ray passes through the object surface according to a preset step, includes:

[0053] Starting from the first starting point, multiple step points are determined on the detection path according to a preset step; the first starting point is the starting point of the detection path;

[0054] For each step point, obtain the fifth depth value and the sixth depth value. The fifth depth value is the depth value of the first ray at the step point, and the sixth depth value is the depth value of the depth texture at the step point.

[0055] Compare the fifth and sixth depth values;

[0056] In response to the detection of a flip in the comparison results of two adjacent step points, it is determined that the first ray has crossed.

[0057] In some cases, the executing entity can start from the first starting point (the starting point of the detection path) and determine multiple step points on the detection path according to a preset step. The executing entity can advance step by step on the detection path according to the preset step, determining a step point at each position reached, until the first end point of the detection path is reached, thus obtaining multiple step points on the detection path.

[0058] In some cases, the preset step can be one texel per step along the detection path. At each step point, the execution entity acquires the fifth and sixth depth values ​​corresponding to that step point. The fifth depth value is the depth value of the corresponding point on the first ray, i.e., the three-dimensional spatial distance from the corresponding point on the first ray to the reference point. The sixth depth value is the depth value of the depth texture at that step point. After acquiring the fifth and sixth depth values ​​respectively, the execution entity compares their magnitudes.

[0059] In some cases, the executing entity compares the fifth and sixth depth values, yielding three possible results: if the fifth depth value is less than the sixth depth value, it indicates that the first ray is in front of the object surface at that step point; if the fifth depth value is greater than the sixth depth value, it indicates that the first ray is behind the object surface at that step point; if the fifth depth value is equal to the sixth depth value, it indicates that the first ray is exactly on the object surface at that step point. If the fifth depth value is equal to the sixth depth value, a crossing can be confirmed.

[0060] In some cases, as the executing entity advances along the detection path, it can record the comparison results at each step. When the executing entity detects a reversal in the comparison results between two adjacent step points, it determines that the first ray has crossed between these two step points. A reversal in the comparison result means that the comparison result at the previous step point changed from the fifth depth value being less than the sixth depth value to the current step point's comparison result being the fifth depth value being greater than the sixth depth value.

[0061] In some cases, a reversal of the comparison result can also include a change from the comparison result at the previous step point being greater than the sixth depth value to the comparison result at the current step point being less than the sixth depth value. A reversal of the comparison result also occurs if the first ray crosses from inside the object to the outside. For example, if the executing entity finds that the fifth depth value is less than the sixth depth value at the Nth step point, it means that the first ray is in front of the object surface at that step point; if it finds that the fifth depth value is greater than the sixth depth value at the (N+1)th step point, it means that the first ray is behind the object surface at that step point. If the executing entity detects that the comparison result of these two adjacent step points has reversed from "less than" to "greater than," it determines that the first ray has crossed between these two step points.

[0062] In some cases, the fifth depth value can be subtracted from the sixth depth value, and the difference can be used as the comparison result. If the signs of the differences between two adjacent step points are opposite, the comparison result is determined to have been reversed.

[0063] Understandably, when a crossing (collision) occurs, of the two adjacent stepping points before and after the crossing, one (e.g., the preceding stepping point) can be considered the first stepping point, and the other (e.g., the following stepping point) can be considered the second stepping point. The fifth depth value of the first stepping point can be regarded as the first depth value described below, and the sixth depth value corresponding to the first stepping point can be regarded as the second depth value. The fifth depth value of the second stepping point can be regarded as the third depth value described below, and the sixth depth value corresponding to the second stepping point can be regarded as the fourth depth value described below.

[0064] In some cases, starting from the first starting point of the detection path, multiple step points are generated along the detection path according to a preset step. For each step point, the fifth depth value corresponding to the first ray and the sixth depth value obtained from the depth texture sampling are obtained respectively, and the two sets of depth values ​​are compared. When the comparison results of two adjacent step points are flipped, it is determined that the first ray has crossed, so that the crossing detection is performed on the complete projection path of the first ray in the depth texture, which improves the integrity of the detection. This allows the execution subject to accurately locate which two step points the crossing event occurred between, and the crossing interval can be determined without additional sampling, which improves the efficiency and accuracy of crossing detection to a certain extent.

[0065] S204: In response to the detection of crossing, obtain the first coordinate and the second coordinate. The first coordinate is the texture coordinate of the first step point, and the second coordinate is the texture coordinate of the second step point. The first step point and the second step point are two adjacent step points on the detection path before and after crossing.

[0066] A step point is a position in the two-dimensional texture space reached by the executing entity as it proceeds along the detection path in preset steps. A step point can correspond to the position of a texel. In some cases, the first step point may be the last position reached by the executing entity before traversing the detection path, and the second step point may be the first position reached by the executing entity after traversing the detection path. The first and second step points are adjacent positions on the detection path.

[0067] The first coordinate is the coordinate of the first step point in the 2D texture space. The first step point is the last step point on the detection path before the crossing occurs. The second coordinate is the coordinate of the second step point in the 2D texture space. The second step point is the first step point on the detection path after the crossing occurs. The crossing event occurs somewhere between the first and second step points.

[0068] In some cases, the executing entity can maintain the texture coordinates of the previous step point and the texture coordinates of the current step point during the detection process. When the executing entity detects a crossing at the current step point (i.e., the previous step point is in front of the surface and the current step point is behind the surface), the executing entity can use the coordinates of the previous step point as the first coordinate and the coordinates of the current step point as the second coordinate.

[0069] S205: Interpolate the first and second coordinates, and determine the third coordinate based on the interpolation result. The third coordinate is the corresponding coordinate of the collision point in the two-dimensional texture space.

[0070] Interpolation is used to estimate values ​​at unknown locations near known discrete data points. In some cases, interpolation can be non-linear, such as using quadratic functions or higher-order polynomials to fit crossing points.

[0071] In some cases, interpolation can be linear interpolation, which means estimating an approximate value at a certain position between two known texel positions using a linear function. For example, a third coordinate can be obtained by linearly interpolating the first and second coordinates according to certain interpolation coefficients. The accuracy of this third coordinate is higher than that of texel coordinates, reaching sub-texel accuracy.

[0072] In some cases, when a crossing is detected, the algorithm does not directly return any pixel center. Instead, it uses the depth difference between the two step points before and after the crossing to calculate the precise proportional position of the crossing point between two adjacent pixels through linear interpolation. Since this proportional position can take any value between 0 and 1, the coordinates of the collision point in texture space are continuous floating-point numbers, capable of falling at any position within a pixel. This means that when the ray steps along the inclined direction, the position of the crossing point corresponding to each step point can continuously change with the continuous change in depth, and the output collision point coordinates are no longer limited to discrete pixel grids. This reduces integer pixel quantization error. Furthermore, the collision points mapped in 3D space can be continuously arranged along the inclined plane direction, improving the inclined plane fit and avoiding jagged discrete errors.

[0073] In some cases, traversal detection is performed by projecting rays onto the 2D texture space of the depth texture and performing discrete step-by-step detection along the detection path. Between two adjacent step points where a traversal occurs, the texture coordinates of the two points are interpolated to obtain the collision point coordinates in the texture space. This eliminates the need to continuously reduce the step interval, improving the detection accuracy of collision points and reducing errors while controlling computational load. The coordinates of the collision point in the texture space are continuous floating-point numbers, which helps to improve jitter or jagged edges that occur when the user's position or movement changes in low-resolution depth textures. Furthermore, traversal judgment of the object surface is achieved based on the depth values ​​of the depth texture texels, eliminating the need for original 3D model geometric data and reducing scene data loading and storage overhead. Additionally, the above scheme can obtain the third coordinate of the traversal point in the 2D texture space by interpolating the first and second coordinates, eliminating the need for iterative binary search and multiple texture sampling, thus reducing computational overhead. This improves the real-time interactive experience of extended reality devices.

[0074] In some cases, the preset step size includes a horizontal step component and a vertical step component; the method also includes:

[0075] Determine the first span and the second span. The first span is the horizontal span of the detection path, and the second span is the vertical span of the detection path. The horizontal span is determined by the horizontal coordinate components of the first starting point and the first ending point, and the vertical span is determined by the vertical coordinate components of the first starting point and the first ending point. The first starting point is the starting point of the detection path, and the first ending point is the ending point of the detection path.

[0076] Determine the largest span between the first and second spans, and use the largest span as the third span;

[0077] The horizontal step component is determined based on the first and third spans, and the vertical step component is determined based on the second and third spans.

[0078] In some cases, after determining the detection path, the executing entity can determine a preset step size. This preset step size can include a horizontal step component (e.g., a U-axis component) and a vertical step component (e.g., a V-axis component). The preset step size can be determined based on these two components. The horizontal step component represents the amount of movement in the horizontal direction for each step, and the vertical component represents the amount of movement in the vertical direction for each step.

[0079] In some cases, the executing entity can obtain a first horizontal coordinate and a second horizontal coordinate. The first horizontal coordinate is the horizontal coordinate component of the first starting point, and the second horizontal coordinate is the horizontal coordinate component of the first ending point. The difference between the first and second horizontal coordinates is calculated, and the absolute value of this difference is taken as the first span. Similarly, the second span can be calculated.

[0080] The executing entity determines the larger of the first and second spans and uses that as the third span. The third span represents the maximum extension distance of the detection path in the horizontal or vertical direction.

[0081] In some cases, the executing entity can divide the first span by the third span to obtain the horizontal step component, which represents the amount of horizontal movement per step. Since the third span is the largest of the first and second spans, when the first span is the largest, the horizontal step component is 1.0, indicating a horizontal movement of one texel per step; when the first span is not the largest, the horizontal step component is a positive number less than 1.0. The executing entity can divide the second span by the third span to obtain the vertical step component, which represents the amount of vertical movement per step. Since the third span is the largest of the first and second spans, when the second span is the largest, the vertical step component is 1.0, indicating a vertical movement of one texel per step; when the second span is not the largest, the vertical step component is a positive number less than 1.0, meaning the vertical step component is less than one texel in two-dimensional texel space.

[0082] In some cases, if the horizontal coordinate of the first starting point of the detection path is 10.0 and the vertical coordinate is 20.0, and the horizontal coordinate of the first ending point is 110.0 and the vertical coordinate is 70.0, then the first span is 100.0 (the absolute value of 110.0 minus 10.0), the second span is 50.0 (the absolute value of 70.0 minus 20.0), and the third span is 100.0 (the larger of the first and second spans). The executing entity calculates the horizontal step component as 1.0 (100.0 divided by 100.0) and the vertical step component as 0.5 (50.0 divided by 100.0).

[0083] After determining the horizontal and vertical step components, the executing entity can advance step by step along the detection path from the first starting point. At each step, the executing entity adds the horizontal step component to the horizontal coordinate of the current step point and the vertical step component to the vertical coordinate, obtaining the position of the next step point. The executing entity repeats this operation until all step points on the detection path have been traversed.

[0084] The above calculation method ensures that the total number of steps equals the number of texels in the detection path along the maximum span direction, thus ensuring that the executing entity advances exactly one texel per step in the maximum span direction and a proportional amount of texels in the other direction. The executing entity performs a traversal detection operation at each step point.

[0085] In some cases, by splitting the preset step into horizontal and vertical step components, and obtaining the horizontal and vertical spans of the detection path based on the first starting and ending points, the larger of the two is selected as the third span. Then, the horizontal step component is determined by combining the first and third spans, and the vertical step component is determined by combining the second and third spans. This allows for adaptive allocation of the horizontal and vertical step components according to the length and width span of the detection path itself. This makes the step sampling step size adapt to the geometry of the detection path, avoiding the problem of excessively dense sampling in one direction due to a single fixed step size, which leads to unnecessary computational overhead, or the problem of missed detection due to insufficient sampling due to insufficient sampling. Relatively uniform step sampling can be achieved under detection paths with different orientations and different length-to-width ratios, taking into account both the detection reliability and computational efficiency of collision point detection. In addition, the total number of steps determined by the above scheme is equal to the number of texels in the detection path in the direction of the maximum span. Thus, each step advances exactly one texel in the direction of the maximum span, and advances the corresponding number of texels in the other direction proportionally. This makes the step points on the detection path evenly distributed to a certain extent, and to a certain extent ensures that the above step detection can completely cover all texels on the detection path, which is conducive to improving the accuracy of the crossing point detection results.

[0086] In some cases, the aforementioned fifth depth value is determined based on the following steps:

[0087] Obtain the first inverse depth, which is the inverse depth corresponding to the previous step point. The initial value of the first inverse depth is the second inverse depth, which is obtained by taking the reciprocal of the seventh depth value. The seventh depth value is the depth value of the second starting point, which is the starting point of the first ray.

[0088] The first inverse depth and the inverse depth are summed step by step, and the sum is used as the third inverse depth. The third inverse depth is the inverse depth of the step point.

[0089] The fifth depth value is obtained by taking the reciprocal of the third inverse depth.

[0090] The inverse depth is the reciprocal of the depth value, i.e., 1 divided by the depth value. The seventh depth value is the depth value of the starting point of the first ray (i.e., the second starting point), which represents the three-dimensional spatial distance from the starting point of the first ray to the reference point. The second inverse depth is the reciprocal of the seventh depth value. The depth value of the second starting point can be obtained by transforming the second starting point from world space to view space and extracting the Z-axis depth. Alternatively, the depth value of the second starting point can be calculated by inputting the three-dimensional coordinates of the second starting point into a preset world coordinate to linear depth transformation function.

[0091] The inverse depth step is the change in inverse depth for each step forward. This change is determined by the inverse depth values ​​of the first ray at the start and end points of the detection path, as well as the total number of steps.

[0092] In some cases, the executing entity can obtain a first inverse depth. The first inverse depth is the inverse depth corresponding to the previous step point, that is, the inverse depth used by the executing entity at the step point preceding the current step point. The initial value of the first inverse depth is a second inverse depth, which is the first inverse depth at the starting point of the detection path. In other words, when determining the inverse depth of a step point, the inverse depth of the starting point of the detection path is used as the initial value. In some cases, if the executing entity is currently at the first step point of the detection path, and the previous step point does not exist, the executing entity can set the first inverse depth to the aforementioned second inverse depth.

[0093] In some cases, at each step point, the executing entity adds the inverse depth value of the previous step point to the inverse depth step, and uses the sum of the above as the third inverse depth, which is the inverse depth of the current step point.

[0094] The executing entity takes the reciprocal of the third inverse depth to obtain the fifth depth value. This fifth depth value is the depth value of the first ray at the current step point, and is used to compare it with the depth value of the depth texture at the current step point.

[0095] In some cases, if the depth of the second starting point is 2.0 meters, the executing entity can calculate the second inverse depth as 0.5 (1 divided by 2.0). The executing entity can set the initial value of the first inverse depth to 0.5. Assuming the inverse depth step is 0.02, the executing entity can sum the first inverse depth with the inverse depth step at the first stepping point to obtain a third inverse depth of 0.52. The executing entity takes the reciprocal of the third inverse depth to obtain a fifth depth value of approximately 1.923 meters (1 divided by 0.52). This fifth depth value indicates that the distance from the first ray at the first stepping point to the reference point is approximately 1.923 meters.

[0096] In some cases, the fifth depth value corresponding to each step point is determined by iterative accumulation of inverse depth. The second inverse depth, obtained by taking the reciprocal of the seventh depth value at the starting point of the first ray, is used as the initial first inverse depth. The third inverse depth is obtained by superimposing the first inverse depth of the previous step point with the inverse depth step, and then taking the reciprocal of the third inverse depth to convert it into the fifth depth value. Since the inverse depth changes linearly in the texture space, it is only necessary to accumulate the inverse depth step by the inverse depth value of the previous step point and then take the reciprocal of the accumulation result to obtain the ray depth value of the current step point. Compared with re-performing the coordinate transformation from world space to view space and the 3D spatial distance calculation at each step point, this scheme can reduce the amount of calculation at each step point to a certain extent, which is beneficial to reducing the computational overhead. At the same time, since the inverse depth step is a fixed increment, the ray depth value obtained by the execution subject at each step point on the detection path changes uniformly with the step, so that the sampling points are evenly distributed along the direction of the first ray, which ensures the consistency of depth value comparison to a certain extent.

[0097] In some cases, the inverse depth step is positively correlated with the inverse depth span and negatively correlated with the third span;

[0098] The third span is the larger of the first span and the second span. The first span is the horizontal span of the detection path, and the second span is the vertical span of the detection path.

[0099] The inverse depth span is determined by the inverse depth of the second inverse depth and the inverse depth of the second endpoint, the second endpoint being the designated endpoint on the first ray; the projection of the second endpoint in the two-dimensional texture space coincides with the first endpoint, the first endpoint being the endpoint of the detection path.

[0100] The inverse depth span can be defined as the total change in inverse depth between the starting and ending points of the first ray along the detection path. The second endpoint is a designated endpoint on the first ray, determined based on a preset maximum detection distance—the position reached after extending the first ray along its direction from its starting point. The first endpoint is the endpoint of the detection path in the depth texture space, i.e., the position after projecting the second endpoint of the first ray into the depth texture space. The projection of the second endpoint into the depth texture space coincides with the first endpoint.

[0101] In some cases, the inverse depth span is determined by the inverse depth of the second inverse depth (i.e., the inverse depth of the starting point of the first ray) and the inverse depth of the second endpoint. Specifically, the executing entity can calculate the inverse depth of the second endpoint and the difference between the inverse depth of the second endpoint and the second inverse depth, and use this difference as the inverse depth span.

[0102] In some cases, the executing entity can divide the inverse depth span by the third span to obtain the inverse depth step. Since the projection position of the endpoint of the first ray (i.e., the second endpoint) in the depth texture space coincides with the endpoint of the detection path (i.e., the first endpoint), the total texel span of the first ray in the depth texture space is the total texel span of the detection path (i.e., the third span). When the executing entity evenly distributes the inverse depth span across the total span of the detection path, the change in inverse depth at each step point can be obtained. Since the inverse depth changes linearly in texture space, evenly distributing the change in the inverse depth span along the detection path to each step point ensures that the change in inverse depth at each step point is consistent, that is, the depth value of the first ray at each step point changes uniformly with the step.

[0103] As an illustration, if the second inverse depth is 0.5 and the inverse depth of the second endpoint is 0.2, then the inverse depth span is -0.3 (0.2 minus 0.5). If the horizontal span of the detection path is 100 texels, the vertical span is 50 texels, and the third span is 100 texels, then the executing entity divides the inverse depth span of 0.3 by the third span of 100, resulting in an inverse depth step of -0.003. Starting from the starting point, the executing entity adds -0.003 to the current inverse depth at each step, reducing it to exactly 0.2 at the second endpoint, so that the inverse depth of the first ray on the detection path changes linearly from 0.5 at the starting point to 0.2 at the endpoint.

[0104] In some cases, the inverse depth step is set to be positively correlated with the inverse depth span and negatively correlated with the third span. The projection of the second endpoint on the first ray in the two-dimensional texture space coincides with the first endpoint of the detection path. The size of the inverse depth step can be adaptively adjusted according to the range of inverse depth change of the first ray itself and the span of the detection path in the texture space. This makes the inverse depth sampling step size in the ray direction match the number of sampling steps in the texture space, ensuring that the step points in the texture space correspond one-to-one with the sampling points in the ray depth direction. This avoids misalignment of the sampling steps and alleviates the problem of excessively sparse sampling at the far end or excessively dense sampling at the near end caused by the nonlinear change in depth under perspective projection. It also improves the calculation accuracy of the fifth depth value of the ray at each step point. Moreover, all parameters can be directly calculated from the acquired inverse depth and span parameters. The calculation logic is simple and the computational overhead is small.

[0105] Figure 3 Here is a flowchart illustrating collision point detection methods for other scenarios, such as... Figure 3 As shown, the method includes, in addition to, with Figure 2 In addition to steps S301-S304 that are the same as or similar to steps S201-S204 shown, the following steps are also included:

[0106] S305: Obtain the first depth value, the second depth value, the third depth value, and the fourth depth value. The first depth value is the depth value of the first ray at the first step point, the second depth value is the depth value of the depth texture at the first step point, the third depth value is the depth value of the first ray at the second step point, and the fourth depth value is the depth value of the depth texture at the second step point.

[0107] The first depth value refers to the three-dimensional spatial distance from the first step point on the first ray to the reference point (also known as the ray depth value), and the unit can be meters. The second depth value refers to the depth value associated with the texel corresponding to the first step point in the depth texture (also known as the environment depth value), and the unit can be meters. This value characterizes the distance from the object surface corresponding to the first step point to the reference point. The third depth value refers to the three-dimensional spatial distance from the first ray's position on the first ray to the reference point, and the unit can be meters. The fourth depth value refers to the depth value associated with the texel corresponding to the second step point in the depth texture, and the unit can be meters. This value characterizes the distance from the object surface corresponding to the second step point to the reference point.

[0108] In some cases, the executing entity can determine the corresponding point on the first ray based on the position of the first stepping point on the detection path, and determine the depth value of the corresponding point (the distance from the corresponding point to the reference point) as the first depth value. In other cases, the executing entity can obtain the first depth value by taking the reciprocal of the inverse depth value of the corresponding point.

[0109] In some cases, the executing entity can sample the environmental depth value at the location of the first step point in the depth texture based on the texture coordinates of that location. The first step point is located before the traversal occurs, and the object surface corresponding to the second depth value is located in front of or behind the current position reached by the first ray, depending on the state before the first ray traversed.

[0110] Similarly, the executing entity can obtain a third depth value and a fourth depth value, where the object surface corresponding to the fourth depth value is located behind or in front of the current position reached by the first ray.

[0111] S306: Obtain the first difference and the second difference, where the first difference is the difference between the first depth value and the second depth value, and the second difference is the difference between the third depth value and the fourth depth value.

[0112] The first difference is the difference between the first depth value and the second depth value. This difference characterizes the positional relationship of the first ray relative to the object surface at the first step-entry point. In some cases, the executing agent can calculate the first difference by subtracting the second depth value from the first depth value. Since the first step-entry point is located before the crossing occurs, the sign of the first difference can indicate whether the first ray is in front of or behind the object surface at the first step-entry point. In some cases, the first step-entry point is in front of the object surface, and the first difference can be negative (i.e., the first depth value is less than the second depth value).

[0113] The second difference is the difference between the third and fourth depth values. This difference characterizes the positional relationship of the first ray relative to the object surface at the second step point. In some cases, the executing agent can calculate the second difference by subtracting the fourth depth value from the third depth value. Since the second step point is located after the crossing occurs, the sign of the second difference can indicate whether the first ray is in front of or behind the object surface at the second step point. In some cases, the second step point is behind the object surface, and the second difference is positive (i.e., the third depth value is greater than the fourth depth value).

[0114] S307: Determine the interpolation coefficients based on the first and second differences.

[0115] In some cases, linear interpolation can be used for interpolation. If the first difference is close to the second difference, the interpolation coefficient can be a preset value, such as 0.5.

[0116] After the execution entity detects a crossing, it needs to determine the specific location of the crossing point between the first and second stepping points. Assuming the difference value changes linearly between the first and second stepping points, the execution entity can calculate the ratio of the crossing point's position relative to the two stepping points. In some cases, the execution entity can determine the ratio of the absolute value of the first difference value to the total change between the first and second difference values; this ratio represents the relative position of the crossing point between the first and second stepping points.

[0117] In some cases, step S307 above includes: obtaining a third difference, which is the difference between the second difference and the first difference; and determining interpolation coefficients based on the first difference and the third difference.

[0118] In some cases, the executing entity subtracts the first difference from the second difference to obtain the third difference. In some cases, the first stepping point is in front of the object surface, and the first difference is negative; the second stepping point is behind the object surface, and the second difference is positive. The third difference, obtained by subtracting the first difference from the second difference, is a positive value, representing the total change in difference within the interval between the first and second stepping points. In some cases, the executing entity can calculate the proportion of the absolute value of the first difference in the third difference; this proportion represents the relative position of the crossing point between the first and second stepping points, and is used as an interpolation factor.

[0119] Specifically, the executing entity divides the absolute value of the first difference by the third difference to obtain the interpolation coefficient. Since the absolute value of the first difference reflects the distance between the crossing point and the first step point, while the third difference reflects the total change from the first step point to the second step point, the ratio of the two can indicate the relative position of the crossing point between the first step point and the second step point.

[0120] For example, if the first difference is -0.1 and the second difference is 0.15, then the third difference is 0.25. The executing entity divides the absolute value of the first difference (i.e., 0.1) by the third difference (i.e., 0.25) to obtain an interpolation coefficient of 0.4. This interpolation coefficient indicates that the crossing point is located at 40% of the distance between the first and second stepping points.

[0121] In some cases, by obtaining the first and second differences representing the depth deviation, a third difference is further calculated, and then the interpolation coefficients are determined by combining the first and third differences. The above scheme only performs difference-type arithmetic operations, with simple calculation logic and low computational overhead, making it easy to process in parallel on extended reality devices; at the same time, it directly reuses the difference output from the collision point detection stage, without introducing additional configuration parameters, thus reducing the burden of parameter configuration.

[0122] S308: Interpolate the first and second coordinates according to the interpolation coefficients, and determine the third coordinate based on the interpolation result. The third coordinate is the corresponding coordinate of the collision point in the two-dimensional texture space.

[0123] In some cases, the executing entity can use linear interpolation to blend the first and second coordinates based on interpolation coefficients. The interpolation coefficients indicate the relative position of the crossing point between the first and second stepping points. The executing entity then weights and combines the first and second coordinates according to the proportion corresponding to these interpolation coefficients, resulting in a new coordinate system located between the two; this new coordinate system is the third coordinate system.

[0124] Through the above interpolation, the accuracy of the third coordinate can be higher than that of a single texel, reaching the sub-texel level. Figure 4 This provides an interpolation diagram for certain situations. For example... Figure 4 As shown, assuming the texture coordinates of the first stepping point A are the 100th texel in the horizontal direction (u) and the 200th texel in the vertical direction (v), and the texture coordinates of the second stepping point B are the 101st texel in the horizontal direction and the 201st texel in the vertical direction, with an interpolation coefficient of 0.3, the execution body mixes the first and second coordinates at a ratio of 0.3, resulting in a third coordinate C that is the 100.3th texel in the horizontal direction and the 200.3th texel in the vertical direction. This coordinate indicates that the collision point is located at 30% of the distance between the 100th and 101st texels, and also at 30% of the distance between the 200th and 201st texels.

[0125] In some cases, by acquiring the depth values ​​of the ray and the depth values ​​obtained from depth texture sampling at two adjacent step points where a collision is detected, a first depth value, a second depth value, a third depth value, and a fourth depth value are obtained. The differences between these depth values ​​yield a first difference and a second difference, and interpolation coefficients are determined based on these two sets of differences. These interpolation coefficients are then used to interpolate the first and second coordinates. This dynamically generates interpolation coefficients by reusing the relative relationships of multiple sets of depth deviations obtained during the step detection process, which can accurately reflect the position of the ray relative to the object surface. Compared to interpolation methods using fixed interpolation coefficients, this scheme can adapt to different depth deviation scenarios, improve the rationality of the interpolation coefficients, and thus improve the accuracy of collision point detection results. Furthermore, the above process involves only simple calculations, has low computational overhead, is easy to process in parallel on hardware, and is suitable for real-time rendering collision point detection scenarios.

[0126] In some cases, the method also includes:

[0127] The second and fourth depth values ​​are interpolated using interpolation coefficients to obtain the collision point depth value.

[0128] The fourth coordinate is obtained by performing an inverse projection transformation based on the third coordinate and the depth value of the collision point. The fourth coordinate is the coordinate of the collision point in three-dimensional space.

[0129] In some cases, the executing entity can use the interpolation coefficient as a weight to weight and merge the second and fourth depth values ​​to obtain a new depth value that lies between the two. This new depth value is the collision point depth value.

[0130] For example, if the second depth value is 2.0 meters, the fourth depth value is 1.7 meters, and the interpolation factor is 0.4, then the executing entity will mix the second and fourth depth values ​​according to the interpolation factor to obtain a collision point depth value of 1.88 meters. This collision point depth value represents the distance from the object surface corresponding to the collision point to the reference point.

[0131] In some cases, after obtaining the collision point depth value, the executing entity can perform an inverse projection transformation operation to map the collision point from a two-dimensional texture space to a three-dimensional space. The executing entity can determine the fourth coordinate, which is the coordinate of the collision point in three-dimensional space, based on the third coordinate (i.e., the texture coordinate of the collision point in the two-dimensional texture space) and the collision point depth value through the inverse projection transformation.

[0132] In some cases, the executing entity can convert the third coordinate from two-dimensional texture coordinates to normalized device coordinates, then determine the depth of the point in view space based on the collision point depth value, and then transform the point from view space to world space through the inverse view projection matrix to obtain the fourth coordinate.

[0133] Schematic, after obtaining the third coordinates (100.3, 200.3) and the collision point depth value of 1.88 meters, the executing entity inputs these texture coordinates and depth value into the inverse projection transformation function. This function calculates the point coordinates in three-dimensional space corresponding to the pixel position, i.e., the fourth coordinates, based on the inverse matrix of the current view projection matrix. These fourth coordinates describe the precise three-dimensional position of the collision point in world space.

[0134] In some cases, by interpolating the second and fourth depth values ​​using predetermined interpolation coefficients, the collision point depth value is obtained. Then, combining the collision point coordinates and the collision point depth value, an inverse projection transformation is performed to obtain the fourth coordinate representing the 3D spatial position. This transforms the collision point detection result in 2D texture space into the collision point position in 3D space, establishing a complete processing chain for collision point detection using depth textures. Depth interpolation reuses existing parameters, and the inverse projection transformation is a matrix operation, resulting in controllable overall computational overhead and leveraging the high efficiency of the 2D detection stage. Simultaneously, it can output collision results in both 2D and 3D dimensions, adapting to various upper-level business scenarios such as ray picking and virtual object interaction. Furthermore, reusing the same interpolation coefficients for depth interpolation ensures consistency between the texture coordinates and depth values ​​of the collision point, avoiding data deviations caused by multiple sets of interpolation weights.

[0135] In some cases, the method further includes: projecting the fourth coordinate onto the first ray, correcting the fourth coordinate based on the projection result to obtain the fifth coordinate, which is located on the first ray.

[0136] In some cases, due to limitations in numerical accuracy during inverse projection transformation and interpolation, the fourth coordinate often exhibits a slight deviation. To ensure the stability of the collision point, the executing entity can project the fourth coordinate onto the first ray for correction.

[0137] The correction operation forces the calculated collision position back to the ray, with the aim of moving the fourth coordinate, which may deviate from the first ray, onto the first ray along a direction perpendicular to the first ray, so that the corrected fourth coordinate is strictly located on the first ray.

[0138] In some cases, this post-correction process ensures that the output collision point is strictly located on the first ray path, eliminating output results with unreasonable geometric positions.

[0139] In some cases, the above-mentioned projection of the fourth coordinate onto the first ray to obtain the corrected fourth coordinate includes:

[0140] Determine the projection distance, which is the distance from the fourth coordinate along the first direction to the starting point of the first ray; the first direction is the direction of the first ray.

[0141] The fourth coordinate is corrected based on the projection distance on the first ray to obtain the fifth coordinate. The starting point of the first ray is the same as the starting point of the first ray. In some cases, the executing entity can determine the projection distance of the fourth coordinate relative to the starting point of the first ray on the first direction vector.

[0142] Specifically, the executing entity can calculate the vector pointing from the starting point of the first ray to the fourth coordinate. The executing entity can then calculate the dot product between this vector and the first direction vector; the result of this dot product is the projected distance of the fourth coordinate relative to the starting point of the first ray in the first direction. This projected distance can be expressed as a signed numerical value, the absolute value of which represents the distance of the fourth coordinate along the first direction to the starting point of the first ray. The sign of this projected distance indicates whether the fourth coordinate is located in front of or behind the starting point of the first ray.

[0143] In some cases, a negative projection distance indicates that the fourth coordinate is located behind the starting point of the first ray. The executing entity can set the projection distance to zero to ensure that the corrected position is not located behind the starting point of the first ray.

[0144] In some cases, the executing entity can reposition the fourth coordinate on the first ray based on the projection distance to obtain the corrected fourth coordinate, which is also the fifth coordinate.

[0145] Specifically, the executing entity starts from the starting point of the first ray and moves the projected distance along the first direction vector to reach a new three-dimensional spatial position. This position is the fifth coordinate, which lies precisely on the first ray.

[0146] This repositioning operation makes the vector between the corrected fourth coordinate (i.e., the fifth coordinate) and the starting point of the first ray parallel to the first direction vector, meaning the fifth coordinate lies on the first ray. This operation preserves the positional information of the fourth coordinate along the first direction (i.e., the distance from the fourth coordinate to the starting point of the first ray) while eliminating the offset component of the fourth coordinate perpendicular to the first direction.

[0147] In some cases, by determining the projected distance of the fourth coordinate relative to the starting point of the first ray on the first direction vector, and repositioning the fourth coordinate on the first ray based on this projected distance, the fifth coordinate is obtained. This corrects the fourth coordinate, which might have deviated from the first ray, onto the first ray. Since the fifth coordinate is strictly located on the first ray, this correction operation eliminates the problem of collision points deviating from the ray due to numerical errors, thereby avoiding visual jitter caused by jumps in the position of collision points during extended reality device use, and improving the stability of the interaction to a certain extent.

[0148] The following is combined with Figure 5 Please provide an explanation. Figure 5This is a schematic diagram of a scenario-based collision point detection process. In this scenario, the user wears an augmented reality (AR) headset, which includes left and right eye depth sensors and corresponding depth textures. The headset calculates the view projection matrices for each eye based on data from the image sensor and IMU. The following explanation uses the left eye as an example; the processing flow for the right eye is identical. The AR headset's execution unit obtains the left eye depth texture of the current frame from the depth sensor. This depth texture is a two-dimensional texture space, where each texel is associated with a depth value representing the distance from the corresponding spatial point to the left eye's reference point.

[0149] The executing entity can acquire multiple frames of environmental images captured by the image sensor and construct a first coordinate system (i.e., the world space coordinate system) based on these images. The origin of the first coordinate system is set to the initial position when the device starts up. Once established, the first coordinate system remains fixed throughout the interaction and is used to describe the position of each object in the real environment and the spatial pose of the device in that environment.

[0150] After establishing the first coordinate system, the executing agent can acquire the environmental image and depth texture of the current frame and construct a three-dimensional space 501 in the first coordinate system. Specifically, the executing agent uses the environmental image to determine the spatial pose of the image sensor when acquiring the image, and then transforms each texel in the depth texture from the depth sensor coordinate system to the first coordinate system to generate a three-dimensional point cloud, which is used to describe the geometry and position of the object surface in the current real environment.

[0151] Assuming the user holds the controller pointing directly forward, the application running in the extended reality device can emit a first ray from the controller's position to detect whether the controller's direction collides with the surface of an object in the real environment. Before performing collision detection, the aforementioned execution entity can acquire the depth texture 502 corresponding to the current 3D space. For example, it can obtain the depth texture from the depth sensor or from the data used to generate the currently displayed 3D space.

[0152] In some cases, the aforementioned execution entity can obtain the first ray from the handle tracking module, including the position and direction vector of the second starting point of the first ray in the first coordinate system. The execution entity can project the first ray onto the two-dimensional texture space and determine the detection path 503 based on the projection result. Specifically, the aforementioned execution entity can project the second starting point and the second ending point of the first ray from the left-eye world space to the left-eye depth texture space, respectively, to obtain the texture coordinates of the first starting point and the texture coordinates of the first ending point. Assume that the texture coordinates of the first starting point in the left-eye depth texture are column 64, row 64, and the texture coordinates of the first ending point are column 64, row 32. The projection line segment between the first starting point and the second starting point can be the projection of the first ray from the second starting point to the second ending point. This projection line segment is a vertical line segment in the depth texture, extending from column 64, row 64 to column 64, row 32. The detection path is the texel sequence traversed from the starting point to the ending point of this line segment, which traverses a total of 32 texels.

[0153] After determining the detection path, the execution entity can calculate the step parameter 504. Specifically, the execution entity can determine the span of the detection path in the horizontal and vertical directions. In the example above, the horizontal span is 0 (the horizontal coordinates of both the start and end points are 64), and the vertical span is 32 (the vertical coordinates of the start point are 64, and the vertical coordinates of the end point are 32, with a difference of 32). The execution entity determines the larger of the horizontal and vertical spans as the third span, which is 32, and the vertical direction is the direction of the maximum span.

[0154] The executing entity can calculate the horizontal step component and the vertical step component. Since the horizontal span is 0, the horizontal step component is 0, which means that each step is stationary in the horizontal direction; the vertical step component can be the vertical span divided by the third span, that is, 32 divided by 32 equals 1, which means that each step moves one texel in the vertical direction.

[0155] The executing entity can calculate the inverse depth of the second starting point and the second ending point. The inverse depth is the reciprocal of the depth value. The depth value at the second starting point is 2.0 meters (the distance from the starting point to the reference point corresponding to the left eye), and the inverse depth is 0.5 (1 / meter); the depth value at the second ending point is 7.0 meters, and the inverse depth is approximately 0.143 (1 / meter). The executing entity can calculate the inverse depth span, which is the inverse depth of the second ending point minus the inverse depth of the first starting point, resulting in a negative value, indicating that the inverse depth decreases along the detection path. The executing entity divides the inverse depth span by the third span to obtain the inverse depth step, representing the change in inverse depth at each step point. Each step reduces the inverse depth by approximately 0.0112 (1 / meter).

[0156] The execution entity can perform collision point detection by progressively advancing along the detection path based on the aforementioned stepping parameters 501. Starting from the first starting point (column 64, row 64), the execution entity advances progressively according to the stepping components. At each stepping point, the execution entity can determine whether the stepping point is within the boundary of the two-dimensional texture space based on its coordinates (or the corresponding texel row and column index) 506. If not, it returns information indicating that the point exceeds the view frustum 507. If the determination result is yes, it can obtain the fifth and sixth depth values ​​corresponding to the stepping point 508 and compare their magnitudes.

[0157] The fifth depth value of each step point can be obtained through iterative steps. Starting from the second starting point, the inverse depth of the second starting point is 0.5 (1 / m), and the inverse depth step is -0.0112 (1 / m). After the first step, 0.5 (1 / m) plus the inverse depth step of -0.0112 (1 / m) is used to obtain the inverse depth corresponding to the first step point, which is 0.4888 (1 / m). Taking the reciprocal of the above inverse depth, the fifth depth corresponding to the first step point is obtained as 2.045827 meters. After the second step, the above inverse depth of 0.4888 (1 / m) plus the inverse depth step of -0.0112 (1 / m) is used to obtain the inverse depth corresponding to the second step point, which is 0.4776 (1 / m), thus obtaining the fifth depth value of the second step point as 2.093802 meters. This process continues, with the execution entity sequentially calculating the fifth depth values ​​of multiple step points until the fifth depth value of the first endpoint is obtained.

[0158] For each step point, the aforementioned execution entity can obtain the sixth depth value from the depth texture based on the texel index corresponding to the step point, which represents the distance from the object surface corresponding to that texel to the left eye reference point.

[0159] For each step point, the executing entity can compare the fifth depth value and the sixth depth value: if the fifth depth value is less than the sixth depth value, it means that the first ray is in front of the object surface at that step point; if the fifth depth value is greater than the sixth depth value, it means that the first ray is behind the object surface at that step point.

[0160] At the initial stepping point, the executing entity obtains a fifth depth value of 2.0 meters and a sixth depth value of 3.0 meters. The comparison shows that the fifth depth value is less than the sixth depth value, and the first ray is located in front of the object surface. After several steps, at stepping point 17 (column 64, row 48), the fifth depth value is approximately 3.12 meters, and the sixth depth value is 3.0 meters. The comparison shows that the fifth depth value is greater than the sixth depth value, and the first ray is located behind the object surface.

[0161] The aforementioned execution entity can determine whether a crossing 509 has occurred based on the comparison results corresponding to any two adjacent step points. For example, if the execution entity detects that the comparison result at step point 17 is flipped compared to the comparison result at step point 16 (the fifth depth value at step point 16 is less than the sixth depth value, and the fifth depth value at step point 17 is greater than the sixth depth value), then it is determined that the first ray crossed the object surface between step points 16 and 17, that is, the first ray collided with the object surface between step points 16 and 17.

[0162] In response to the detection of a crossing, the executing entity can determine the interpolation coefficient 510. Specifically, the executing entity can acquire a first coordinate and a second coordinate. The first coordinate is the texture coordinate of step point 16 (column 64, row 49), and the second coordinate is the texture coordinate of step point 17 (column 64, row 48). The executing entity can acquire four depth values: the first depth value is the depth value of the first ray at step point 16 (approximately 3.08 meters), the second depth value is the depth value of the depth texture at step point 16 (approximately 3.1 meters), the third depth value is the depth value of the first ray at step point 17 (approximately 3.12 meters), and the fourth depth value is the depth value of the depth texture at step point 17 (approximately 3.0 meters).

[0163] The executing entity can calculate the first difference and the second difference. The first difference is the first depth value minus the second depth value (approximately -0.02 meters), and the second difference is the third depth value minus the fourth depth value (approximately +0.12 meters). A negative first difference indicates that the ray at step point 16 is in front of the surface, and a positive second difference indicates that the ray at step point 17 is behind the surface.

[0164] The executing entity can calculate the third difference, which is the second difference minus the first difference (approximately 0.14 meters), representing the total change in the difference value from the first step to the second step. The executing entity can use the absolute value of the first difference divided by the third difference (approximately 0.143 meters) to obtain the interpolation coefficient, which indicates the relative position of the crossing point between the first and second step points, i.e., at approximately the 14.3% mark.

[0165] The executing entity can interpolate the first and second coordinates based on the interpolation coefficients to obtain the third coordinate 511. In the example above, the horizontal coordinate remains unchanged at 64, while the vertical coordinate moves by approximately 14.3% in the direction from 49 to 48, i.e., 49 minus 0.143, resulting in approximately 48.857. The third coordinate (64, 48.857) indicates that the collision point is located at the 64th column and 48.857th row in the depth texture, achieving sub-vellium level precision.

[0166] The executing entity can interpolate the second and fourth depth values ​​using interpolation coefficients to obtain the collision point depth value. The collision point depth value is 3.1 meters plus (3.0 meters minus 3.1 meters) multiplied by 0.143, which is approximately 3.0857 meters. This means that the distance from the object surface corresponding to the collision point to the reference point corresponding to the left eye is approximately 3.0857 meters.

[0167] The executing entity performs an inverse projection transformation based on the third coordinate and the collision point depth value, using the inverse matrix of the left eye view projection matrix to map the two-dimensional texture coordinates and depth value back to the three-dimensional world space, obtaining the fourth coordinate 512. The fourth coordinate is (0.3 m, 0.2 m, -3.08 m), indicating that the collision point is located on the object surface approximately 3.08 m directly in front of the handle.

[0168] The executing entity projects the fourth coordinate onto the first ray, obtaining the fifth coordinate (corrected fourth coordinate) 513. Specifically, the executing entity can calculate the projected distance of the fourth coordinate relative to the starting point of the first ray on the first direction vector (approximately 3.58 meters), and then move this projected distance along the direction vector from the starting point of the first ray to obtain the corrected fourth coordinate (0.2 meters, 0.8 meters, -3.08 meters). The fifth coordinate lies strictly on the first ray (collinear with the direction vector) and is output as the final collision point coordinate. The fifth coordinate is then returned to 514.

[0169] For the right eye, the execution entity can use the right-eye depth texture and the right-eye view projection matrix to perform the exact same processing flow as the left eye. Because the position of the reference point corresponding to the right eye (e.g., the right-eye viewpoint) is offset from the left eye's interpupillary distance (e.g., the right-eye reference point is located approximately 0.06 meters to the right of the left-eye viewpoint), the projection position of the first ray in the depth texture, the traversal detection result, and the final collision point coordinates from the right-eye perspective may have slight differences compared to the left eye. The left and right eyes each output their respective collision point coordinates for use by the upper-layer rendering engine to generate virtual object displays from the left and right eye perspectives.

[0170] Figure 6 This is a structural block diagram of a collision point detection device provided in some situations. For example... Figure 6 As shown, the device 60 includes: a first acquisition unit 601, a first projection unit 602, a step detection unit 603, a second acquisition unit 604, and an interpolation unit 605. Wherein,

[0171] The first acquisition unit 601 is used to acquire a depth texture. The depth texture has a two-dimensional texture space, which includes multiple texels, and each texel is associated with a corresponding depth value.

[0172] The first projection unit 602 is used to project the first ray onto the two-dimensional texture space and determine the detection path based on the projection result.

[0173] The step detection unit 603 is used to detect whether the first ray passes through the object surface by a preset step on the detection path. The object surface is characterized by the depth values ​​of multiple corresponding texels.

[0174] The second acquisition unit 604 is used to acquire a first coordinate and a second coordinate in response to the detection of crossing. The first coordinate is the coordinate of the first step point in the two-dimensional texture space, and the second coordinate is the coordinate of the second step point in the two-dimensional texture space. The first step point and the second step point are two adjacent step points on the detection path before and after crossing.

[0175] Interpolation unit 605 is used to interpolate the first coordinate and the second coordinate, and determine the third coordinate based on the interpolation result. The third coordinate is the corresponding coordinate of the collision point in the two-dimensional texture space.

[0176] In some cases, traversal detection is performed by projecting rays onto the 2D texture space of the depth texture and performing discrete step-by-step detection along the detection path. Between two adjacent step points where a traversal occurs, the texture coordinates of the two points are interpolated to obtain the collision point coordinates in the texture space. This eliminates the need to continuously reduce the step interval, improving the detection accuracy of collision points and reducing errors while controlling computational load. The coordinates of the collision point in the texture space are continuous floating-point numbers, which helps to improve jitter or jagged edges that occur when the user's position or movement changes in low-resolution depth textures. Furthermore, traversal judgment of the object surface is achieved based on the depth values ​​of the depth texture texels, eliminating the need for original 3D model geometric data and reducing scene data loading and storage overhead. Additionally, the above scheme can obtain the third coordinate of the traversal point in the 2D texture space by interpolating the first and second coordinates, eliminating the need for iterative binary search and multiple texture sampling, thus reducing computational overhead.

[0177] In some cases, the interpolation unit 605 is further used for:

[0178] Obtain the first depth value, the second depth value, the third depth value, and the fourth depth value. The first depth value is the depth value of the first ray at the first step point, the second depth value is the depth value of the depth texture at the first step point, the third depth value is the depth value of the first ray at the second step point, and the fourth depth value is the depth value of the depth texture at the second step point.

[0179] Obtain the first difference and the second difference. The first difference is the difference between the first depth value and the second depth value, and the second difference is the difference between the third depth value and the fourth depth value.

[0180] The interpolation coefficients are determined based on the first and second differences.

[0181] Interpolate the first and second coordinates based on the interpolation coefficients.

[0182] In some cases, the interpolation unit 605 is further used to: obtain a third difference, which is the difference between the second difference and the first difference;

[0183] The interpolation coefficients are determined based on the first and third differences.

[0184] In some cases, the interpolation unit 605 is further used to: interpolate the second depth value and the fourth depth value according to the interpolation coefficients to obtain the collision point depth value;

[0185] The fourth coordinate is obtained by performing an inverse projection transformation based on the third coordinate and the depth value of the collision point. The fourth coordinate is the coordinate of the collision point in three-dimensional space.

[0186] In some cases, device 60 further includes a correction unit (not shown in the figure), which is used for:

[0187] Project the fourth coordinate onto the first ray, and correct the fourth coordinate based on the projection result to obtain the fifth coordinate, which is located on the first ray.

[0188] In some cases, the correction unit is further used for:

[0189] Determine the projection distance, which is the distance from the fourth coordinate along the first direction to the starting point of the first ray; the first direction is the direction of the first ray.

[0190] The fourth coordinate is corrected based on the projection distance on the first ray to obtain the fifth coordinate.

[0191] In some cases, the step detection unit 603 is further used for:

[0192] Starting from the first starting point, multiple step points are determined on the detection path according to a preset step; the first starting point is the starting point of the detection path;

[0193] For each step point, obtain the fifth depth value and the sixth depth value. The fifth depth value is the depth value of the first ray at the step point, and the sixth depth value is the depth value of the depth texture at the step point.

[0194] Compare the fifth and sixth depth values;

[0195] In response to the detection of a flip in the comparison results of two adjacent step points, it is determined that the first ray has crossed.

[0196] In some cases, the preset step includes a horizontal step component and a vertical step component; the step detection unit 603 is further used for:

[0197] Determine the first span and the second span. The first span is the horizontal span of the detection path, and the second span is the vertical span of the detection path. The horizontal span is determined by the horizontal coordinate components of the first starting point and the first ending point, and the vertical span is determined by the vertical coordinate components of the first starting point and the first ending point. The first starting point is the starting point of the detection path, and the first ending point is the ending point of the detection path.

[0198] Determine the largest span between the first and second spans, and use the largest span as the third span;

[0199] The horizontal step component is determined based on the first and third spans, and the vertical step component is determined based on the second and third spans.

[0200] In some cases, the device 60 further includes a depth value determination unit (not shown in the figure), which determines a fifth depth value based on the following steps:

[0201] Obtain the first inverse depth, which is the inverse depth corresponding to the previous step point. The initial value of the first inverse depth is the second inverse depth, which is obtained by taking the reciprocal of the seventh depth value. The seventh depth value is the depth value of the second starting point, which is the starting point of the first ray.

[0202] The first inverse depth and the inverse depth are summed step by step, and the sum is used as the third inverse depth. The third inverse depth is the inverse depth of the step point.

[0203] The fifth depth value is obtained by taking the reciprocal of the third inverse depth.

[0204] In some cases, the inverse depth step is positively correlated with the inverse depth span and negatively correlated with the third span;

[0205] The third span is the larger of the first span and the second span. The first span is the horizontal span of the detection path, and the second span is the vertical span of the detection path.

[0206] The inverse depth span is determined by the inverse depth of the second inverse depth and the inverse depth of the second endpoint, the second endpoint being the designated endpoint on the first ray; the projection of the second endpoint in the two-dimensional texture space coincides with the first endpoint, the first endpoint being the endpoint of the detection path.

[0207] In order to achieve the solutions provided above, an electronic device is also provided in some cases.

[0208] refer to Figure 7The diagram illustrates the structure of an electronic device 700 suitable for implementing solutions in various scenarios. This electronic device 700 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), tablet computers, portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital televisions (TVs) and desktop computers. Figure 7 The electronic device shown is merely an example and should not impose any limitations on the functionality and scope of use of the solution in various situations.

[0209] like Figure 7 As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700.

[0210] Processing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0211] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0212] Specifically, the processes described in the flowchart above can be implemented as computer software programs. For example, some cases include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart. In such cases, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of some cases.

[0213] It should be noted that in some cases, the aforementioned computer-readable storage medium may be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. Computer-readable storage media may be, for example,—but not limited to—electrical, magnetic, optical, electromagnetic, infrared, or semiconductor program servers, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM, also known as flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In some cases, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution program server, device, or device. In other cases, a computer-readable signal medium may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution program server, device, or apparatus. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, radio frequency (RF), etc., or any suitable combination thereof.

[0214] The aforementioned computer-readable storage medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0215] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method described above.

[0216] Computer program code for performing operations under certain conditions can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0217] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of parsing servers, methods, and computer program products under various scenarios. In this respect, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the figures. For example, two consecutively indicated blocks may actually be executed substantially in parallel, or sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based parsing server that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0218] The units described in some situations can be implemented in software or hardware. In some cases, the name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0219] In some cases, the functions described above can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field-Programmable Gate Array (FPGA), Application-Specific Integrated Circuit (ASIC), Application-Specific Standard Product (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), and so on.

[0220] In some cases, electronic devices, computer-readable storage media, and computer program products perform collision detection by projecting rays onto the two-dimensional texture space of a depth texture and taking discrete steps along the detection path. Between two adjacent step points where a collision occurs, the texture coordinates of the two points are interpolated to obtain the collision point coordinates in the texture space. This eliminates the need to continuously reduce the step interval, improving the detection accuracy of collision points and reducing detection errors while controlling computational load. The coordinates of the collision point in the texture space are continuous floating-point numbers, which helps to improve jitter or jagged edges that occur when the user's position or movement changes in low-resolution depth textures. Furthermore, the collision detection of the object surface is based on the depth values ​​of the depth texture texels, eliminating the need for original 3D model geometric data and reducing scene data loading and storage overhead. Additionally, the above scheme can obtain the third coordinate of the crossing point in the two-dimensional texture space by interpolating the first and second coordinates, eliminating the need for iterative binary search and multiple texture sampling, thus reducing computational overhead.

[0221] In a first aspect, a collision point detection method is provided according to one or more scenarios, comprising: acquiring a depth texture, the depth texture having a two-dimensional texture space, the two-dimensional texture space including multiple texels, each texel being associated with a corresponding depth value; projecting a first ray onto the two-dimensional texture space, and determining a detection path based on the projection result; detecting whether the first ray crosses the object surface on the detection path by a preset step, the object surface being characterized by the depth values ​​of multiple corresponding texels; in response to the detection of crossing, acquiring a first coordinate and a second coordinate, the first coordinate being the coordinate of the first step point in the two-dimensional texture space, the second coordinate being the coordinate of the second step point in the two-dimensional texture space, the first step point and the second step point being two adjacent step points before and after crossing on the detection path; interpolating the first coordinate and the second coordinate, and determining a third coordinate based on the interpolation result, the third coordinate being the corresponding coordinate of the collision point in the two-dimensional texture space.

[0222] Interpolating the first and second coordinates based on one or more conditions includes: obtaining a first depth value, a second depth value, a third depth value, and a fourth depth value, wherein the first depth value is the depth value of the first ray at the first step point, the second depth value is the depth value of the depth texture at the first step point, the third depth value is the depth value of the first ray at the second step point, and the fourth depth value is the depth value of the depth texture at the second step point; obtaining a first difference and a second difference, wherein the first difference is the difference between the first and second depth values, and the second difference is the difference between the third and fourth depth values; determining interpolation coefficients based on the first and second differences; and interpolating the first and second coordinates based on the interpolation coefficients.

[0223] Based on one or more scenarios, determining interpolation coefficients based on a first difference and a second difference includes: obtaining a third difference, which is the difference between the second difference and the first difference; and determining interpolation coefficients based on the first difference and the third difference.

[0224] Depending on one or more scenarios, the method further includes: interpolating the second depth value and the fourth depth value according to the interpolation coefficient to obtain the collision point depth value; and performing an inverse projection transformation based on the third coordinate and the collision point depth value to obtain the fourth coordinate, which is the coordinate of the collision point in three-dimensional space.

[0225] Depending on one or more scenarios, the method further includes: projecting the fourth coordinate onto the first ray, correcting the fourth coordinate based on the projection result to obtain the fifth coordinate, which is located on the first ray.

[0226] Depending on one or more scenarios, the fourth coordinate is projected onto the first ray, and the fourth coordinate is corrected based on the projection result to obtain the fifth coordinate. This includes: determining the projection distance, which is the distance from the fourth coordinate along the first direction to the starting point of the first ray; the first direction is the direction of the first ray; and correcting the fourth coordinate on the first ray based on the projection distance to obtain the fifth coordinate.

[0227] Based on one or more scenarios, the detection of whether a first ray traverses the object surface is performed according to a preset step, including: starting from a first starting point, determining multiple step points on the detection path according to a preset step; the first starting point is the starting point of the detection path; for each step point, acquiring a fifth depth value and a sixth depth value, where the fifth depth value is the depth value of the first ray at the step point and the sixth depth value is the depth value of the depth texture at the step point; comparing the fifth depth value and the sixth depth value; and determining that the first ray has traversed the object surface in response to the detection of a flip in the comparison result of two adjacent step points.

[0228] According to one or more scenarios, the preset step includes a horizontal step component and a vertical step component; the method further includes: determining a first span and a second span, wherein the first span is the horizontal span of the detection path, and the second span is the vertical span of the detection path; the horizontal span is determined by the horizontal coordinate components of the first starting point and the first ending point, and the vertical span is determined by the vertical coordinate components of the first starting point and the first ending point, wherein the first starting point is the starting point of the detection path; and the first ending point is the ending point of the detection path; determining the larger of the first span and the second span, and taking the larger of the two spans as the third span; determining the horizontal step component based on the first span and the third span, and determining the vertical step component based on the second span and the third span.

[0229] The fifth depth value is determined based on one or more scenarios using the following steps: obtaining the first inverse depth, which is the inverse depth corresponding to the previous step point, with the initial value of the first inverse depth being the second inverse depth, which is obtained by taking the reciprocal of the seventh depth value, where the seventh depth value is the depth value of the second starting point, which is the starting point of the first ray; summing the first inverse depth and the inverse depth step by step, and using the summation result as the third inverse depth, which is the inverse depth of the step point; and taking the reciprocal of the third inverse depth to obtain the fifth depth value.

[0230] Depending on one or more scenarios, the inverse depth step is positively correlated with the inverse depth span and negatively correlated with the third span; the third span is the largest of the first span and the second span, the first span is the horizontal span of the detection path, and the second span is the vertical span of the detection path; the inverse depth span is determined by the second inverse depth and the inverse depth of the second endpoint, the second endpoint is the designated endpoint on the first ray; the projection of the second endpoint in the two-dimensional texture space coincides with the first endpoint, the first endpoint is the endpoint of the detection path.

[0231] Secondly, according to one or more scenarios, a collision point detection device is provided, comprising: a first acquisition unit for acquiring a depth texture, the depth texture having a two-dimensional texture space, the two-dimensional texture space including multiple texels, each texel being associated with a corresponding depth value; a first projection unit for projecting a first ray onto the two-dimensional texture space, and determining a detection path based on the projection result; a step detection unit for detecting whether the first ray crosses an object surface on the detection path according to a preset step, the object surface being characterized by the depth values ​​of multiple corresponding texels; a second acquisition unit for acquiring a first coordinate and a second coordinate in response to the detection of a crossing, the first coordinate being the coordinate of a first step point in the two-dimensional texture space, the second coordinate being the coordinate of a second step point in the two-dimensional texture space, the first step point and the second step point being two adjacent step points before and after the crossing on the detection path; and an interpolation unit for interpolating the first coordinate and the second coordinate, and determining a third coordinate based on the interpolation result, the third coordinate being the corresponding coordinate of the collision point in the two-dimensional texture space.

[0232] Depending on one or more scenarios, the interpolation unit is further configured to: acquire a first depth value, a second depth value, a third depth value, and a fourth depth value, wherein the first depth value is the depth value of the first ray at the first step point, the second depth value is the depth value of the depth texture at the first step point, the third depth value is the depth value of the first ray at the second step point, and the fourth depth value is the depth value of the depth texture at the second step point; acquire a first difference and a second difference, wherein the first difference is the difference between the first depth value and the second depth value, and the second difference is the difference between the third depth value and the fourth depth value; determine interpolation coefficients based on the first difference and the second difference; and interpolate the first coordinate and the second coordinate based on the interpolation coefficients.

[0233] Depending on one or more scenarios, the interpolation unit is further configured to: obtain a third difference, which is the difference between the second difference and the first difference; and determine interpolation coefficients based on the first difference and the third difference.

[0234] In some cases, the interpolation unit is further used to: interpolate the second depth value and the fourth depth value according to the interpolation coefficient to obtain the collision point depth value; and perform an inverse projection transformation based on the third coordinate and the collision point depth value to obtain the fourth coordinate, which is the coordinate of the collision point in three-dimensional space.

[0235] Depending on one or more scenarios, the apparatus further includes a correction unit, which is used to: project a fourth coordinate onto a first ray, and correct the fourth coordinate based on the projection result to obtain a fifth coordinate, the fifth coordinate being located on the first ray.

[0236] Depending on one or more scenarios, the correction unit is further configured to: determine the projection distance, which is the distance from the fourth coordinate along the first direction to the starting point of the first ray; the first direction is the direction of the first ray; and correct the fourth coordinate on the first ray according to the projection distance to obtain the fifth coordinate.

[0237] In some cases, the step detection unit is further configured to: starting from a first starting point, determine multiple step points on the detection path according to a preset step; the first starting point is the starting point of the detection path; for each step point, acquire a fifth depth value and a sixth depth value, the fifth depth value being the depth value of the first ray at the step point, and the sixth depth value being the depth value of the depth texture at the step point; compare the fifth depth value and the sixth depth value; and determine that the first ray has crossed in response to the detection of a flip in the comparison result of two adjacent step points.

[0238] According to one or more scenarios, the preset step includes a horizontal step component and a vertical step component; the step detection unit is further configured to: determine a first span and a second span, wherein the first span is the horizontal span of the detection path, and the second span is the vertical span of the detection path; the horizontal span is determined by the horizontal coordinate components of the first starting point and the first ending point, and the vertical span is determined by the vertical coordinate components of the first starting point and the first ending point, wherein the first starting point is the starting point of the detection path; and the first ending point is the ending point of the detection path; determine the larger of the first span and the second span, and use the larger of them as the third span; determine the horizontal step component based on the first span and the third span, and determine the vertical step component based on the second span and the third span.

[0239] According to one or more scenarios, the device further includes a depth value determination unit, which is used to determine a fifth depth value based on the following steps: obtaining a first inverse depth, the first inverse depth being the inverse depth corresponding to the previous step point, the initial value of the first inverse depth being a second inverse depth, the second inverse depth being obtained by taking the reciprocal of a seventh depth value, the seventh depth value being the depth value of a second starting point, the second starting point being the starting point of the first ray; summing the first inverse depth and the inverse depth step by step, and using the summation result as a third inverse depth, the third inverse depth being the inverse depth of the step point; taking the reciprocal of the third inverse depth to obtain the fifth depth value.

[0240] Depending on one or more scenarios, the inverse depth step is positively correlated with the inverse depth span and negatively correlated with the third span; the third span is the largest of the first span and the second span, the first span is the horizontal span of the detection path, and the second span is the vertical span of the detection path; the inverse depth span is determined by the second inverse depth and the inverse depth of the second endpoint, the second endpoint is the designated endpoint on the first ray; the projection of the second endpoint in the two-dimensional texture space coincides with the first endpoint, the first endpoint is the endpoint of the detection path.

[0241] Thirdly, according to one or more scenarios, an electronic device is provided, comprising: at least one processor and a memory; the memory storing computer-executable instructions; the at least one processor executing the computer-executable instructions stored in the memory, such that the at least one processor performs the methods described in the first aspect above and various possible designs of the first aspect.

[0242] Fourthly, according to one or more scenarios, a computer-readable storage medium is provided, which stores computer-executable instructions, and when a processor executes the computer-executable instructions, implements the methods described in the first aspect above and various possible designs of the first aspect.

[0243] Fifthly, according to one or more circumstances, a computer program product is provided, including a computer program, and a method for implementing the first aspect and various possible designs of the first aspect when the computer program is executed by a processor.

[0244] In summary, in some cases, by projecting rays onto the 2D texture space of the depth texture and performing cross-travel detection by discretely stepping along the detection path, and interpolating the texture coordinates of two adjacent step points where a cross-travel occurs, the collision point coordinates in the texture space can be obtained. This eliminates the need to continuously reduce the step interval, improving the detection accuracy of collision points and reducing the error of the detection results while controlling the computational load. The coordinates of the collision points in the texture space are continuous floating-point numbers, which helps to improve the jitter or jagged edges that occur when the user's position or movement changes in low-resolution depth textures. In addition, cross-travel judgment of the object surface is realized based on the depth values ​​of the depth texture texels, eliminating the need for the original 3D model geometric data and reducing the scene data loading and storage overhead. Furthermore, the above scheme can obtain the third coordinate of the cross-travel point in the 2D texture space by interpolating the first and second coordinates, eliminating the need for iterative binary search and multiple texture sampling, thus reducing computational overhead.

[0245] The above description is merely a preferred embodiment and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in some cases is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in some cases.

[0246] Furthermore, while the operations are described in a specific order, this should not be construed as requiring them to be executed in the indicated order or in a sequential manner. Multitasking and parallel processing may be advantageous in certain contexts. Similarly, while some specific implementation details are included in the above discussion, these should not be interpreted as limitations on the scope of certain cases. Certain features described in the context of a single case can also be implemented in combination within that single case. Conversely, various features described in the context of a single case can also be implemented individually or in any suitable sub-combination in multiple cases.

[0247] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A collision point detection method, comprising: A depth texture is obtained, wherein the depth texture has a two-dimensional texture space, the two-dimensional texture space includes multiple texels, and each texel is associated with a corresponding depth value; Project the first ray onto the two-dimensional texture space, and determine the detection path based on the projection result; On the detection path, the first ray is detected to see if it passes through the object surface at a preset step. The object surface is characterized by the depth values ​​of multiple corresponding texels. In response to the detection of crossing, a first coordinate and a second coordinate are obtained. The first coordinate is the coordinate of the first step point in the two-dimensional texture space, and the second coordinate is the coordinate of the second step point in the two-dimensional texture space. The first step point and the second step point are two adjacent step points on the detection path before and after crossing. Interpolate the first and second coordinates, and determine the third coordinate based on the interpolation result. The third coordinate is the corresponding coordinate of the collision point in the two-dimensional texture space.

2. The method according to claim 1, wherein interpolating the first coordinate and the second coordinate comprises: Obtain a first depth value, a second depth value, a third depth value, and a fourth depth value. The first depth value is the depth value of the first ray at the first step point, the second depth value is the depth value of the depth texture at the first step point, the third depth value is the depth value of the first ray at the second step point, and the fourth depth value is the depth value of the depth texture at the second step point. Obtain a first difference and a second difference, wherein the first difference is the difference between the first depth value and the second depth value, and the second difference is the difference between the third depth value and the fourth depth value; Based on the first difference and the second difference, the interpolation coefficients are determined; Interpolate the first coordinate and the second coordinate according to the interpolation coefficient.

3. The method according to claim 2, wherein determining the interpolation coefficients based on the first difference and the second difference comprises: Obtain a third difference, which is the difference between the second difference and the first difference; The interpolation coefficients are determined based on the first difference and the third difference.

4. The method according to claim 2, further comprising: Based on the interpolation coefficients, the second depth value and the fourth depth value are interpolated to obtain the collision point depth value; A fourth coordinate is obtained by performing an inverse projection transformation based on the third coordinate and the collision point depth value. The fourth coordinate is the coordinate of the collision point in three-dimensional space.

5. The method according to claim 4, further comprising: The fourth coordinate is projected onto the first ray, and the fourth coordinate is corrected based on the projection result to obtain the fifth coordinate, which is located on the first ray.

6. The method according to claim 5, wherein projecting the fourth coordinate to the first ray and correcting the fourth coordinate based on the projection result to obtain the fifth coordinate includes: Determine the projection distance, which is the distance from the fourth coordinate along the first direction to the starting point of the first ray; The first direction is the direction of the first ray; The fourth coordinate is corrected on the first ray according to the projection distance to obtain the fifth coordinate.

7. The method according to claim 1, wherein detecting whether the first ray traverses the object surface according to a preset step comprises: Starting from the first starting point, multiple step points are determined on the detection path according to the preset step; The first starting point is the starting point of the detection path; For each step point, a fifth depth value and a sixth depth value are obtained, wherein the fifth depth value is the depth value of the first ray at the step point, and the sixth depth value is the depth value of the depth texture at the step point; The fifth depth value and the sixth depth value are compared in magnitude; In response to the detection of a reversal in the comparison results of two adjacent step points, it is determined that the first ray has crossed.

8. The method according to claim 1 or 7, wherein the preset step includes a horizontal step component and a vertical step component; the method further includes: Determine a first span and a second span, wherein the first span is the horizontal span of the detection path and the second span is the vertical span of the detection path; The horizontal span is determined by the horizontal coordinate components of the first starting point and the first ending point, and the vertical span is determined by the vertical coordinate components of the first starting point and the first ending point. The first starting point is the starting point of the detection path. The first endpoint is the endpoint of the detection path; The largest of the first span and the second span is determined, and the largest is taken as the third span; The horizontal step component is determined based on the first span and the third span, and the vertical step component is determined based on the second span and the third span.

9. The method according to claim 7, wherein the fifth depth value is determined based on the following steps: Obtain the first inverse depth, which is the inverse depth corresponding to the previous step point. The initial value of the first inverse depth is the second inverse depth, which is obtained by taking the reciprocal of the seventh depth value. The seventh depth value is the depth value of the second starting point, which is the starting point of the first ray. The first inverse depth and the inverse depth are summed step by step, and the summation result is used as the third inverse depth, which is the inverse depth of the step point; The fifth depth value is obtained by taking the reciprocal of the third inverse depth.

10. The method according to claim 9, wherein the inverse depth step is positively correlated with the inverse depth span and negatively correlated with the third span; The third span is the largest of the first span and the second span, where the first span is the horizontal span of the detection path and the second span is the vertical span of the detection path. The inverse depth span is determined by the inverse depth of the second inverse depth and the inverse depth of the second endpoint, the second endpoint being a designated endpoint on the first ray; the projection of the second endpoint in the two-dimensional texture space coincides with the first endpoint, the first endpoint being the endpoint of the detection path.

11. A collision point detection device, comprising: The first acquisition unit is used to acquire a depth texture, the depth texture having a two-dimensional texture space, the two-dimensional texture space including a plurality of texels, each of the texels being associated with a corresponding depth value; The first projection unit is used to project the first ray onto the two-dimensional texture space and determine the detection path based on the projection result. The step detection unit is used to detect whether the first ray passes through the object surface by a preset step along the detection path. The object surface is characterized by the depth values ​​of multiple corresponding texels. The second acquisition unit is used to acquire a first coordinate and a second coordinate in response to the detection of crossing. The first coordinate is the coordinate of the first step point in the two-dimensional texture space, and the second coordinate is the coordinate of the second step point in the two-dimensional texture space. The first step point and the second step point are two adjacent step points on the detection path before and after crossing. The interpolation unit is used to interpolate the first and second coordinates, and determine the third coordinate based on the interpolation result. The third coordinate is the corresponding coordinate of the collision point in the two-dimensional texture space.

12. An electronic device, comprising: Processor and memory; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 10.

14. A computer program product comprising a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 10.

Images