Gesture point selection and distance measurement method in three-dimensional point cloud perspective display mode

Abstract

The invention discloses a gesture point selection and distance measurement method in a three-dimensional point cloud perspective display mode, and the method comprises the steps: obtaining a two-dimensional coordinate of a user gesture through a touch screen of an intelligent mobile device, mapping the two-dimensional coordinate into a ray end point in a three-dimensional space coordinate system through normalization conversion and inverse matrix operation, and obtaining a ray end point in a three-dimensional space coordinate system; the method comprises the following steps of: acquiring a ray, converting the ray into an actual three-dimensional coordinate, converting three-dimensional point cloud data from a Cartesian coordinate system into a cylindrical coordinate system, judging whether the ray is intersected with a cylinder or not through a ray equation, screening intersection points meeting conditions, calculating the shortest distance between the ray and the central axis of the cylinder, and determining whether a target point is selected or not through a distance threshold value. According to the method, misoperation is effectively avoided through minimum distance judgment and screening, the interaction experience of a user is improved, the method is suitable for multiple application scenes such as augmented reality and virtual reality, coordinate conversion and ray projection algorithms are optimized, and the response speed and accuracy of the system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional data processing and visualization, and particularly to a method for gesture point selection and distance measurement in a three-dimensional point cloud perspective display mode. Background Art

[0002] With the rapid development of augmented reality (AR), virtual reality (VR) and mixed reality (MR) technologies, three-dimensional point cloud data plays an increasingly important role in the precise digitization of visual or scene surface shapes; these technologies are widely involved in fields such as robotic medical navigation, autonomous navigation, and images, providing users with rich interactive experiences; however, the existing two-dimensional display methods are already difficult to meet the user's visual preview and operation requirements for three-dimensional space information, and there is an urgent need to develop a three-dimensional point cloud perspective display mode and its interaction technology suitable for intelligent mobile devices.

[0003] Currently, the existing technical solutions have the following deficiencies:

[0004] First, the priority recognition accuracy and robustness are insufficient: the existing priority recognition algorithms are difficult to maintain storage and robustness in complex environments, resulting in inaccurate point selection operations;

[0005] Second, the complexity of coordinate transformation and beam transformation is relatively high: the existing coordinate transformation and beam transformation have relatively high complexity, and are easily affected by factors such as device performance, data scale, and algorithm efficiency, resulting in slow processing speed and affecting the user experience;

[0006] Third, the implementation of the distance measurement function is complex and the accuracy is limited: the existing distance measurement function is complex to implement and has limited accuracy, making it difficult to meet the requirements of high-precision measurement.

[0007] CN111831118A discloses a three-dimensional electroencephalogram display method and display system based on augmented reality, aiming to achieve three-dimensional visualization display of electroencephalogram signals. By establishing a three-dimensional model of the cerebral cortex and a method for spatial localization of electrophysiological sources of brain signals, the spatial distribution of electroencephalogram signals is displayed in the three-dimensional model of the cerebral cortex in different colors; however, this solution mainly focuses on the three-dimensional visualization display of electrical signals, lacks support for the relay interaction and distance measurement functions of three-dimensional point cloud data, and cannot meet the user's perception and operation requirements for three-dimensional space information.

[0008] CN104050859A discloses an interactive digital three-dimensional sand table system, aiming to achieve interactive display of three-dimensional visual scenes or three-dimensional model data. By controlling the display of three-dimensional visual scenes or three-dimensional model data in a three-dimensional control stereoscopic image generation system, user interaction with the three-dimensional model is achieved; however, this system mainly focuses on the display of scenes or models, lacks support for the pre-recognition and distance measurement functions of three-dimensional point cloud data, and cannot meet the precise interaction requirements for three-dimensional point cloud data. Summary of the Invention

[0009] The purpose of this section is to outline some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of the present application, to avoid obscuring the purpose of this section, the abstract, and the title. However, such simplifications or omissions shall not be used to limit the scope of the present invention.

[0010] In view of the above existing problems, the present invention is proposed.

[0011] Therefore, the present invention aims to provide a high-precision, high-efficiency, three-dimensional point cloud perspective display mode and its point selection and ranging method to enhance the user experience and meet the precise interaction requirements for three-dimensional point cloud data.

[0012] To solve the above technical problems, the present invention provides the following technical solutions: obtaining the two-dimensional coordinates of the gesture touch points of the user on the screen through the touch screen of the intelligent mobile device;

[0013] Normalizing and converting the two-dimensional coordinates according to the width and height of the touch screen to obtain normalized coordinates, and converting the normalized coordinates into the ray endpoints in the three-dimensional space coordinate system by combining inverse matrix operations;

[0014] Converting the ray endpoints into actual three-dimensional point coordinates, and then converting the three-dimensional point cloud data from the Cartesian coordinate system to the cylindrical coordinate system to obtain the three-dimensional data in the cylindrical coordinate system where, is the radial distance, θ is the angle, and z is the height;

[0015] By solving the ray equation, determining whether the ray intersects the cylinder. If the intersection point exists and meets the preset conditions, record the intersection point; otherwise, it is considered that there is no corresponding point within the current touch point range;

[0016] After recording the intersection point, calculate the shortest distance R between the ray and the central axis of the cylinder min , and screen out points less than R min , and screen out points that do not belong to the height z value range interval of the three-dimensional point cloud data;

[0017] Traverse the filtered three-dimensional point cloud data, calculate the distance from each point to the ray, and obtain the point with the minimum distance and the minimum distance d min ; where:

[0018] If the minimum distance d min ≤ the threshold t, then regard the point with the minimum distance as the point selected by the gesture;

[0019] If the minimum distance dmin If it is greater than the threshold value t, it is considered that the gesture selection point is not selected.

[0020] As a preferred solution of the gesture selection point and ranging method in the three-dimensional point cloud perspective display mode described in the present invention, the method of judging whether the ray intersects with the cylinder by solving the ray equation, and if the intersection point exists and meets the preset conditions, recording the intersection point includes:

[0021] Using the average value method to obtain the center point of the three-dimensional point cloud data to obtain the center point coordinates;

[0022] Obtain the maximum value of the radial distance and the minimum value of the height z is z min and the maximum value z max , record the one-to-one correspondence between the points in the cylindrical coordinate system and the points in the three-dimensional Cartesian rectangular coordinate system, and arrange them in ascending order of the z value, that is, the three-dimensional point cloud data is regarded as being wrapped in a cylinder with the center of the point cloud as the radius and the z value range (z max , z min ) as the height;

[0023] Calculate the two intersection points (P1, P2) of the ray and the cylinder through the ray obtained by the ray casting algorithm. If there is no intersection point, directly return that the selection point fails, that is, there is no corresponding point within the current touch point range.

[0024] As a preferred solution of the gesture selection point and ranging method in the three-dimensional point cloud perspective display mode described in the present invention, the preset condition is defined as that the z value of P1 is greater than the z value of P2, that is, z1≥z2.

[0025] As a preferred solution of the gesture selection point and ranging method in the three-dimensional point cloud perspective display mode described in the present invention, the ray obtained by the ray casting algorithm includes:

[0026] Obtain the homogeneous coordinate expressions of the farthest and nearest endpoints of the ray through matrix inverse operation, where V farWorld represents the homogeneous coordinate expression of the point on the ray farthest from the observer in the world coordinate system, and V nearWorld represents the homogeneous coordinate expression of the point on the ray nearest to the observer in the world coordinate system;

[0027] Convert the homogeneous coordinates (V farWorld , V nearWorld ) into three-dimensional points (P nearWorld , P farWorld ), and the conversion relationship is:

[0028] V farWorld =[x,y,z,w]T

[0029]

[0030] V nearWorld = [x, y, z, w] T

[0031]

[0032] Wherein, x represents the position in the horizontal direction, y represents the position in the vertical direction, z represents the position in the depth direction, i.e., height, and w is a scaling factor for implementing matrix transformation;

[0033] By obtaining the three-dimensional point P closest to the observer nearWorld and the point P farthest from the observer farWorld , determine the ray projected by the touch point.

[0034] As a preferred solution of the gesture point selection and ranging method in the three-dimensional point cloud perspective display mode described in the present invention, it further includes:

[0035] Define the two end points P of the ray nearWorld and P farWorld , and the direction vector of the ray is calculated by the difference between these two end points:

[0036]

[0037] Wherein, P nearWorld is the starting point of the ray, and P farWorld is the ending point of the ray, is the direction vector of the ray;

[0038] Define the surface equation of the cylinder as Solve for the parameter λ through the ray equation to determine the intersection coordinates. The ray equation is:

[0039]

[0040] Wherein, λ is a parameter representing the position of a certain point on the ray. When the ray intersects the cylinder, the value will satisfy the cylinder equation x and y are the coordinates of the point on the plane, is the distance from the point to the central axis of the cylinder;

[0041] Substitute the ray equation into the cylinder equation, solve to obtain the parameter λ of the intersection of the ray and the cylinder, and define the x and y coordinates in the ray equation as:

[0042] x(λ) = xnear +λ·D x

[0043] y(λ) = y near +λ·D y

[0044] Substitute these two expressions into the cylinder equation to obtain:

[0045]

[0046] Based on the cylinder equation, obtain:

[0047]

[0048] where R is the radius of the cylinder;

[0049] Solve for λ using the quadratic formula:

[0050]

[0051] where ± indicates that the equation has two solutions, corresponding to the two intersections of the ray and the cylinder respectively, A is the coefficient of the λ 2 term, representing the sum of the squares of the components of the ray direction vector in the x and y directions, B is the coefficient of the λ term, representing the linear combination of the projections of the ray starting point and the ray direction vector in the x and y directions, and C is the constant term, representing the relationship between the ray starting point and the cylinder radius R;

[0052] Among the two solutions, select the solution with the smaller λ value as the nearest intersection point, and represent the solution with the larger λ value as the intersection point of the ray and the cylinder in the direction away from the starting point.

[0053] As a preferred solution of the method for gesture point selection and distance measurement in the three-dimensional point cloud perspective display mode according to the present invention, when performing point cloud screening, it is necessary to calculate the shortest distance from each center point P point in the point cloud to the ray, including:

[0054] Calculate the vector difference between the point P point to be detected and the near-end point P nearWorld of the ray:

[0055]

[0056] Then calculate the cross product of the ray direction vector and the vector difference :

[0057]

[0058] where, represents a vector perpendicular to these two vectors, whose length is equal to the area of the parallelogram spanned by these two vectors;

[0059] Calculate the modulus length of, which is the distance from the ray to point P point :

[0060]

[0061] Calculate the modulus length of the ray direction vector :

[0062]

[0063] By dividing the modulus length of the cross product by the modulus length of the ray direction vector, the shortest distance d from the point to the ray is obtained:

[0064]

[0065] wherein, is the modulus length of the cross product, representing the projection distance from the point to the ray, and is the modulus length of the ray direction vector, representing the length of the ray.

[0066] As a preferred solution of the method for gesture point selection and distance measurement in the three-dimensional point cloud perspective display mode described in the present invention, the conversion of the three-dimensional point cloud data from the Cartesian coordinate system to the cylindrical coordinate system to obtain the three-dimensional data in the cylindrical coordinate system includes:

[0067]

[0068] z = z

[0069] wherein, the height z value remains unchanged, is the perpendicular distance from the point to the z-axis, that is, the radius in the two-dimensional coordinate, and θ is the angle between the projection line of the ray on the xoy plane and the x-axis.

[0070] As a preferred solution of the method for gesture point selection and distance measurement in the three-dimensional point cloud perspective display mode described in the present invention, the comparison of the threshold t is defined as the space length occupied by each point, then the threshold is calculated as:

[0071]

[0072] wherein, c is the initial point cloud quantity, is the maximum value of the radial distance of, z min is the minimum value of the height z, z max is the maximum value of the height z.

[0073] Advantages of the present invention:

[0074] 1. By using the high-precision sensors of the touch screen, the operation intention of the user can be obtained in real time, and the touch points can be directly mapped into coordinate data, thus realizing an intuitive and flexible interaction method, which further improves the user experience;

[0075] 2. The two-dimensional coordinates are converted into a unified scale through normalization processing, avoiding the problem of inconsistent coordinates caused by differences in screen sizes and resolutions of different devices. Through this conversion, the obtained normalized coordinates can be consistent under various devices and screen sizes, ensuring the compatibility and stability of the system. In addition, combined with inverse matrix operations, the normalized coordinates are converted into the ray endpoints in the three-dimensional space coordinate system. This step ensures the accurate mapping from two-dimensional screen operations to three-dimensional space, providing an accurate starting point for subsequent three-dimensional point cloud data processing and gesture interaction;

[0076] 3. By converting the three-dimensional point cloud data into a cylindrical coordinate system, data storage and processing can be carried out more efficiently. Especially in applications such as augmented reality and virtual reality, it shows better visual effects and enhances the system's ability to process complex scenes;

[0077] 4. Effectively filter out the intersection points that meet the conditions, reduce unnecessary calculations, and improve the efficiency of the overall system;

[0078] 5. By calculating the shortest distance between the ray and the central axis of the cylinder, the interference points irrelevant to the target object are excluded, and data screening is carried out according to the set height range, ensuring that the system can efficiently identify the actual interaction operations of the user, avoiding misoperations or calculating too many invalid points, and improving the response speed and accuracy of the system;

[0079] 6. By calculating the distance from each filtered three-dimensional point to the ray, the selected points of the user's intention are accurately determined, providing a precise interaction method. Especially in a virtual environment, the user can select specific three-dimensional points through touch, and then operate virtual objects or scenes, improving the intuitiveness of the interaction and the user experience;

[0080] 7. Realize the efficient filtering and accurate recognition of gesture selection, ensuring that only when the user really intends to select a certain point, the system will feedback the corresponding result. This not only effectively prevents accidental touches or interference, but also enhances the user's trust in operating the system, improving the intelligence and usability of the interaction. Description of the Drawings

[0081] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. Among them:

[0082] Figure 1 It is a schematic flowchart of the gesture point selection and distance measurement method in the three-dimensional point cloud perspective display mode shown in the present invention;

[0083] Figure 2 It is a schematic diagram of the two-dimensional coordinates of the screen of the intelligent mobile operating system shown in the present invention;

[0084] Figure 3 It is a schematic diagram of the three-dimensional space coordinate system shown in the present invention;

[0085] Figure 4 It is a schematic diagram for comparing the data processing efficiency of the method of the present invention and the traditional method shown in the present invention. Detailed implementation manners

[0086] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will make a detailed description of the specific implementation manners of the present invention with reference to the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments.

[0087] Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0088] Many specific details are set forth in the following description in order to provide a thorough understanding of the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0089] According to the embodiments of the present invention, in combination with Figure 1 the flowchart shown, a gesture point selection and distance measurement method in a three-dimensional point cloud perspective display mode specifically includes the following steps:

[0090] S1. Obtain the two-dimensional coordinates of the gesture touch points of the user on the screen through the touch screen of the intelligent mobile device;

[0091] S2. Perform normalization conversion on the two-dimensional coordinates according to the width and height of the touch screen to obtain normalized coordinates, and combine inverse matrix operations to convert the normalized coordinates into ray endpoints in the three-dimensional space coordinate system;

[0092] S3. Convert the ray endpoint to the actual three-dimensional point coordinates, and then convert the three-dimensional point cloud data from the Cartesian coordinate system to the cylindrical coordinate system to obtain the three-dimensional data in the cylindrical coordinate system. Among them, is the radial distance, θ is the angle, and z is the height.

[0093] S4. By solving the ray equation, determine whether the ray intersects with the cylinder. If the intersection point exists and meets the preset conditions, record the intersection point; otherwise, it is considered that there is no corresponding point within the current touch point range.

[0094] S5. After recording the intersection point, calculate the shortest distance R between the ray and the central axis of the cylinder min , and filter out points less than R min , and filter out points whose height z values do not belong to the range interval of the three-dimensional point cloud data.

[0095] S6. Traverse the filtered three-dimensional point cloud data, calculate the distance from each point to the ray, and obtain the point with the minimum distance and the minimum distance d min ; where:

[0096] a. If the minimum distance d min ≤ the threshold t, then regard the point with the minimum distance as the point selected by the gesture.

[0097] b. If the minimum distance d min > the threshold t, then it is considered that the gesture selection point is not selected.

[0098] It should be noted that the method of the present invention realizes high-precision and high-efficiency gesture interaction and point selection functions by closely combining the user's gesture operations with the interaction of three-dimensional point cloud data, solves the problems of accuracy, robustness, and complexity in the prior art, and improves the user experience.

[0099] Furthermore, it should be noted that the method of the present invention improves the accuracy of gesture recognition by optimizing the three-dimensional point cloud processing flow, effectively compresses the data volume and reduces the storage requirements by representing the data in the cylindrical coordinate system; processing the point coordinates in the cylindrical coordinate system instead of the Cartesian coordinate system effectively reduces the computational complexity and improves the real-time performance of gesture recognition and distance measurement, meeting the requirements for fast response in practical applications, and enabling the system to provide instant feedback to the user during gesture operations.

[0100] Next, in combination with Figures 2 to 3 the schematic diagram shown and some preferred or optional examples of the present invention, the implementation process and / or effects of certain examples of the present invention will be described more specifically.

[0101] Refer to Figure 2, the user touches the touch screen to perform a point selection operation, that is, the user gently touches a certain point on the screen with a finger or a stylus. In the intelligent mobile operating system (such as Android, IOS), the two-dimensional coordinates of the screen take a certain point on the screen as the origin. For example, the origin of the two-dimensional coordinate system of the screen is set at the upper left vertex of the screen, and the position of any point on the screen is determined by the coordinate values in the horizontal and vertical directions. The horizontal direction (the x-axis) extends from left to right, and the vertical direction (the y-axis) extends from top to bottom.

[0102] Refer to Figure 3 , when the user performs a touch operation on the touch screen, the two-dimensional coordinates (x, y) of the touch point can be obtained through the operating system. Among them, x is the value of the touch point on the x-axis, and y is the value of the touch point on the y-axis, with the unit of pixel.

[0103] In an optional implementation, the Android operating system uses OpenGL (a cross-programming language and cross-platform programming graphics program interface) to transform the point cloud data from local coordinates to screen coordinates, and finally draws and displays the three-dimensional point cloud information on the Android system screen. OpenGL uses a homogeneous coordinate system (x, y, z, w) to represent three-dimensional space. x represents the position in the horizontal direction, y represents the position in the vertical direction, z represents the position in the depth direction, and w is the fourth component of the homogeneous coordinates, which is a scaling factor used to implement various matrix transformations (such as translation, rotation, scaling, projection); among them:

[0104] Input the three-dimensional point cloud data. At this time, the three-dimensional point cloud data is in the original local space under the Cartesian coordinate system, and the initial homogeneous coordinate representation of the three-dimensional point cloud data is [x, y, z, 1] T ;

[0105] According to the relative position, relative rotation, and relative scaling information in the world space, the point cloud coordinates are transformed from the local space to the world space through the model matrix, and the object is restored to its distribution in the real world. The model matrix is equivalent to a set of translation, rotation, and scaling operations. By calculating the model matrix for each object, the relative position between objects can be determined;

[0106] The world space coordinates are transformed into the coordinates in front of the user's field of view through the view matrix, and the scene is translated or rotated so that a specific object is transformed in front of the camera;

[0107] Perform a projection operation on the coordinates reaching the view space, and clip the coordinates to the range of [-1, 1];

[0108] The coordinates in the range of [-1, 1] are transformed into screen coordinates through the viewport transformation, and the transformed coordinates are transmitted to the rasterizer and converted into fragments for display on the screen;

[0109] Exemplarily, the coordinate transformation operation relationship for OpenGL to display 3D point clouds is as follows:

[0110] V clip = M projection · M view · M model · [x, y, z, 1] T

[0111] where V clip represents the homogeneous coordinates of the normalized vertices displayed on the OpenGL screen, M projection represents the projection matrix, indicating how to project an object in three-dimensional space onto a two-dimensional screen, M view represents the view matrix, indicating the position, orientation, and field of view of the camera, M model is the model matrix, indicating the position, rotation, and scaling of the object, [x, y, z, 1] T is the homogeneous representation of the Cartesian coordinate data of the 3D point cloud.

[0112] In an alternative embodiment, the two-dimensional coordinates (x, y) of the touch point are converted into normalized coordinate points (x normalized , y normalized ), and the conversion method is as follows:

[0113]

[0114] where width is the screen width and height is the screen height. In OpenGL, the z value range of the normalized coordinates is between [1, -1], indicating the depth range in the clipping space. At this time, the normalized point farthest from the observer is (x normalized , y normalized , -1), and the normalized point closest to the observer on the ray is (x normalized , y normalized , 1).

[0115] As an example, the homogeneous coordinate expressions of the farthest and closest endpoints of the ray are obtained through matrix inverse operations, including:

[0116] When the point cloud is displayed on the screen, then M projection · M view · M model is known. At this time, the inverse matrix M projection · M view · M model of M invert can be obtained:

[0117] V farWorld = M invert · [x normalized , y normalized , 1, 1] T

[0118] V nearWorld = M invert ·[x normalized , y normalized , -1, 1] T

[0119] Among them, V farWorld represents the homogeneous coordinate expression of the point farthest from the observer on the ray in the world coordinate system, and V nearWorld represents the homogeneous coordinate expression of the point closest to the observer on the ray in the world coordinate system.

[0120] In an optional embodiment, the center point O of the three-dimensional point cloud is obtained in advance. The x, y, and z coordinates of all points in the point cloud are traversed and averaged respectively, and the obtained coordinates are the coordinates of the center point O. Then, the three-dimensional point cloud data is converted from the Cartesian coordinate system to the cylindrical coordinate system to obtain the three-dimensional data in the cylindrical coordinate system including:

[0121]

[0122] z = z

[0123] Among them, the height z value remains unchanged, is the perpendicular distance from the point to the z-axis, that is, the radius in the two-dimensional coordinates, and θ is the angle between the projection line of the ray on the xoy plane and the x-axis.

[0124] As an example, by solving the ray equation, it is judged whether the ray intersects the cylinder. If the intersection point exists and meets the preset conditions, the intersection point is recorded, including:

[0125] Using the average value method to obtain the center point of the three-dimensional point cloud data to get the center point coordinates;

[0126] Obtain the maximum value of the radial distance and the minimum value of the height z, the minimum value z and the maximum value z min max , record the one-to-one correspondence between the points in the cylindrical coordinate system and the points in the three-dimensional Cartesian rectangular coordinate system, and arrange them in ascending order according to the z value, that is, the three-dimensional point cloud data is regarded as wrapped in a cylinder with the center of the point cloud as the radius and the z value range (z max , z min ) as the height;

[0127] Calculate the two intersection points (P1, P2) of the ray and the cylinder through the ray obtained by the ray casting algorithm. If there is no intersection point, directly return that the point selection fails, that is, there is no corresponding point within the current touch point range.

[0128] Exemplarily, the preset condition is defined as the z value of P1 being greater than the z value of P2, i.e., z1≥z2.

[0129] In an alternative embodiment, the ray obtained based on the ray casting algorithm includes:

[0130] Obtain the homogeneous coordinate expressions of the farthest and nearest endpoints of the ray through matrix inverse operation, where V farWorld represents the homogeneous coordinate expression of the point farthest from the observer on the ray in the world coordinate system, and V nearWorld represents the homogeneous coordinate expression of the point nearest to the observer on the ray in the world coordinate system;

[0131] Convert the homogeneous coordinates (V farWorld , V nearWorld ) into three-dimensional points (P nearWorld , P farWorld ), and the conversion relationship is:

[0132] V farWorld = [x, y, z, w] T

[0133]

[0134] V nearWorld = [x, y, z, w] T

[0135]

[0136] where x represents the position in the horizontal direction, y represents the position in the vertical direction, z represents the position in the depth direction, i.e., height, and w is a scaling factor for matrix transformation;

[0137] Determine the ray projected by the touch point through the obtained three-dimensional point P nearWorld nearest to the observer and the point P farWorld farthest from the observer.

[0138] Furthermore, define the two endpoints P nearWorld and P farWorld of the ray, and the direction vector of the ray is calculated by the difference between these two endpoints:

[0139]

[0140] where P nearWorld is the starting point of the ray, P farWorld is the ending point of the ray, is the direction vector of the ray;

[0141] Define the surface equation of the cylinder as Solve for the parameter λ using the ray equation to determine the intersection coordinates. The ray equation is:

[0142]

[0143] where λ is the parameter representing the position of a point on the ray. When the ray intersects the cylinder, the value will satisfy the cylinder equation x and y are the coordinates of the point in the plane, is the distance from the point to the central axis of the cylinder;

[0144] Substitute the ray equation into the cylinder equation and solve to obtain the parameter λ of the intersection of the ray and the cylinder. Define the x and y coordinates in the ray equation as:

[0145] x(λ) = x near + λ·D x

[0146] y(λ) = y near + λ·D y

[0147] Substitute these two expressions into the cylinder equation to get:

[0148]

[0149] Based on the cylinder equation, we get:

[0150]

[0151] where R is the radius of the cylinder;

[0152] Use the quadratic formula to solve for λ:

[0153]

[0154] where ± indicates that the equation has two solutions, corresponding to the two intersections of the ray and the cylinder respectively. A is the coefficient of the λ 2 term, representing the sum of the squares of the components of the ray direction vector in the x and y directions. B is the coefficient of the λ term, representing the linear combination of the projections of the ray starting point and the ray direction vector in the x and y directions. C is the constant term, representing the relationship between the ray starting point and the cylinder radius R;

[0155] Among the two solutions, select the solution with the smaller λ value as the nearest intersection, and represent the solution with the larger λ value as the intersection of the ray and the cylinder in the direction away from the starting point.

[0156] In an alternative embodiment, when performing point cloud screening, it is necessary to calculate the shortest distance from each center point P in the point cloud to the ray, including: point Calculate the vector difference between the point P to be detected

[0157] and the near-end point P of the ray point nearWorld :

[0158]

[0159] Then calculate the cross product of the ray direction vector and the vector difference :

[0160]

[0161] wherein, represents a vector perpendicular to these two vectors, and its length is equal to the area of the parallelogram formed by these two vectors;

[0162] Calculate the magnitude of , which is the distance from the ray to the point P point :

[0163]

[0164] Calculate the magnitude of the ray direction vector :

[0165]

[0166] By dividing the magnitude of the cross product by the magnitude of the ray direction vector, the shortest distance d from the point to the ray is obtained:

[0167]

[0168] wherein, is the magnitude of the cross product, representing the projection distance from the point to the ray, while is the magnitude of the ray direction vector, representing the length of the ray.

[0169] Furthermore, the comparison of the threshold t is defined as the spatial length occupied by each point, and the threshold is calculated as:

[0170]

[0171] wherein, c is the initial number of point clouds, is the maximum value of the radial distance , z min is the minimum value of the height z, and z max is the maximum value of the height z. Among them:

[0172] ​If the minimum distance d min ≤ the threshold t, the point with the minimum distance is regarded as the point selected by the gesture;

[0173] If the minimum distance d min > the threshold t, it is regarded that the point selected by the gesture is not selected.

[0174] It should be noted that the embodiment of the present invention solves the problem of intersection determination in three-dimensional space. Through the implementation of the above steps, the intersection of the cornea and the cylinder can be accurately calculated, ensuring the accuracy of the cornea point selection. This calculation method avoids the intersection accumulation in three-dimensional space caused by inaccurate calculation in the traditional method, making the application of corneal recognition in complex three-dimensional space more reliable.

[0175] Preferably, the present invention quickly calculates the distance of each point, screens out the point closest to the ray as the effective target, and can exclude misidentified points, thereby improving the accuracy of the interaction system and ensuring the accuracy of point cloud screening.

[0176] Preferably, the embodiment of the present invention introduces matrix transformation and homogeneous coordinates, solves the problem of conversion between two-dimensional touch screen coordinates and three-dimensional space coordinates, and realizes the efficiency and accuracy of coordinate system conversion; at the same time, by setting the threshold t, points that are too far away can be filtered according to actual needs, preventing irrelevant points selected due to noise or other reasons, improving the operation accuracy of users, and thus optimizing the user interaction experience.

[0177] In order to better verify the beneficial effects of a point selection and ranging method based on three-dimensional point cloud data provided by the embodiment of the present invention, especially the advantages in terms of point selection accuracy and data processing efficiency, this experiment analyzes the actual effects of the present invention in processing complex three-dimensional point cloud data in detail through comparison with traditional methods, and demonstrates its superiority under various conditions; among them:

[0178] Currently, traditional three-dimensional point cloud processing methods (such as point selection methods based on Cartesian coordinate systems, ranging algorithms based on straight-line distance and Euclidean distance) select points through Euclidean distance or projection, ignoring the precise processing of viewing angle and data depth information, which may lead to accuracy errors. The ranging algorithms of traditional methods use simple geometric formulas for calculation, without considering the differences in sparsity and density of point cloud data. Traditional methods do not have the ability to adaptively adjust to the density change of point cloud data. When facing large-scale and dense point cloud data, the processing efficiency is low; that is, there are problems of poor point selection accuracy (especially in dense or sparse point cloud data, it is easy to misselect invalid points), low data processing efficiency (when facing complex scenes, it requires a long processing time and consumes a large amount of computing resources), and lack of adaptability (when processing point cloud data with different distributions, the algorithm performance is inconsistent).

[0179] In this experiment, two three-dimensional point cloud datasets with different densities and shapes were used. Dense point cloud: high-density point cloud data simulating urban buildings or industrial environments; Sparse point cloud: low-density point cloud data simulating remote sensing data or large-scale measurements; The experiment will be simulated through point cloud data obtained from a three-dimensional laser scanner. The generation method of the point cloud data is to randomly generate three-dimensional coordinates and add noise;

[0180] Experimental environmental conditions: High-performance Android phone Xiaomi 14 Pro (CPU: Snapdragon 8 Gen 3, Memory: 16GB, Body storage: 256GB);

[0181] Dense point cloud dataset: 2500*1000 to 5000*1000 three-dimensional points, and the coordinate range of each point is x, y, z ∈ [0, 10];

[0182] Sparse point cloud dataset: 1000*1000 to 2500*1000 three-dimensional points, and the coordinate range of each point is x, y, z ∈ [0, 10];

[0183] Target point: The target point is set as [5, 5, 5], and the accuracy of point selection is analyzed with this as a reference;

[0184] Experimental process: The traditional method uses the Euclidean distance method to select the closest point by calculating the Euclidean distance from each point to the target point, calculates the point selection error and processing time, and analyzes the point selection accuracy and processing efficiency; The method of the present invention combines ray casting and cylindrical coordinate system transformation of three-dimensional point cloud data, considers the perspective and depth information for point cloud selection, and uses an optimized distance algorithm to select the accurate target point through the limitation of the shortest distance and relative height;

[0185] In this experiment, the distance error between the selected point and the target point is calculated to show the error comparison, compare the point selection accuracy of the traditional method and the method of the present invention in dense point clouds and sparse point clouds, and judge the accuracy of point selection in turn; The processing time of the traditional method and the method of the present invention when processing point cloud data with different densities is recorded respectively, and the processing efficiency of the two is shown through a curve graph;

[0186] Refer to Figure 4 , it can be intuitively seen the time comparison between the traditional method and the method of the present invention when processing point cloud data with different densities. Among them, the horizontal axis is the point cloud data volume (density), the vertical axis is the processing time, the curve of the method of the present invention is a solid line, and the curve of the traditional method is a dotted line. According to Figure 4 shown, when processing point cloud data with different densities, the method of the present invention shows higher processing efficiency. As the data volume increases, the processing time of the method of the present invention increases slowly, and the time consumption is significantly reduced compared with the traditional method.

[0187] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A gesture point selection and distance measurement method in a three-dimensional point cloud perspective display mode, characterized in that: include: Acquire the two-dimensional coordinates of the touch point of the user's gesture on the screen through the touch screen of the smart mobile device; Normalizing the two-dimensional coordinates according to the width and height of the touch screen to obtain normalized coordinates, and converting the normalized coordinates into ray endpoints in a three-dimensional space coordinate system by combining an inverse matrix operation; The ray endpoints are converted into actual three-dimensional point coordinates, and then the three-dimensional point cloud data is converted from the Cartesian coordinate system to the cylindrical coordinate system to obtain the three-dimensional data in the cylindrical coordinate system. in, is the radial distance, θ is the angle, and z is the height; By solving the ray equation, determine whether the ray intersects with the cylinder. If the intersection exists and meets the preset conditions, the intersection is recorded; otherwise, it is deemed that there is no corresponding point within the current touch point range; After the intersection is recorded, the shortest distance R between the ray and the central axis of the cylinder is calculated. min , filter out Less than R min points, and filter out points that do not belong to the range of height z values ​​of the three-dimensional point cloud data; Traverse the filtered three-dimensional point cloud data, calculate the distance from each point to the ray, and obtain the point with the shortest distance and the minimum distance d min ;in: If the minimum distance d min ≤threshold t, the point with the smallest distance is regarded as the point selected by the gesture; If the minimum distance d min > threshold t, the gesture point is considered unselected.

2. The gesture point selection and distance measurement method in the three-dimensional point cloud perspective display mode according to claim 1, characterized in that: The method of solving the ray equation to determine whether the ray intersects with the cylinder, and recording the intersection point if the intersection point exists and meets the preset conditions, includes: The center point of the three-dimensional point cloud data is obtained by using the average value method to obtain the coordinates of the center point; Get radial distance The maximum value of With minimum The minimum value of height z min With the maximum value z max , record the one-to-one correspondence between the points in the cylindrical coordinate system and the points in the three-dimensional Cartesian rectangular coordinate system, and arrange them in sequence according to the size of the z value, that is, the three-dimensional point cloud data is regarded as wrapped in a point cloud center is the radius, z value range (z max , z min ) is a cylinder with a height of The two intersection points (P1, P2) of the ray and the cylinder are calculated by using the ray obtained based on the ray casting algorithm. If there is no intersection point, the point selection failure is directly returned, that is, there is no corresponding point within the current touch point range.

3. The gesture point selection and distance measurement method in the three-dimensional point cloud perspective display mode according to claim 2, characterized in that: The preset condition is defined as the z value of P1 being greater than the z value of P2, ie, z1 ≥ z2.

4. The gesture point selection and distance measurement method in the three-dimensional point cloud perspective display mode according to claim 2, characterized in that: The rays obtained based on the ray casting algorithm include: The homogeneous coordinates of the farthest and nearest two endpoints of the ray are obtained by matrix inversion operation, where V farWorld V represents the homogeneous coordinate expression of the point farthest from the observer in the world coordinate system. nearWorld Represents the homogeneous coordinate expression of the point where the ray is closest to the observer in the world coordinate system; The homogeneous coordinates (V farWorld , V nearWorld ) is converted into a three-dimensional point (P nearWorld , P farWorld ), the conversion relationship is: V farWorld =[x,y,z,w] T V nearWorld =[x,y,z,w] T Among them, x represents the position in the horizontal direction, y represents the position in the vertical direction, z represents the position in the depth direction, that is, the height, and w is a scaling factor used to implement matrix transformation; By obtaining the 3D point P closest to the observer nearWorld and the point P farthest from the observer farWorld , determine the ray cast by the touch point.

5. The gesture point selection and distance measurement method in the three-dimensional point cloud perspective display mode according to claim 4, characterized in that: Also includes: Define the two endpoints P of the ray nearWorld and P farWorld , the direction vector of the ray The difference between these two endpoints is calculated as: Among them, P nearWorld is the starting point of the ray, P farWorld is the end point of the ray, is the ray direction vector; The surface equation of the cylinder is defined as The parameter λ is solved by the ray equation to determine the coordinates of the intersection point. The ray equation is: Among them, λ is a parameter, which represents the position of a point on the ray. When the ray intersects the cylinder, the intersection point The value will satisfy the cylinder equation x and y are the coordinates of the point on the plane, is the distance from the point to the central axis of the cylinder; Substitute the ray equation into the cylinder equation and solve for the parameter λ of the intersection of the ray and the cylinder. Define the x and y coordinates in the ray equation as follows: x(λ)=x near +λ·D x y(λ)=y near +λ·D y Substituting these two expressions into the cylinder equation, we obtain: Based on the cylinder equation, we get: Where R is the radius of the cylinder; Use the quadratic equation solution formula to solve λ: Among them, ± means that the equation has two solutions, corresponding to the two intersection points of the ray and the cylinder, and A is λ 2 The coefficient of the term represents the ray direction vector The sum of the squares of the components in the x and y directions. B is the coefficient of the λ term, which represents the ray starting point and ray direction vector. The linear combination of the projections in the x and y directions, C is a constant term that represents the relationship between the starting point of the ray and the radius R of the cylinder; Of the two solutions, the solution with the smaller λ value is selected as the nearest intersection point, and the solution with the larger λ value represents the intersection point of the ray and the cylinder in a direction away from the starting point.

6. The gesture point selection and distance measurement method in the three-dimensional point cloud perspective display mode according to claim 1, characterized in that: When performing point cloud screening, it is necessary to calculate each center point P in the point cloud point The shortest distance to the ray, including: Calculate the point P to be detected point and the near end point P of the ray nearWorld The vector difference between: Then calculate the ray direction vector and vector difference The cross product of: in, represents a vector perpendicular to these two vectors, whose length is equal to the area of ​​the parallelogram formed by these two vectors; calculate The modulus length is the length from the ray to point P point Distance: Calculate the ray direction vector Module length: The shortest distance d from a point to a ray is found by dividing the magnitude of the cross product by the magnitude of the ray direction vector: in, is the modulus of the cross product, which represents the projection distance from the point to the ray, and It is the magnitude of the ray direction vector and represents the length of the ray.

7. The gesture point selection and distance measurement method in the three-dimensional point cloud perspective display mode according to claim 1, characterized in that: The three-dimensional point cloud data is converted from the Cartesian coordinate system to the cylindrical coordinate system to obtain the three-dimensional data in the cylindrical coordinate system. include: z=z Among them, the height z value remains unchanged, is the vertical distance between the point and the z-axis, that is, the radius in two-dimensional coordinates, and θ is the angle between the projection line of the ray on the xoy plane and the x-axis.

8. The gesture point selection and distance measurement method in the three-dimensional point cloud perspective display mode according to claim 1, characterized in that: The comparison of the threshold t is defined as the length of the space occupied by each point, and the threshold is calculated as: Among them, c is the number of initial point clouds, The radial distance The maximum value, z min is the minimum value of height z, z max is the maximum value of height z.

Images