Gesture point selection and distance measurement method in three-dimensional point cloud perspective display mode

Abstract

The application discloses a gesture point selection and distance measurement method in a three-dimensional point cloud perspective display mode, comprising the following steps: obtaining two-dimensional coordinates of a user gesture by using a touch screen of a smart mobile device, mapping the two-dimensional coordinates into a ray endpoint in a three-dimensional space coordinate system through normalization conversion and inverse matrix operation, converting the ray endpoint into actual three-dimensional coordinates, converting three-dimensional point cloud data from a Cartesian coordinate system into a cylindrical coordinate system, judging whether the ray intersects with a cylinder through a ray equation, screening intersection points meeting the conditions, calculating the shortest distance between the ray and the central axis of the cylinder, and determining whether a target point is selected through a distance threshold value. The application effectively avoids misoperation through minimum distance judgment and screening, improves the user's interactive experience, is suitable for various application scenarios such as augmented reality and virtual reality, optimizes the coordinate conversion and ray projection algorithm, and improves the response speed and accuracy of the system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional data processing and visualization, and in particular to a gesture point selection and distance measurement method in a three-dimensional point cloud perspective display mode. BACKGROUND

[0002] With the rapid development of augmented reality (AR), virtual reality (VR) and mixed reality (MR) technologies, three-dimensional point cloud data plays an increasingly important role in the accurate digitization of visual or scene surface shapes. These technologies are widely used in fields such as robot medical navigation, automatic navigation and image processing, providing users with a rich interactive experience. However, existing two-dimensional display methods have been unable to meet the user's needs for visual preview and operation of three-dimensional spatial information, and there is an urgent need to develop a three-dimensional point cloud perspective display mode suitable for smart mobile devices and its interactive technology.

[0003] Currently, the existing technical solutions have the following shortcomings:

[0004] First, the priority recognition accuracy and robustness are insufficient: the existing priority recognition algorithm is difficult to maintain storage and robustness in complex environments, resulting in inaccurate point selection operation.

[0005] Second, the coordinate conversion and beam conversion complexity is high: the existing coordinate conversion and beam conversion complexity is high, which is easily affected by factors such as device performance, data size and algorithm efficiency, resulting in slow processing speed and affecting user experience.

[0006] Third, the distance measurement function implementation is complex and the precision is limited: the existing distance measurement function implementation is complex and the precision is limited, which is difficult to meet the demand of high-precision measurement.

[0007] CN111831118A discloses a three-dimensional electroencephalogram display method and display system based on augmented reality, aiming to realize three-dimensional visualization display of electroencephalogram signals. It establishes a three-dimensional model of the cerebral cortex and a method of spatial positioning of brain signal electrophysiological sources, and displays the spatial distribution of electroencephalogram signals in the three-dimensional model of the cerebral cortex with different colors. However, this scheme mainly focuses on the three-dimensional visualization display of electroencephalogram signals, lacks support for relay interaction and distance measurement function of three-dimensional point cloud data, and cannot meet the user's perception and operation demand of three-dimensional spatial information.

[0008] CN104050859A discloses an interactive digital three-dimensional sand table system, aiming to realize interactive display of three-dimensional visual scene or three-dimensional model data. It generates a three-dimensional visual scene or three-dimensional model data display in a three-dimensional control stereoscopic image generation system, and realizes user interaction with the three-dimensional model. However, this system mainly focuses on the display of scenes or models, lacks support for pre-identification and distance measurement function of three-dimensional point cloud data, and cannot meet the accurate interaction demand of three-dimensional point cloud data. SUMMARY

[0009] This section is intended to summarize some aspects of the embodiments of the present application and briefly introduce some preferred embodiments, and some simplifications or omissions may be made in this section and the abstract and title of the specification to avoid obscuring the purpose of this section, the abstract and the title, and such simplifications or omissions cannot be used to limit the scope of the present application.

[0010] In view of the above existing problems, the present application is proposed.

[0011] Therefore, the present application aims to provide a high-precision, high-efficiency, three-dimensional point cloud perspective display mode and a point selection and distance measurement method, to improve user experience and meet the precise interaction requirements of three-dimensional point cloud data.

[0012] To solve the above technical problems, the present application provides the following technical solutions: obtaining the two-dimensional coordinates of the gesture touch point of the user on the screen through the touch screen of the intelligent mobile device;

[0013] According to the width and height of the touch screen, the two-dimensional coordinates are normalized and converted to obtain normalized coordinates, and the normalized coordinates are converted into the end point of the ray in the three-dimensional space coordinate system by combining the inverse matrix operation;

[0014] The end point of the ray is converted into the actual three-dimensional point coordinates, and the three-dimensional point cloud data is converted from the Cartesian coordinate system to the cylindrical coordinate system to obtain the three-dimensional data in the cylindrical coordinate system wherein, is the radial distance, θ is the angle, and z is the height;

[0015] By solving the ray equation, it is determined whether the ray intersects the cylinder, if the intersection point exists and meets the preset condition, the intersection point is recorded, otherwise it is considered that there is no corresponding point in the current touch point range;

[0016] After recording the intersection point, the shortest distance R min of the ray and the central axis of the cylinder is calculated Points less than R min are screened out, and points not belonging to the height z value range interval of the three-dimensional point cloud data are screened out;

[0017] The three-dimensional point cloud data after screening is traversed, the distance of each point to the ray is calculated, and the point with the minimum distance d min is obtained; wherein:

[0018] If the minimum distance d min ≤ threshold t, the point with the minimum distance is considered as the point selected by the gesture;

[0019] If the minimum distance dmin If the value is greater than the threshold t, then the gesture selection point is considered not selected.

[0020] As a preferred embodiment of the gesture point selection and distance measurement method in the three-dimensional point cloud perspective display mode of the present invention, the step of solving the ray equation to determine whether the ray intersects with the cylinder, and recording the intersection point if the intersection point exists and meets the preset conditions, includes:

[0021] The center point of the three-dimensional point cloud data is obtained by using the average value method, and the coordinates of the center point are obtained.

[0022] Obtain radial distance maximum value and minimum value Minimum value of height z min With the maximum value z max The system records the one-to-one correspondence between points in the cylindrical coordinate system and points in the three-dimensional Cartesian coordinate system, and arranges them sequentially according to their z-values. In other words, the three-dimensional point cloud data is considered to be enclosed in a point cloud centered at the point cloud. For radius, z-range (z max , z min () is a cylinder of height;

[0023] The two intersection points (P1, P2) between the ray obtained by the ray projection algorithm and the cylinder are calculated. If there is no intersection point, the selection point is returned as failed, that is, there is no corresponding point within the current touch point range.

[0024] As a preferred embodiment of the gesture point selection and distance measurement method in the three-dimensional point cloud perspective display mode described in this invention, the preset condition is defined as the z value of P1 being greater than the z value of P2, i.e., z1≥z2.

[0025] As a preferred embodiment of the gesture-based point selection and distance measurement method in the three-dimensional point cloud perspective display mode described in this invention, the ray obtained based on the ray projection algorithm includes:

[0026] The homogeneous coordinates of the farthest and nearest endpoints of a ray are obtained through matrix inversion, where V farWorld V represents the homogeneous coordinates of the point farthest from the observer in the world coordinate system. nearWorld The homogeneous coordinates of the point closest to the observer in the world coordinate system representing the ray;

[0027] The homogeneous coordinates (V) farWorld V nearWorld Transform into a three-dimensional point (P) nearWorld P farWorld The transformation relationship is:

[0028] V farWorld = [x, y, z, w]T

[0029]

[0030] V nearWorld = [x, y, z, w] T

[0031]

[0032] wherein x represents a position in the horizontal direction, y represents a position in the vertical direction, z represents a position in the depth direction, i.e. height, and w is a scaling factor for implementing a matrix transformation;

[0033] The three-dimensional point P closest to the observer is obtained by the distance nearWorld and the point P farthest from the observer farWorld , the ray projected by the touch point is determined.

[0034] As a preferred scheme of the gesture selection point and distance measurement method in the three-dimensional point cloud perspective display mode of the application, it further comprises:

[0035] Two end points P nearWorld and P farWorld of the ray are defined, and the direction vector of the ray is calculated by the difference between the two end points:

[0036]

[0037] wherein P nearWorld is the starting point of the ray, P farWorld is the end point of the ray, and is the direction vector of the ray;

[0038] The surface equation of the cylinder is defined as The parameter λ is solved by the ray equation to determine the intersection coordinates, and the ray equation is:

[0039]

[0040] wherein λ is a parameter, representing the position of a point on the ray, and when the ray intersects with the cylinder, the value of the intersection point will satisfy the cylinder equation x and y are the coordinates of the point on the plane, is the distance from the point to the center axis of the cylinder;

[0041] The ray equation is substituted into the cylinder equation to solve the parameter λ of the intersection point of the ray and the cylinder, and the x and y coordinates in the ray equation are defined as:

[0042] x(λ) = xnear + λ · D x

[0043] y(λ) = y near + λ · D y

[0044] Substitute the two expressions into the cylinder equation to obtain:

[0045]

[0046] Based on the cylinder equation, obtain:

[0047]

[0048] Wherein, R is the radius of the cylinder;

[0049] Solve λ by using the solution formula of the quadratic equation:

[0050]

[0051] Wherein, ± indicates that the equation has two solutions, which respectively correspond to two intersection points of the ray and the cylinder, A is the coefficient of the λ term, indicates the square sum of the components of the ray direction vector in the x and y directions, B is the coefficient of the λ term, indicates the linear combination of the projections of the ray origin and the ray direction vector in the x and y directions, and C is the constant term, indicating the relationship between the ray origin and the radius R of the cylinder. 2

[0052] Among the two solutions, the solution with the smaller λ value is selected as the nearest intersection point, and the solution with the larger λ value indicates that the intersection point of the ray and the cylinder is in the direction away from the origin.

[0053] As a preferred scheme of the gesture point selection and distance measurement method in the three-dimensional point cloud perspective display mode of the application, when the point cloud is selected, the shortest distance from each center point P point to the ray needs to be calculated, including:

[0054] Calculate the vector difference between the to-be-detected point P point and the near-end point P nearWorld of the ray:

[0055]

[0056] Then, calculate the cross product of the ray direction vector and the vector difference :

[0057]

[0058] Wherein,​​​ denotes a vector perpendicular to the two vectors, whose length is equal to the area of the parallelogram spanned by the two vectors;

[0059] The modulus of is calculated, which is the distance from the ray to the point P point :

[0060]

[0061] The modulus of the ray direction vector is calculated:

[0062]

[0063] The shortest distance d from the point to the ray is obtained by dividing the modulus of the cross product by the modulus of the ray direction vector:

[0064]

[0065] wherein is the modulus of the cross product, representing the projection distance from the point to the ray, and is the modulus of the ray direction vector, representing the length of the ray.

[0066] As a preferred scheme of the gesture point selection and distance measurement method in the three-dimensional point cloud perspective display mode, the three-dimensional point cloud data is converted from the Cartesian coordinate system to the cylindrical coordinate system, and the three-dimensional data in the cylindrical coordinate system is obtained , which comprises:

[0067]

[0068] z=z

[0069] wherein the height z value is constant, is the perpendicular distance of the point to the z axis, i.e. the radius in the two-dimensional coordinate, and θ is the included angle between the projection line of the ray in the xoy plane and the x axis.

[0070] As a preferred scheme of the gesture point selection and distance measurement method in the three-dimensional point cloud perspective display mode, the comparison of the threshold value t is defined as the length of the space occupied by each point, and the threshold value is calculated as:

[0071]

[0072] wherein c is the initial point cloud quantity, is the maximum value of the radial distance , z min is the minimum value of the height z, and z max is the maximum value of the height z.

[0073] The present application has the following advantages:

[0074] 1. The high-precision sensor of the touch screen can obtain the user's operation intention in real time, directly map the touch point to coordinate data, thus realizing an intuitive and flexible interaction mode, and further improving the user experience;

[0075] 2. The two-dimensional coordinates are converted to a unified scale through normalization processing, avoiding the problem of inconsistent coordinates caused by differences in screen size and resolution of different devices. Through this conversion, the normalized coordinates obtained can remain consistent under various devices and screen sizes, ensuring the compatibility and stability of the system. In addition, the normalized coordinates are converted to the end point of the ray in the three-dimensional coordinate system through inverse matrix operation, which ensures accurate mapping from two-dimensional screen operation to three-dimensional space, providing an accurate starting point for subsequent three-dimensional point cloud data processing and gesture interaction;

[0076] 3. By converting the three-dimensional point cloud data to a cylindrical coordinate system, more efficient data storage and processing can be achieved, especially in applications such as augmented reality and virtual reality, which exhibit good visual effects and enhance the system's ability to handle complex scenarios;

[0077] 4. The system effectively filters out the intersection points that meet the conditions, reducing unnecessary calculations and improving the overall system efficiency;

[0078] 5. By calculating the shortest distance between the ray and the central axis of the cylinder, the system can exclude irrelevant interference points and perform data filtering based on the set height range, ensuring efficient recognition of the user's actual interaction operation and avoiding false operations or excessive invalid point calculations, thus improving the system's response speed and accuracy;

[0079] 6. By calculating the distance from each filtered three-dimensional point to the ray, the system accurately determines the selected point of the user's intention, providing a precise interaction mode. In a virtual environment, users can select specific three-dimensional points through touch, then operate virtual objects or scenes, improving the intuitiveness of interaction and user experience;

[0080] 7. The system realizes efficient filtering and accurate recognition of gesture selection, ensuring that only when the user truly intends to select a point will the system provide the corresponding result. This not only effectively prevents false touches or interference, but also enhances the user's trust in the system, improving the intelligence and usability of the interaction. BRIEF DESCRIPTION OF DRAWINGS

[0081] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only represent some of the embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0082] Figure 1 A flowchart of a gesture point selection and distance measurement method in a three-dimensional point cloud perspective display mode shown in the present application;

[0083] Figure 2 A two-dimensional coordinate diagram of a smart mobile operating system screen shown in the present application;

[0084] Figure 3 A three-dimensional space coordinate system diagram shown in the present application;

[0085] Figure 4 A data processing efficiency comparison diagram of the present application method and the traditional method shown in the present application. DETAILED DESCRIPTION

[0086] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0087] Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort should belong to the protection scope of the present application.

[0088] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited by the specific embodiments disclosed below.

[0089] According to the embodiments of the present application, combined with the flowchart shown in the present application, a gesture point selection and distance measurement method in a three-dimensional point cloud perspective display mode, specifically includes the following steps: Figure 1 S1, obtaining the two-dimensional coordinates of the gesture touch point of the user on the screen through the touch screen of the smart mobile device;

[0090] S2, normalizing the two-dimensional coordinates according to the width and height of the touch screen to obtain normalized coordinates, and converting the normalized coordinates into a ray endpoint in a three-dimensional space coordinate system combined with inverse matrix operation;

[0091]

[0092] ​S3. Convert the ray endpoints to actual 3D point coordinates, and then convert the 3D point cloud data from Cartesian coordinates to cylindrical coordinates to obtain 3D data in cylindrical coordinates. in, θ is the radial distance, z is the angle, and z is the height.

[0093] S4. By solving the ray equation, determine whether the ray intersects the cylinder. If the intersection point exists and meets the preset conditions, record the intersection point; otherwise, it is considered that there is no corresponding point within the current touch point range.

[0094] S5. After recording the intersection points, calculate the shortest distance R between the ray and the central axis of the cylinder. min , screening Less than R min The points are selected and those that do not belong to the height z-value range of the 3D point cloud data are filtered out.

[0095] S6. Traverse the filtered 3D point cloud data, calculate the distance from each point to the ray, and obtain the point with the minimum distance and the minimum distance d. min ;in:

[0096] a) If the minimum distance d min If the distance is less than or equal to the threshold t, then the point with the smallest distance is considered as the point selected by the gesture;

[0097] b. If the minimum distance d min If the value is greater than the threshold t, then the gesture selection point is considered not selected.

[0098] It should be noted that the method of the present invention closely integrates the user's gesture operation with the interaction of 3D point cloud data, realizing high-precision and high-efficiency gesture interaction and point selection functions, solving the problems of accuracy, robustness and complexity in the prior art, and improving the user experience.

[0099] It should be further noted that the method of the present invention improves the accuracy of gesture recognition by optimizing the three-dimensional point cloud processing flow, effectively compresses the amount of data and reduces storage requirements by representing data in a cylindrical coordinate system, and effectively reduces the computational complexity by converting the point coordinates from the rectangular coordinate system to the cylindrical coordinate system for processing, thereby improving the real-time performance of gesture recognition and ranging, meeting the needs of rapid response in practical applications, and enabling users to receive immediate feedback from the system when performing gesture operations.

[0100] The following is combined with Figures 2-3 The schematic diagrams shown, along with some preferred or optional examples of the present invention, more specifically describe the implementation process and / or effects of certain embodiments of the present invention.

[0101] Reference Figure 2, the user touches a point on the touch screen to perform a point selection operation, that is, the user taps a certain point on the screen with a finger or a stylus, and an intelligent mobile operating system (such as Android or IOS) takes the certain point on the screen as an origin of a two-dimensional coordinate system of the screen, for example, the origin of the two-dimensional coordinate system of the screen is defined at a top left corner vertex of the screen, and the position of any point on the screen is determined by a coordinate value in a horizontal direction (x-axis) extending from left to right and a coordinate value in a vertical direction (y-axis) extending from top to bottom.

[0102] With reference to Figure 3 When the user performs a touch operation on the touch screen, the two-dimensional coordinates (x, y) of the touch point can be obtained by the operating system, where x is a numerical value of the touch point in the x-axis, and y is a numerical value of the touch point in the y-axis, and the unit is pixel.

[0103] In an optional embodiment, the Android operating system uses OpenGL (a programming graphical interface across programming languages and platforms) to transform the point cloud data from a local coordinate into a screen coordinate, and finally displays the three-dimensional point cloud information on the screen of the Android system. OpenGL uses a homogeneous coordinate system (x, y, z, w) to represent a three-dimensional space, where x represents a position in the horizontal direction, y represents a position in the vertical direction, z represents a position in the depth direction, and w is the fourth component of the homogeneous coordinate, which is a scaling factor for implementing various matrix transformations (such as translation, rotation, scaling, and projection).

[0104] The three-dimensional point cloud data is input, and at this time, the three-dimensional point cloud data is in a raw local space in a Cartesian coordinate system, and the initial homogeneous coordinates of the three-dimensional point cloud data are represented as [x, y, z, 1] T ;

[0105] According to the relative position, the relative rotation, and the relative scaling information of the world space, the point cloud coordinates are transformed from the local space to the world space through a model matrix, and the object is restored to the distribution in the real world. The model matrix is equivalent to a set of translation, rotation, and scaling operations, and the relative positions between objects can be determined by calculating the model matrix for each object.

[0106] The world space coordinates are converted into coordinates in front of the user's field of view through a view matrix, and the scene is translated or rotated so that a specific object is transformed to the front of the camera.

[0107] A projection operation is performed on the coordinates reaching the view space, and the coordinates are clipped to a range of [-1, 1].

[0108] The coordinates in the range of [-1, 1] are transformed into screen coordinates through a viewport transformation, and the transformed coordinates are transmitted to a rasterizer and displayed on the screen after being converted into fragments.

[0109] For example, the coordinate transformation operations for displaying a 3D point cloud in OpenGL are as follows:

[0110] V clip =M projection ·M view ·M model ·[x,y,z,1] T

[0111] Among them, V clip M represents the homogeneous coordinates of the normalized vertices displayed on the OpenGL screen. projection M represents the projection matrix, indicating how to project an object in three-dimensional space onto a two-dimensional screen. view The observation matrix represents the camera's position, orientation, and field of view. M model This is the model matrix, representing the object's position, rotation, and scaling, [x, y, z, 1]. T This is a homogeneous representation of the Cartesian coordinates of a 3D point cloud.

[0112] In an optional implementation, the two-dimensional coordinates (x, y) of the touch point are transformed into normalized coordinates (x, y). normalized ,y normalized The conversion method is as follows:

[0113]

[0114] Where width is the screen width and height is the screen height. In OpenGL, the normalized coordinate z-value ranges from [1, -1], representing the depth range in clip space. The normalized point furthest from the observer is (x...). normalized ,y normalized The normalized point closest to the observer for the ray is (x - 1). normalized ,y normalized ,1).

[0115] As an example, the homogeneous coordinates of the farthest and nearest endpoints of a ray are obtained through matrix inversion, including:

[0116] Once the point cloud is displayed on the screen, M is known. projection ·M view ·M model At this point, M can be calculated. projection ·M view ·M model The inverse matrix M invert :

[0117] V farWorld =M invert ·[x normalized ,y normalized ,1,1] T

[0118] V nearWorld = M invert · [x normalized , y normalized , -1, 1] T

[0119] wherein V farWorld represents the homogeneous coordinate expression of the point farthest from the observer in the ray distance in the world coordinate system, and V nearWorld represents the homogeneous coordinate expression of the point closest to the observer in the ray distance in the world coordinate system.

[0120] In an optional embodiment, the center point O of the three-dimensional point cloud is pre-calculated, the x, y, and z coordinates of all points in the point cloud are averaged respectively to obtain the coordinates of the center point O, and then the three-dimensional point cloud data is converted from the Cartesian coordinate system to the cylindrical coordinate system to obtain the three-dimensional data in the cylindrical coordinate system including:

[0121]

[0122] z = z

[0123] wherein the height z value is constant, is the perpendicular distance of the point to the z axis, that is, the radius in the two-dimensional coordinate, and θ is the included angle between the projection line of the ray in the xoy plane and the x axis.

[0124] As an example, by solving the ray equation, it is determined whether the ray intersects the cylinder, and if the intersection point exists and meets the preset condition, the intersection point is recorded, including:

[0125] The center point of the three-dimensional point cloud data is calculated by using the average value method to obtain the center point coordinates;

[0126] The maximum value and the minimum value of the radial distance The minimum value z min and the maximum value z max of the height z are obtained, the one-to-one correspondence between the points in the cylindrical coordinate system and the points in the three-dimensional Cartesian rectangular coordinate system is recorded, and they are arranged in order according to the z value, that is, the three-dimensional point cloud data is regarded as wrapped in a cylinder with the center of the point cloud as the radius and the z value range (z max , z min ) as the height;

[0127] Two intersection points (P1, P2) of the ray and the cylinder are calculated through the ray obtained based on the ray projection algorithm, and if there is no intersection point, the selection point fails, that is, there is no corresponding point in the current touch point range.

[0128] For example, the preset condition is defined as z value of P1 is greater than z value of P2, i.e. z1≥z2.

[0129] In an optional embodiment, the ray obtained based on the ray casting algorithm comprises:

[0130] The homogeneous coordinate expression of the farthest and nearest endpoints of the ray is obtained by matrix inversion, wherein V farWorld represents the homogeneous coordinate expression of the point farthest from the observer in the world coordinate system, V nearWorld represents the homogeneous coordinate expression of the point closest to the observer in the world coordinate system;

[0131] The homogeneous coordinates (V farWorld , V nearWorld ) are converted into three-dimensional points (P nearWorld , P farWorld ), and the conversion relationship is:

[0132] V farWorld = [x, y, z, w] T

[0133]

[0134] V nearWorld = [x, y, z, w] T

[0135]

[0136] wherein x represents the position in the horizontal direction, y represents the position in the vertical direction, z represents the position in the depth direction, i.e. height, and w is a scaling factor for realizing matrix transformation;

[0137] The ray projected by the touch point is determined by the obtained three-dimensional point P nearWorld closest to the observer and the point P farWorld farthest from the observer.

[0138] Further, the two endpoints P nearWorld and P farWorld of the ray are defined, and the direction vector of the ray is calculated by the difference between the two endpoints:

[0139]

[0140] wherein P nearWorld is the starting point of the ray, P farWorld is the end point of the ray, and is the direction vector of the ray;

[0141] The surface equation of the cylinder is defined as The parameter λ is solved by a ray equation to determine the intersection coordinates, the ray equation is:

[0142]

[0143] Where λ is a parameter, representing the position of a point on the ray, when the ray intersects the cylinder, the intersection point will satisfy the cylinder equation x and y are the coordinates of the point on the plane, is the distance from the point to the center axis of the cylinder; Substitute the ray equation into the cylinder equation, and solve to get the intersection point of the ray and the cylinder parameter λ, define the x and y coordinates in the ray equation as:

[0144] x(λ) = x near + λ·D x

[0145] y(λ) = y near + λ·D y

[0146] Substitute these two expressions into the cylinder equation to get:

[0147]

[0148] Based on the cylinder equation, we get:

[0149]

[0150] Where R is the radius of the cylinder;

[0151] Use the quadratic equation solving formula to solve λ:

[0152]

[0153] Where ± indicates that the equation has two solutions, corresponding to the two intersection points of the ray and the cylinder, A is the coefficient of the λ term, representing the square sum of the components of the ray direction vector 2 in the x and y directions, B is the coefficient of the λ term, representing the linear combination of the projections of the ray origin and the ray direction vector

[0154] in the x and y directions, C is the constant term, representing the relationship between the ray origin and the cylinder radius R. Of the two solutions, choose the smaller λ value as the nearest intersection point, and the larger λ value represents the intersection point of the ray and the cylinder in the direction away from the origin.

[0155]

[0156] ​​In an optional implementation, when the point cloud screening is performed, the shortest distance from each center point P point to the ray needs to be calculated, including:

[0157] The vector difference between the to-be-detected point P point and the ray proximal point P nearWorld is calculated:

[0158]

[0159] The cross product of the ray direction vector and the vector difference is calculated:

[0160]

[0161] wherein, represents a vector perpendicular to the two vectors, and the length thereof is equal to the area of a parallelogram spanned by the two vectors;

[0162] The modulus of is calculated, that is, the distance from the ray to the point P point :

[0163]

[0164] The modulus of the ray direction vector is calculated:

[0165]

[0166] The shortest distance d from the point to the ray is obtained by dividing the modulus of the cross product by the modulus of the ray direction vector:

[0167]

[0168] wherein, is the modulus of the cross product, represents the projection distance from the point to the ray, and is the modulus of the ray direction vector, represents the length of the ray.

[0169] Further, the comparison of the threshold value t is defined as the length of the space occupied by each point, and the threshold value is calculated as:

[0170]

[0171] wherein c is the initial point cloud quantity, is the maximum value of the radial distance , z min is the minimum value of the height z, and z max is the maximum value of the height z. Wherein:

[0172] If the minimum distance d min is less than the threshold t, the point with the minimum distance is regarded as the selected point of the gesture;

[0173] If the minimum distance d min is greater than the threshold t, it is regarded that the gesture-selected point is not selected.

[0174] It should be noted that the embodiment of the present application solves the intersection determination problem in three-dimensional space, and through the implementation of the above steps, the intersection of the cornea and the cylinder can be accurately calculated, and the accuracy of the cornea selection point is ensured. This calculation method avoids the accumulation of intersections in three-dimensional space due to inaccurate calculation in the traditional method, so that the application of cornea recognition in complex three-dimensional space is more reliable.

[0175] Preferably, the present application can exclude the points of false recognition by quickly calculating the distance of each point and selecting the point closest to the ray as the effective target, thereby improving the accuracy of the interactive system and ensuring the accuracy of the point cloud screening.

[0176] Preferably, the embodiment of the present application introduces matrix transformation and homogeneous coordinates to solve the conversion problem between two-dimensional touch screen coordinates and three-dimensional space coordinates, and realizes the efficiency and accuracy of coordinate system conversion. At the same time, by setting the threshold t, the points with too far distance can be filtered according to the actual demand, so as to prevent the selection of irrelevant points due to noise or other reasons, improve the operation accuracy of the user, and thus optimize the interactive experience of the user.

[0177] In order to better verify the beneficial effects of the point selection and distance measurement method based on three-dimensional point cloud data provided by the embodiment of the present application, especially the advantages in point selection accuracy and data processing efficiency, the present experiment compares with the traditional method to analyze the actual effect of the present application in processing complex three-dimensional point cloud data, and demonstrates its superiority under various conditions. Among them:

[0178] At present, the traditional three-dimensional point cloud processing method (such as the point selection method based on Cartesian coordinate system, the distance measurement algorithm based on straight line distance and Euclidean distance) selects points by Euclidean distance or projection, ignores the accurate processing of viewing angle and data depth information, and may cause precision error. The distance measurement algorithm of the traditional method uses simple geometric formula for calculation, and does not consider the sparsity and density difference of point cloud data. The traditional method has no adaptive adjustment ability for the density change of point cloud data, and the processing efficiency is low when facing large-scale and dense point cloud data; that is, there are problems of poor point selection accuracy (especially in dense or sparse point cloud data, invalid points are easily misselected), low data processing efficiency (when facing complex scenes, long processing time is needed, and large amount of computing resources is consumed), and no adaptive characteristics (when processing point cloud data with different distribution, the algorithm performance is inconsistent).

[0179] The experiment uses two different density and shape three-dimensional point cloud data sets, dense point cloud: high-density point cloud data simulating urban buildings or industrial environments; sparse point cloud: low-density point cloud data simulating remote sensing data or large-scale measurement; the experiment will be simulated by point cloud data obtained from a three-dimensional laser scanner, and the point cloud data is generated by randomly generating three-dimensional coordinates and adding noise;

[0180] Experimental environment conditions: high-performance Android phone Xiaomi 14pro (CPU: Snapdragon 8gen3, memory: 16GB, body storage: 256GB);

[0181] Dense point cloud data set: 2500*1000 to 5000*1000 three-dimensional points, and the coordinate range of each point is x, y, z [0, 10];

[0182] Sparse point cloud data set: 1000*1000 to 2500*1000 three-dimensional points, and the coordinate range of each point is x, y, z [0, 10];

[0183] Target point: set the target point as [5, 5, 5] to analyze the accuracy of point selection;

[0184] Experimental process: the traditional method uses the Euclidean distance method to select the closest point by calculating the Euclidean distance from each point to the target point, calculate the point selection error and processing time, analyze the point selection accuracy and processing efficiency; the method of the present application combines the ray projection of three-dimensional point cloud data and the cylindrical coordinate system transformation, considers the perspective and depth information for point cloud point selection, uses the optimized distance algorithm, selects the accurate target point by limiting the shortest distance and relative height;

[0185] The experiment shows the error comparison by calculating the distance error between the selected point and the target point, compares the point selection accuracy of the traditional method and the method of the present application in dense point cloud and sparse point cloud, and judges the point selection accuracy in turn; and records the processing time of the traditional method and the method of the present application when processing different density point cloud data, and shows the processing efficiency of the two through a curve;

[0186] Reference Figure 4 It can be seen that the processing time of the traditional method and the method of the present application when processing different density point cloud data, wherein the horizontal axis is the point cloud data amount (density), and the vertical axis is the processing time, the curve of the method of the present application is a solid line, and the curve of the traditional method is a dashed line, according to Figure 4 The method of the present application shows higher processing efficiency when processing different density point cloud data, and the processing time of the method of the present application increases slowly with the increase of data amount, and the time consumption of the traditional method is obviously reduced.

[0187] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced, without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A method for gesture point selection and distance measurement in a three-dimensional point cloud perspective display mode, characterized in that, Comprising: Obtaining two-dimensional coordinates of gesture touch points of a user on a screen through a touch screen of a smart mobile device; Converting the two-dimensional coordinates into normalized coordinates according to the width and height of the touch screen, and converting the normalized coordinates into a ray endpoint in a three-dimensional space coordinate system through inverse matrix operation; The ray end point is converted into actual three-dimensional point coordinates, and the three-dimensional point cloud data is converted from a Cartesian coordinate system into a cylindrical coordinate system to obtain three-dimensional data in the cylindrical coordinate system wherein, is a radial distance, is an angle, is a height; Solving a ray equation to determine whether the ray intersects with a cylinder, and if an intersection point exists and meets a preset condition, recording the intersection point, otherwise, considering that there is no corresponding point in the current touch point range; after the intersection point is recorded, the shortest distance between the ray and the central axis of the cylinder is calculated , points less than are screened out , and points not belonging to the height z value range interval of the three-dimensional point cloud data are screened out; Traverse the screened three-dimensional point cloud data, calculate the distance of each point to the ray, get the point with the minimum distance and the minimum distance ; wherein: If the minimum distance ≤ threshold t, the point with the minimum distance is considered as the point selected by the gesture; If the minimum distance is greater than the threshold t, the gesture point is considered not selected. When performing point cloud filtering, it is necessary to calculate the center point of each point cloud. The shortest distance to the ray includes: the vector difference between the detected point and the ray proximal point and the ray proximal point ​ recalculate the ray direction vector and the vector difference the cross product of: wherein denotes a vector perpendicular to the two vectors, whose length is equal to the area of the parallelogram spanned by the two vectors; The length of the segment of the line from the point to the point is the distance from the point to the point Computing a ray direction vector of the module: Obtaining the shortest distance d from a point to a ray by dividing the modulus of a cross product by the modulus of a ray direction vector: where is the magnitude of the cross product, representing the distance of the point to the ray, and is the magnitude of the ray direction vector, representing the length of the ray.

2. The gesture point picking and ranging method in three-dimensional point cloud perspective display mode according to claim 1, characterized in that, The solving of the ray equation to determine whether the ray intersects with the cylinder, and if the intersection point exists and meets the preset condition, recording the intersection point, comprises: Obtaining a center point of the three-dimensional point cloud data by using an average method to obtain a center point coordinate; Obtaining a maximum value of a radial distance Obtaining a minimum value of a radial distance Obtaining a maximum value of a radial distance Obtaining a minimum value of a radial distance Obtaining a maximum value of a radial distance Obtaining a minimum value of a radial distance Recording a one-to-one correspondence between the point in the cylindrical coordinate system and the point in the three-dimensional Cartesian rectangular coordinate system, and arranging in order according to the z value, that is, the three-dimensional point cloud data is regarded as being wrapped in a cylindrical body with a point cloud center as a radius and a z value range , as a height. By means of the ray obtained based on the ray projection algorithm, two intersection points of the ray and the cylinder are calculated , ), if there is no intersection point, directly return that the selected point fails, i.e. there is no corresponding point in the current touch point range.

3. The method of claim 2, wherein, The preset condition is defined as z value of z value, i.e. .

4. The gesture point picking and ranging method in three-dimensional point cloud perspective display mode according to claim 2, characterized in that, The ray obtained based on the ray projection algorithm comprises: The homogeneous coordinate representation of the farthest and nearest endpoints of the ray is solved by matrix inversion, wherein, represents the homogeneous coordinate representation of the point farthest from the observer in the world coordinate system, represents the homogeneous coordinate representation of the point closest to the observer in the world coordinate system; transforming the homogeneous coordinates (x, y, z, 1) , ) into three-dimensional points (x, y, z) , ) with the transformation relationship wherein denotes a position in horizontal direction, denotes a position in vertical direction, denotes a position in depth direction, i.e. height, is a scaling factor for implementing the matrix transformation; By obtaining Three-dimensional points And the point farthest from the observer Determine the ray on which the touch point projects.

5. The gesture point picking and ranging method in three-dimensional point cloud perspective display mode according to claim 4, characterized in that, Further comprising: defining two end points of a ray and a direction vector of the ray is calculated by the difference of the two end points wherein, is the origin of the ray, is the end of the ray, is the ray direction vector; The surface equation of the cylinder is defined as The parameters are solved by a ray equation to determine the intersection coordinates, the ray equation being: wherein, is a parameter representing the position of a point on the ray, the value of the point of intersection when the ray intersects the cylinder will satisfy the cylinder equation , and are the coordinates of the point in the plane, is the distance of the point to the center axis of the cylinder; Substituting the ray equation into the cylinder equation, the parameters of the intersection of the ray and the cylinder are solved , the coordinates of and in the ray equation are defined as: Substituting the two expressions into a cylinder equation to obtain: Based on the cylinder equation, obtaining: Wherein, R is the radius of the cylinder; solving formula of quadratic equation : where, represents the equation has two solutions, corresponding to the two intersection points of the ray and the cylinder, A is the coefficient of the term, representing the direction vector of the ray In and The square sum of the components in the direction, B is the coefficient of the term, representing the starting point of the ray and the direction vector of the ray In and The linear combination of the projections in the direction, C is the constant term, representing the relationship between the starting point of the ray and the radius R of the cylinder; Of the two solutions, the one with the smaller value is chosen as the closest intersection, and the one with the larger value indicates that the intersection of the ray with the cylinder is in the direction away from the origin.

6. The method of claim 1, wherein, The three-dimensional point cloud data is converted from a Cartesian coordinate system to a cylindrical coordinate system to obtain three-dimensional data in the cylindrical coordinate system , comprising: z z where the height z value is constant, is the perpendicular distance of the point from the z-axis, i.e. the radius in two-dimensional coordinates, is the angle between the projection of the ray on the xoy plane and the x-axis.

7. The method of claim 1, wherein, The comparison of the threshold value t is defined as the length of the space occupied by each point, and the threshold value is calculated as: where c is the initial point cloud quantity, is the maximum value of the radial distance , is the minimum value of the height , is the maximum value of the height z.

Images