Curved surface deviation detection method based on virtual-real superposition and multi-user cooperation

Abstract

The invention discloses a curved surface deviation detection method based on virtual-real superposition and multi-user cooperation, and relates to the technical field of deviation detection. Comprising the steps of calculating point cloud data to obtain a curvature feature matrix; the reverse model data and the real object point cloud data are aligned, curved surface deviation detection operation is executed, a deviation field is coded, and preliminary visual display is carried out in a VR / AR environment; a plurality of users carry out collaborative recheck based on the deviation field, dynamic visual projection is carried out on annotation space coordinates according to curved surface curvature features, meanwhile, the annotation space coordinates, user gesture tracks and text keywords are jointly coded, a two-dimensional R tree annotation index is established, and annotation data and a deviation field result are spatially correlated; and dynamically updating the visual content and correcting the deviation according to the spatial association. According to the method, the problems of inaccurate alignment, low rechecking efficiency, difficulty in coverage of a blind area and the like in curved surface deviation detection are effectively solved, and the accuracy, interpretability and team cooperation efficiency of a detection result are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of deviation detection, and particularly to a curved surface deviation detection method for virtual-real superposition and multi-user collaboration. Background Art

[0002] With the continuous development of high-precision measurement technology and virtual reality / augmented reality (VR / AR) technology, the industrial manufacturing field has put forward higher requirements for collaborative, visual and intelligent complex curved surface detection and verification. Traditional three-dimensional measurement methods rely on single-point measurement or regular grid structures, and it is difficult to cover the detailed changes in complex structure areas such as free-form surfaces. In recent years, technologies such as point cloud data processing, three-dimensional reconstruction, and deviation analysis have gradually matured. Especially with the cooperation of high-precision devices (such as laser scanners and structured light systems), the curved surface detection has evolved from single measurement to multi-source data fusion and interactive visualization. On this basis, the introduction of multi-user collaboration mechanisms and visual interaction means has further improved the flexibility and verification efficiency of detection, becoming an important development direction for intelligent detection of complex structures.

[0003] CN107421462A provides a three-dimensional contour measurement system for objects based on line laser scanning. It mainly realizes contour data acquisition through the cooperation of left and right image sensors and a line laser, and has the characteristics of simple system structure, small imaging error, and stable data. This method can efficiently obtain the geometric information of objects under static conditions, but its detection process mainly focuses on the single-user static measurement stage, lacking subsequent data visualization verification process. In particular, it does not involve the fusion processing of point cloud data and CAD models, nor does it support deviation field visualization and collaborative annotation in the VR / AR environment, and it is difficult to meet the multi-person verification and dynamic feedback requirements in complex structure scenarios.

[0004] CN109813219B proposes a method and system for collecting and processing information on the detection, appraisal and reinforcement of existing structures, emphasizing the realization of data sharing and transfer during the structure detection, appraisal and subsequent construction processes, effectively reducing the frequency of repeated modeling and on-site operations. The prominent feature of this solution is the unified management and display of detection information in multiple stages, but its core still focuses on the macroscopic collection of structural information and BIM visual presentation. It does not involve refined curved surface deviation calculation and spatial precise alignment processing of point cloud and reverse model, and at the same time does not introduce multi-user interaction and spatial annotation mechanisms, and there are certain limitations for microscopic analysis such as abnormal curvature and local deviation.

[0005] CN104137030A discloses a method for measuring a three-dimensional sample and the measuring device including a laser scanning microscope. By interactively controlling the measuring action and visualizing the measuring result in a virtual space, a linkage mapping between the measuring space and the virtual space is realized. Although this method is innovative in the field of microscopic scale measurement, its main objectives are virtual control of the operation and observation feedback. It has not yet constructed a geometric difference expression mechanism between the point cloud and the model, nor established a multi-user collaboration annotation model at the spatial coordinate level, and thus cannot meet the requirements of multi-person joint decision-making, joint review, and automatic report generation in industrial complex structures.

[0006] In summary, the existing surface detection technologies generally have the following problems: First, there is a lack of a fusion analysis mechanism for point cloud data and reverse engineering models, making it impossible to achieve accurate spatial alignment and deviation quantification; second, there is a lack of a multi-user collaboration mechanism and annotation index structure, making it difficult to meet the requirements of collaborative review and dynamic interaction; third, the visualization means are relatively primitive, and the VR / AR environment has not been fully utilized to realize the real-time interactive presentation of the deviation field. Summary of the Invention

[0007] In view of the problems existing in the existing surface detection technologies, the present invention is proposed.

[0008] Therefore, the problem to be solved by the present invention is how to realize an integrated process of high-precision surface deviation detection, spatial annotation review, and automatic report generation, and improve the intelligence and collaboration level of complex structure detection.

[0009] To solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides a surface deviation detection method for virtual-real superposition and multi-user collaboration, which includes collecting point cloud data of a target physical object, calculating a curvature feature matrix from the point cloud data; at the same time, associatively storing the point cloud data and the reverse engineering model data; performing a six-degree-of-freedom alignment operation on the reverse engineering model data and the physical object point cloud data, and based on the aligned spatial coordinates, performing a surface deviation detection operation, encoding the detection result as a deviation field, and performing preliminary visualization display in a VR / AR environment in a color mapping and contour line manner; multiple users perform collaborative review based on the deviation field, perform dynamic visualization projection on the annotation spatial coordinates according to the surface curvature characteristics, and at the same time jointly encode the annotation spatial coordinates, user gesture trajectories, and text keywords, establish a two-dimensional R-tree annotation index, and perform spatial association between the annotation data and the deviation field result; dynamically update the visualization content and deviation correction according to the spatial association, extract multi-user review data, and automatically generate a detection report.

[0010] As a preferred embodiment of the surface deviation detection method for virtual-real superposition and multi-user collaboration according to the present invention, wherein: the generation of the curvature feature matrix includes: screening local neighborhoods for point cloud data, selecting a number of neighbor points for each data point using a certain neighborhood radius; performing least squares plane fitting on the selected local neighborhoods to obtain the local fitting plane parameters of each neighborhood point; calculating the curvature tensor of each data point based on the local fitting plane, and extracting the principal curvature values of each point; using the principal curvature values of the curvature tensor to calculate the Gaussian curvature and mean curvature of each data point respectively, and constructing a curvature feature matrix according to the spatial positions of the points.

[0011] As a preferred embodiment of the surface deviation detection method for virtual-real superposition and multi-user collaboration according to the present invention, wherein: the six-degree-of-freedom alignment operation includes: extracting feature points from the reverse engineering model data and the physical point cloud data, and performing matching between the feature points; performing six-degree-of-freedom alignment on the reverse engineering model data and the physical point cloud data based on the feature point matching results.

[0012] As a preferred embodiment of the surface deviation detection method for virtual-real superposition and multi-user collaboration according to the present invention, wherein: the execution of the surface deviation detection operation includes: calculating the surface deviation between the reverse engineering model data and the physical point cloud data based on the aligned spatial coordinates; generating a deviation field containing spatial positions and deviation values by analyzing the distance errors of each feature point.

[0013] As a preferred embodiment of the surface deviation detection method for virtual-real superposition and multi-user collaboration according to the present invention, wherein: the dynamic visualization projection includes: when the user makes a comment, judging the spatial coordinate point where the comment is located through the surface curvature feature matrix, and judging whether the comment coordinate point is located in the visual blind area according to the position and shape of the surface; when the comment is located in the surface blind area, projecting it along the surface normal direction to the visible area based on the comment spatial coordinates, and generating a dynamic guiding line to guide the user's line of sight to the actual position of the comment.

[0014] As a preferred embodiment of the surface deviation detection method for virtual-real superposition and multi-user collaboration according to the present invention, wherein: the spatial association of the comment data and the deviation field results includes: jointly encoding the spatial coordinates, user gesture trajectories, and text keywords of each comment, and establishing a spatial index of the comment data through a two-dimensional R-tree structure; quickly querying and matching the spatial coordinates in the comment data with the deviation field data in the detection results through the established two-dimensional R-tree index; confirming the specific detection result positions corresponding to each comment according to the spatial matching results of the comment spatial coordinates and the deviation field; when new user comments are added or modified, dynamically adjusting the spatial matching of the comments and the deviation field according to the R-tree index, and real-time updating the spatial association data.

[0015] As a preferred solution of the surface deviation detection method for virtual-real superposition and multi-user collaboration according to the present invention, wherein: the dynamic update of the visualization content and deviation correction includes: querying the spatial association relationship between the current annotation space coordinates and the deviation values in the deviation field based on the two-dimensional R-tree index, and screening out the deviation field data associated with the annotation area according to the principal curvature values at the corresponding positions in the curvature feature matrix; performing gradient descent optimization on the associated deviation field data according to the user annotation, adjusting the spatial distribution of the deviation field, and updating the deviation values in the visualization content based on color mapping.

[0016] As a preferred solution of the surface deviation detection method for virtual-real superposition and multi-user collaboration according to the present invention, wherein: the screening out of the deviation field data associated with the annotation area includes: extracting the set of user annotation coordinate points based on the two-dimensional R-tree index, and traversing the spatial coordinates of all data points in the deviation field; for each annotation coordinate point, locating the same coordinate point in the curvature feature matrix according to the spatial mapping relationship after six-degree-of-freedom alignment, and extracting the principal curvature value; calculating the difference between the principal curvature of the data points in the deviation field and the principal curvature of the corresponding annotation area, and removing the deviation field data points with the difference less than the preset curvature tolerance threshold.

[0017] In a second aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and wherein: when the computer program instructions are executed by the processor, the steps of the surface deviation detection method for virtual-real superposition and multi-user collaboration as described in the first aspect of the present invention are implemented.

[0018] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and wherein: when the computer program instructions are executed by the processor, the steps of the surface deviation detection method for virtual-real superposition and multi-user collaboration as described in the first aspect of the present invention are implemented.

[0019] The beneficial effects of the present invention are as follows: By constructing a surface deviation detection method for virtual-real superposition and multi-user collaboration, the present invention realizes high-precision detection and efficient collaborative review of complex surface structures, significantly improving the intelligent and practical levels of deviation detection. By introducing a curvature feature matrix, a six-degree-of-freedom alignment mechanism, and deviation field coding, the geometric accuracy of the detection results and the integrity of spatial expression can be effectively improved, enhancing the ability to identify and analyze complex surface errors; with the visualization mapping method in the VR / AR environment, the intuitiveness and interactivity of the detection results are improved, helping users quickly understand and locate the deviation area; in addition, through the multi-user collaborative annotation and dynamic visualization projection mechanism, multi-perspective review and intelligent guidance are realized, greatly reducing the risks of misjudgment and missed detection. The spatial indexing and dynamic update mechanism between the annotation data and the deviation information further support the real-time optimization and correction of the detection results, facilitating the formation of high-quality and structured detection records.

[0020] In summary, the present invention effectively solves the problems existing in surface deviation detection, such as inaccurate alignment, low review efficiency, and difficulty in covering blind areas, and improves the accuracy, interpretability of the detection results and the team collaboration efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0022] Figure 1 It is a flowchart of a surface deviation detection method for virtual-real superposition and multi-user collaboration.

[0023] Figure 2 It is a flowchart for generating a curvature feature matrix in a surface deviation detection method for virtual-real superposition and multi-user collaboration. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the following will make a detailed description of the specific embodiments of the present invention with reference to the drawings in the specification.

[0025] In the following description, many specific details are set forth to fully understand the present invention. However, the present invention can also be implemented in other ways different from the description herein. Those skilled in the art can make similar generalizations without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0026] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that can be included in at least one implementation of the present invention. The "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selectively exclusive embodiment from other embodiments.

[0027] As mentioned in the above background art, the existing surface detection technologies generally have the following problems: First, there is a lack of a fusion analysis mechanism for point cloud data and reverse engineering models, and accurate spatial alignment and deviation quantification cannot be achieved; second, there is a lack of a multi-user collaboration mechanism and annotation index structure, making it difficult to meet the requirements of collaborative review and dynamic interaction; third, the visualization means are relatively primitive, and the VR / AR environment has not been fully utilized to realize the real-time interactive presentation of the deviation field. Therefore, a surface deviation detection solution for virtual-real superposition and multi-user collaboration is needed.

[0028] Figure 1Flowchart of a surface deviation detection method for virtual-real superposition and multi-user collaboration according to an embodiment of the present invention. As Figure 1 shown, in the surface deviation detection method for virtual-real superposition and multi-user collaboration, it includes S1: Collect the point cloud data of the target physical object, calculate the curvature feature matrix from the point cloud data; at the same time, associate and store the point cloud data with the reverse engineering model data.

[0029] First, collect the point cloud data of the target physical object through a handheld 3D scanner. The scanner can be a structured light scanner, a laser scanner, or a portable photogrammetry device, which has the functions of multi-view synchronous acquisition, sub-millimeter ranging accuracy, and real-time data preprocessing; in order to make the collected point cloud data maintain a consistent structure with the CAD model or the reverse engineering model based on the NURBS (Non-Uniform Rational B-Spline) surface during the analysis process, automatically call the model data related to the target physical object in the CAD / NURBS model database during collection. The database supports the efficient indexing and loading of models, and is bound to the physical object through a unique identifier ID, such as through RFID tags, QR code markings, or digital twin mapping methods, to ensure the one-to-one correspondence between the point cloud and the model.

[0030] During the establishment of the association, use an optical Tracker device to perform spatial calibration on the collected point cloud data and model data, establish a unified coordinate framework, so that the point cloud data and the model data are associated and stored in the same spatial reference system, and then support subsequent virtual-real superposition comparison analysis.

[0031] After completing the pairing storage of the point cloud data and the model data, as Figure 2 shown, perform a curvature feature extraction algorithm on the point cloud data, calculate the principal curvature value and curvature direction of each point, and obtain the curvature feature matrix, where each data point contains Gaussian curvature and mean curvature. The specific generation process includes: a. Perform local neighborhood screening on the point cloud data, and use a certain neighborhood radius (such as k-nearest neighbor or spherical neighborhood) to select several neighbor points around each data point.

[0032] It should be noted that the number and distribution density of neighborhood points have a significant impact on the curvature calculation result. If the number of neighborhood points is too small, the curvature calculation will be unstable; if the number of neighborhood points is too large, it will mask small curvature changes and reduce the curvature resolution. It is necessary to ensure that each point cloud data has sufficient neighborhood information for subsequent curvature calculation.

[0033] b. Perform least squares plane fitting on the selected local neighborhood to obtain the fitting plane parameters of each neighborhood point, determine the principal direction of each data point in the local area, and provide a plane reference for curvature calculation.

[0034] The specific operation is as follows: construct the least mean square error plane of all points within the neighborhood of the target point, and solve its normal vector and the position of the center point.

[0035] This fitted plane can not only reflect the general trend of the local surface, but also provide a reference plane for curvature calculation, which helps to clarify the curvature direction and the discrimination of the Gaussian curvature sign. Compared with the traditional global fitting method, the local plane fitting adopted in the present invention can better restore the surface morphological features while maintaining local details, and enhance the geometric interpretability of the fitting result.

[0036] c. Based on the locally fitted plane, calculate the curvature tensor of each data point, and extract the principal curvature value and the principal curvature direction of each point through this tensor. Among them, the principal curvature value represents the degree of bending of the surface at this point, and the principal curvature direction determines the direction of the curvature, which can define the shape characteristics of the local surface.

[0037] In the specific calculation process, first construct a local covariance matrix, perform eigenvalue decomposition on it, and obtain the principal curvature value and the corresponding direction.

[0038] d. Use the principal curvature values of the curvature tensor to calculate the Gaussian curvature and the mean curvature of each data point respectively.

[0039] Among them, the Gaussian curvature is obtained by the product of the principal curvatures, which represents the overall bending degree of the surface at this point and is an important basis for judging whether the point is a "hill point, valley point, saddle point or plane point"; the mean curvature is the arithmetic mean of the principal curvatures, which reflects the local smoothness of the surface and is an important index for describing the surface tension and the fitting error. These two curvature values respectively describe the local morphological characteristics of the surface and the average degree of curvature. By obtaining the two curvature parameters simultaneously, the shape characteristics of the target surface can be comprehensively characterized.

[0040] e. Construct a curvature feature matrix according to the Gaussian curvature and the mean curvature of each data point in accordance with the spatial position of the points. Each row corresponds to a point cloud data point, and the column vectors are in turn: spatial coordinates, principal curvatures, and mean curvature, and store them in the database.

[0041] Optionally, according to the normal consistency and neighborhood density of each point in the curvature feature matrix, eliminate the low-confidence point cloud data and update the curvature feature matrix to make the surface features more stable.

[0042] Package the optimized point cloud data, CAD / NURBS reverse engineering model data, curvature feature matrix, and spatial index structure, etc. as a meta-data set and store it in a structured database to achieve efficient retrieval and call.

[0043] It should be noted that the curvature feature matrix is read-only data, and a copy is created during subsequent use to avoid concurrent modification conflicts.

[0044] It can be seen that the present invention not only improves the accuracy of point cloud data in morphological restoration, but also realizes the integrated management of spatial structure between point cloud data and inverse model, providing solid data support for virtual-reality alignment, multi-user collaborative operation and visualization analysis of complex surface deviations.

[0045] S2: Complete the six-degree-of-freedom alignment operation on the inverse model data and the physical point cloud data, perform the surface deviation detection operation based on the aligned spatial coordinates, encode the detection results into a deviation field, and perform a preliminary visualization in the VR / AR environment in the form of color mapping and contour lines.

[0046] S2.1: Extract feature points from CAD / NURBS inverse model data and physical point cloud data.

[0047] For CAD / NURBS inverse model data, key points with significant geometric features are selected; for physical point cloud data, curvature features are used as auxiliary conditions to extract a set of feature points that match the CAD / NURBS inverse model.

[0048] It should be noted that since the geometric information of the inverse model is highly regular, the feature points are mainly selected from the model's geometric boundary points, curvature mutation points, intersection points or hole edge points. These points have obvious geometric identifiability in the geometric structure and can maintain uniqueness in the global model, thereby enhancing the reliability of subsequent matching.

[0049] The physical point cloud data needs to use the curvature feature matrix K. By setting the threshold range of the principal curvature value and the Gaussian curvature, representative data points in the morphological mutation area are screened out as physical feature points to ensure that they correspond to the key geometric structures in the CAD model.

[0050] S2.2: Match the feature points based on the CAD / NURBS inverse model data and the feature points in the physical point cloud data.

[0051] The specific operations include: using Euclidean space distance as the initial screening condition, performing preliminary neighbor search on the feature points in each point cloud data and the model feature points in space, and calculating the curvature feature difference between the two; in order to enhance the geometric robustness of the matching, the present invention encodes the neighborhood curvature, normal vector direction and density information of the feature points into local descriptors, and performs descriptor similarity calculation between all feature points of the two.

[0052] It is worth emphasizing that in order to avoid the amplification of registration errors, the matching process introduces the curvature feature matrix as a false matching elimination constraint.

[0053] In specific operations, by setting a distance function threshold between the first-order curvature and the second-order curvature, it is ensured that only when the curvature change trend between point pairs is within the allowable range, can they be considered to have legal geometric consistency, thereby ensuring the finally established set of point pair relationships.

[0054] S2.3: Through the feature point matching results, perform six-degree-of-freedom alignment on the CAD / NURBS reverse engineering model data and the physical point cloud data.

[0055] Exemplarily, first initialize the transformation matrix as the identity matrix, use each point pair in the point pair relationship set as input data, and adopt the classic Iterative Closest Point (ICP) algorithm for preliminary registration. In the ICP algorithm, repeatedly perform the nearest neighbor search and rigid body transformation estimation between the source point set and the target point set until the convergence condition is met.

[0056] To further optimize the registration accuracy, the present invention introduces a weighted least squares processing mechanism during the ICP iteration, uses the Gaussian curvature of each point in the curvature feature matrix as the weight factor, and regulates the contribution degree of each point in the error function. That is, points with high weights (usually located at geometric mutations) will obtain higher matching priorities, thereby strengthening the alignment effect of the feature regions.

[0057] S2.4: Based on the aligned spatial coordinates, calculate the surface deviation between the CAD / NURBS reverse engineering model data and the physical point cloud data; by analyzing the distance error of each point, generate a deviation field containing spatial positions and deviation values.

[0058] Specifically, during the operation, first select each point in the CAD model, use the k-d tree data structure to search for its nearest neighbor point in the physical point cloud, and calculate the Euclidean distance. This distance is the geometric deviation value of this point in the three-dimensional space.

[0059] S2.5: Initially visualize the generated deviation field data through color mapping and contour lines. In the VR / AR environment, the deviation value is displayed through color gradients, and contour lines are generated according to different error ranges to provide visual feedback to help users quickly identify regions with large deviations.

[0060] Specifically, the present invention first uses the deviation value as the main mapping dimension, performs color mapping through the HSV color space, and maps deviation values of different magnitudes to different hue regions. High-deviation value regions can be marked with red or warm colors, and low-deviation value regions can be marked with blue or cold colors, thereby constructing a color gradient distribution map.

[0061] Meanwhile, to enhance the user's perception of the spatial structure error boundary, the present invention further generates deviation isocontours. During the isocontour extraction process, the Marching Cubes algorithm is used to extract the isosurfaces with the same error value in the deviation field as a wireframe set, which is superimposed on the surface of the model in the VR / AR environment.

[0062] By rendering high-density isocontour layers at different deviation levels, users can clearly identify the regional boundaries and change trends of structural deviations.

[0063] Finally, in combination with a head-mounted display device or a three-dimensional projection system, users can visually and interactively view the structural deviation situation of the target object in an immersive environment.

[0064] S3: Multiple users perform collaborative review based on the deviation field, dynamically visualize and project the annotation spatial coordinates according to the surface curvature characteristics, and at the same time jointly encode the annotation spatial coordinates, user gesture trajectories, and text keywords to establish a two-dimensional R-tree annotation index, and spatially associate the annotation data with the deviation field results.

[0065] S3.1: The dynamic visualization projection includes the following operating steps: First, when the user makes an annotation, the spatial coordinate point where the annotation is located is judged through the surface curvature feature matrix. According to the position and shape of the surface, it is judged whether the annotation coordinate point is located in the visual blind area.

[0066] Exemplarily, obtain the surface curvature feature matrix , where and are the maximum curvature and minimum curvature values of the spatial point in the main direction. Through this surface curvature feature matrix calculate the local surface normal vector , and in combination with the user's perspective vector , analyze the angle between the normal direction and the line of sight. If , where is the set visual threshold; for example, the set visual threshold is 80°, then it can be preliminarily judged that the annotation landing point is already within the range inaccessible to the current user's vision, that is, it falls into the visual blind area.

[0067] This process automatically completes the blind area judgment with the help of surface geometric features without user intervention, significantly improving the automation of the annotation operation and the accuracy of spatial cognition.

[0068] Secondly, when the annotation is located in the surface blind area, project it along the surface normal direction to the visible area based on the annotation spatial coordinates, and generate a dynamic guiding line to direct the user's line of sight to the actual position of the annotation.

[0069] Exemplarily, let the original annotation point be P, and its normal vector be Find the visual point with the highest coincidence degree with this normal direction within the field of view and through the formula: where is the projection step length, which is dynamically adjusted according to the surface occlusion information to ensure that the projection landing point is within the current visible area of the user.

[0070] Meanwhile, a dynamic guiding line is generated synchronously connecting the original annotation point P and the projection point in the form of a time-parameterized curve which is defined as: , .

[0071] The guiding line is rendered in the VR / AR space in real time with a gradient color and arrow indicating the direction, ensuring that users can accurately perceive the real position and spatial semantics of blind area annotations in the annotation view, and improving the interaction understanding efficiency and positioning accuracy in multi-user collaboration.

[0072] S3.2: The spatial association of annotation data and deviation field results includes the following operation steps: First, combine the spatial coordinates, user gesture trajectories, and text keywords of each annotation for joint encoding, and establish a spatial index of the annotation data through a two-dimensional R-tree structure.

[0073] It should be noted that in order to systematically manage and quickly retrieve multi-user annotation information, the present invention proposes to jointly encode the annotation spatial coordinates, user gesture trajectories, and text keyword vectors. Among them, the annotation spatial coordinates are mapped to a two-dimensional plane through a Z-order space filling curve to generate two-dimensional encoded values; the user gesture trajectories are parsed into a time-space sequence of trajectory key points; the text keyword vectors extract annotation semantic vectors through TF-IDF or BERT embedding, and this embodiment is not limited to a unique one.

[0074] On this basis, construct a two-dimensional R-tree structure as a spatial indexing mechanism, and the indexing dimension is mainly selected , and other dimensions are used as index extension attributes to achieve efficient spatial organization.

[0075] Through this joint encoding strategy, not only the rapid indexing of spatial coordinates is realized, but also user interaction behaviors and semantic information are integrated, providing data support for subsequent matching and visualization analysis, and enhancing the multi-dimensional expression ability and associated retrieval efficiency of annotation data in the collaborative review scenario.

[0076] Secondly, through the established two-dimensional R-tree index, quickly query and match the spatial coordinates in the annotation data with the deviation field data in the detection results; according to the spatial matching results of the annotation spatial coordinates and the deviation field, confirm the specific detection result positions corresponding to each annotation.

[0077] Specifically, for any annotation point, quickly query its corresponding area in the deviation field through the R-tree, and search for the adjacent deviation point set within the range where the spatial error does not exceed the threshold. If the match is successful, a one-to-one correspondence is established between the annotation data and the matched deviation points. Through this mechanism, each annotation can be accurately associated with a specific error area in the deviation field, thus realizing the efficient combination of annotation semantics and detection data.

[0078] Again, when new user annotations are added or modified, dynamically adjust the spatial matching between the annotations and the deviation field according to the R-tree index, and update the spatial association data in real time.

[0079] It should be noted that in the multi-user collaborative review scenario, the annotation behavior has significant dynamics and real-time nature. Users may add, delete, or modify the existing annotation content at any time. Therefore, to ensure the response efficiency and the spatial consistency of the annotation data, when new annotations are added or existing annotations are modified, recalculate the joint vector according to their coding results and insert it into the R-tree index structure.

[0080] At the same time, if the annotation affects the boundary of the spatial region, automatically trigger the local R-tree node reconstruction operation to maintain the optimality of the index structure. In addition, the annotation modification will also trigger an incremental update of the deviation field matching process. Only rematch the affected area using the aforementioned matching function, which greatly reduces the global computational burden.

[0081] When the user initiates an annotation modification, calculate the hash value of the spatial block to which the annotation belongs based on the annotation coordinates, and try to obtain the distributed lock for this block; if the lock is already occupied, prompt the user that this area is being edited by other users and please try again later; After successfully locking, record the current operation version number, and verify the version consistency when submitting the modification. If a version conflict occurs (such as the same area has been modified by other users), automatically retain both versions and generate a conflict report for the administrator to arbitrate.

[0082] S4: Dynamically update the visualization content and deviation correction according to the spatial association, extract multi-user review data, and automatically generate a detection report.

[0083] S4.1: Based on the two-dimensional R-tree index, query the spatial association relationship between the current annotation spatial coordinates and the deviation values in the deviation field, and filter out the deviation field data associated with the annotation area according to the principal curvature value at the corresponding position in the curvature feature matrix.

[0084] First, dynamically extract all the user annotation information with established spatial association relationships, specifically including the spatial coordinates of the annotations, gesture trajectories, text keywords, and their corresponding deviation field area numbers.

[0085] Further, screening out the deviation field data associated with the annotation area includes: Based on the two-dimensional R-tree index, extract the set of user annotation coordinate points, and traverse the spatial coordinates of all data points in the deviation field. The principle is to detect the overlap of the minimum distance domain and judge the spatial inclusion relationship to ensure that the spatial positions of the annotation points and the deviation data are highly consistent or approximate; For each annotation coordinate point, locate the same coordinate point in the curvature feature matrix according to the spatial mapping relationship after six-degree-of-freedom alignment, and extract the principal curvature value; Calculate the difference between the principal curvature of the data points in the deviation field and the principal curvature of the corresponding annotation area, and remove the deviation field data points with the difference less than the preset curvature tolerance threshold; it should be noted that a curvature tolerance threshold needs to be set to eliminate those points with small difference values and insufficient deformation degree to constitute an effective deviation.

[0086] It should be noted that when traversing the deviation field data points, first check whether they are marked as the user correction locked state. If it is in the locked state, skip the curvature tolerance screening; for non-locked state points, calculate the difference between their principal curvature and the principal curvature of the annotation area. If the difference is less than the preset threshold, then remove them; if there is a user correction operation in a certain annotation area, automatically expand the curvature tolerance threshold of this area to 1.5 times the original value until the next global deviation field update.

[0087] S4.2: According to the correction parameters in the user annotation, perform gradient descent optimization on the associated deviation field data, adjust the spatial distribution of the deviation field, and update the deviation value in the visualization content based on the color mapping rule.

[0088] In the present invention, the correction parameters mainly include the expected value, calibration value or correction instruction actively input by the user during the annotation process, such as "the offset should be 0, adjust 5 mm to the left", etc.; by extracting the correction target from the user text and standardizing it as the constraint condition of the deviation optimization objective function.

[0089] For the above correction target, adopt a non-linear optimization strategy based on the gradient descent method to adjust the spatial distribution of the deviation field.

[0090] It should be noted that during the deviation field optimization process, if it is detected that there is an unresolved version conflict in a certain area, suspend the optimization calculation of this area until the conflict is resolved.

[0091] In the present invention, the objective function is constructed as the sum of the squares of the errors between the current deviation value and the user correction target, and the adjustable parameter is the deviation vector value of each spatial point in the deviation field.

[0092] In each iteration, the gradient directions and step size factors of all points are calculated separately, and the bias value is updated according to the set learning rate parameter. This method can achieve spatial optimization and correction under the guidance of the user's intention while ensuring global convergence stability, and has strong numerical stability and anti-interference ability.

[0093] After the bias optimization is completed, the visualization image content will be dynamically updated according to the color mapping. Among them, the color mapping rule is to map the bias value to different colors through a certain piecewise or continuous function to realize the visualization of the bias.

[0094] This mechanism not only intuitively presents the spatial distribution changes before and after the bias correction, but also provides real-time visible visual feedback for multi-user interaction, which helps to quickly locate the problem area and confirm the correction effect.

[0095] After the bias field optimization is completed, the point cloud data and CAD model data of the corrected area are automatically extracted, and local six-degree-of-freedom alignment is performed (only for the corrected area rather than globally), and the bias field of this area is recalculated; the updated bias field data will overwrite the original result and trigger the real-time refresh of the visualization content.

[0096] S4.3: Generate a detection report including the bias correction process, associated annotation positions, and final detection results.

[0097] After all bias correction and visualization update steps are completed, a complete detection report is automatically generated. This report is not only used for result archiving, but also can be used as a key basis for quality traceability, process optimization, and multi-user review.

[0098] In particular, the report will also record the interaction logs of each user correction, including detailed information such as the correction time, operator, input parameters, and number of optimization iterations.

[0099] This embodiment also provides a computer device, which is applicable to the case of the surface deviation detection method for virtual-real superposition and multi-user collaboration, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the surface deviation detection method for virtual-real superposition and multi-user collaboration as proposed in the above embodiment.

[0100] The computer device can be a terminal, which includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be achieved through WIFI, carrier networks, NFC (Near Field Communication), or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0101] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the curved surface deviation detection method for realizing virtual-real superposition and multi-user collaboration as proposed in the above embodiment.

[0102] In summary, through the construction of a curved surface deviation detection method for virtual-real superposition and multi-user collaboration, the present invention realizes the high-precision detection and efficient collaborative review of complex curved surface structures, significantly improving the intelligence and practicality levels of deviation detection. By introducing the curvature feature matrix, six-degree-of-freedom alignment mechanism, and deviation field coding, the geometric accuracy of the detection results and the integrity of spatial expression can be effectively improved, enhancing the ability to identify and analyze complex curved surface errors; with the help of the visualization mapping method in the VR / AR environment, the intuitiveness and interactivity of the detection results are improved, which helps users quickly understand and locate the deviation area; in addition, through the multi-user collaborative annotation and dynamic visualization projection mechanism, multi-perspective review and intelligent guidance are realized, greatly reducing the risks of misjudgment and missed detection. The spatial indexing and dynamic update mechanism between annotation data and deviation information further support the real-time optimization and correction of detection results, which is conducive to forming high-quality and structured detection records.

[0103] In summary, the present invention effectively solves the problems existing in curved surface deviation detection, such as inaccurate alignment, low review efficiency, and difficulty in covering blind areas, improving the accuracy, interpretability, and team collaboration efficiency of detection results.

[0104] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A method for detecting surface deviation with virtual-real superposition and multi-user collaboration, characterized in that: Including: Collecting the point cloud data of the target physical object and calculating the curvature feature matrix from the point cloud data; At the same time, associatively storing the point cloud data and the reverse engineering model data; Performing a six-degree-of-freedom alignment operation on the reverse engineering model data and the physical object point cloud data. Based on the aligned spatial coordinates, performing a surface deviation detection operation, encoding the detection result as a deviation field, and performing preliminary visual display in the VR / AR environment in the form of color mapping and contour lines; Multiple users perform collaborative review based on the deviation field, perform dynamic visual projection on the annotation spatial coordinates according to the surface curvature characteristics, and at the same time jointly encode the annotation spatial coordinates, user gesture trajectories and text keywords, establish a two-dimensional R-tree annotation index, and spatially associate the annotation data with the deviation field results; Dynamically update the visual content and deviation correction according to the spatial association, extract the multi-user review data, and automatically generate a detection report.

2. The method for detecting surface deviation with virtual-real superposition and multi-user collaboration according to claim 1, characterized in that: The generation of the curvature feature matrix includes: Performing local neighborhood screening on the point cloud data, using a certain neighborhood radius to select several neighbor points for each data point; performing least squares plane fitting on the selected local neighborhood to obtain the local fitting plane parameters of each neighborhood point; based on the local fitting plane, calculating the curvature tensor of each data point, and extracting the principal curvature value of each point; using the principal curvature value of the curvature tensor, calculating the Gaussian curvature and mean curvature of each data point respectively, and constructing a curvature feature matrix according to the spatial position of the points.

3. The surface deviation detection method for virtual-real superposition and multi-user collaboration according to claim 1, wherein: The six-degree-of-freedom alignment operation includes: Extracting feature points from the reverse engineering model data and the physical object point cloud data, and performing matching between the feature points; Performing six-degree-of-freedom alignment on the reverse engineering model data and the physical object point cloud data through the feature point matching results.

4. The surface deviation detection method for virtual-real superposition and multi-user collaboration according to claim 3, wherein: The performing of the surface deviation detection operation includes: Based on the aligned spatial coordinates, calculating the surface deviation between the reverse engineering model data and the physical object point cloud data; generating a deviation field containing spatial positions and deviation values by analyzing the distance errors of each feature point.

5. The method for detecting surface deviation by virtual-real superposition and multi-user collaboration according to claim 1, characterized in that: The dynamic visual projection includes: When the user makes an annotation, judging the spatial coordinate point where the annotation is located through the surface curvature feature matrix, and judging whether the annotation coordinate point is in the visual blind area according to the position and shape of the surface; When the annotation is in the surface blind area, projecting it along the surface normal direction to the visible area based on the annotation spatial coordinates, and generating a dynamic guiding line to guide the user's line of sight to the actual position of the annotation.

6. The surface deviation detection method for virtual-real superposition and multi-user collaboration according to claim 5, wherein: The spatially associating the annotation data with the deviation field results includes: Combining the spatial coordinates, user gesture trajectories and text keywords of each annotation for joint encoding, and establishing a spatial index of the annotation data through a two-dimensional R-tree structure; Through the established two-dimensional R-tree index, quickly querying and matching the spatial coordinates in the annotation data with the deviation field data in the detection results; According to the spatial matching result of the annotation spatial coordinates and the deviation field, confirming the specific detection result position corresponding to each annotation; When a new user annotation is added or modified, dynamically adjusting the spatial matching of the annotation and the deviation field according to the R-tree index, and real-time updating the spatial association data.

7. The surface deviation detection method for virtual-real superposition and multi-user collaboration according to claim 1, characterized in that: The dynamically updating the visual content and deviation correction includes: Based on the two-dimensional R-tree index, query the spatial association relationship between the current annotation space coordinates and the deviation values in the deviation field, and filter out the deviation field data associated with the annotation area according to the principal curvature values at the corresponding positions in the curvature feature matrix; Perform gradient descent optimization on the associated deviation field data according to the user annotation, adjust the spatial distribution of the deviation field, and update the deviation values in the visualization content based on color mapping.

8. The surface deviation detection method for virtual-real superposition and multi-user collaboration according to claim 7, characterized in that: The filtering out of the deviation field data associated with the annotation area includes: Based on the two-dimensional R-tree index, extract the set of user annotation coordinate points and traverse the spatial coordinates of all data points in the deviation field; For each annotation coordinate point, locate the same coordinate point in the curvature feature matrix according to the spatial mapping relationship after six-degree-of-freedom alignment, and extract the principal curvature value; Calculate the difference between the principal curvature of the data points in the deviation field and the principal curvature of the corresponding annotation area, and remove the deviation field data points with a difference less than the preset curvature tolerance threshold.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the virtual-real superposition and multi-user collaboration surface deviation detection method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the virtual-real superposition and multi-user collaboration surface deviation detection method according to any one of claims 1 to 8.

Images