A virtual-real superimposition and multi-user cooperation curved surface deviation detection method

Abstract

The present invention discloses a surface deviation detection method with virtual-real superposition and multi-user collaboration, which relates to the field of deviation detection technology. It includes calculating the curvature feature matrix of point cloud data; aligning the inverse model data with the physical point cloud data, performing surface deviation detection operations, encoding them into a deviation field, and performing preliminary visual display in a VR / AR environment; multiple users conduct collaborative review based on the deviation field, dynamically visually project the annotation space coordinates according to the surface curvature characteristics, and jointly encode the annotation space coordinates, user gesture trajectories and text keywords to establish a two-dimensional R-tree annotation index, spatially associate the annotation data with the deviation field results; and dynamically update the visualization content and deviation correction based on the spatial association. The present invention effectively solves the problems of inaccurate alignment, low review efficiency, and difficulty in covering blind areas in surface deviation detection, and improves the accuracy, interpretability and team collaboration efficiency of the detection results.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deviation detection, and in particular to a virtual-real superimposition and multi-user cooperation curved surface deviation detection method. BACKGROUND

[0002] With the continuous development of high-precision measurement technology and virtual reality / augmented reality (VR / AR) technology, the industrial manufacturing field has put forward higher demand for the cooperation, visualization and intelligentization of complex curved surface detection and review. Traditional three-dimensional measurement methods rely on single-point measurement or regular grid structure, which is difficult to cover the details of complex structures such as free curved surfaces. In recent years, point cloud data processing, three-dimensional reconstruction, deviation analysis and other technologies have gradually matured, especially with the cooperation of high-precision equipment such as laser scanners and structured light systems, which makes the curved surface detection evolve from single measurement to multi-source data fusion and interactive visualization. On this basis, the introduction of multi-user cooperation mechanism and visual interaction means further improves the flexibility and review efficiency of detection, and becomes an important development direction of intelligent detection of complex structures.

[0003] CN107421462A provides a three-dimensional contour measurement system of an object based on line laser scanning, which mainly realizes contour data acquisition through left and right image sensors and line lasers, and has the characteristics of simple system structure, small imaging error and stable data. This method can efficiently obtain the geometric information of the object under static conditions, but its detection process mainly focuses on the single-user static measurement stage, lacks subsequent data visualization review process, especially does not involve the fusion processing of point cloud data and CAD model, and does not support deviation field visualization and collaborative annotation in VR / AR environment, which is difficult to meet the multi-user review and dynamic feedback needs in complex structure scenarios.

[0004] CN109813219B proposes a method and system for collecting and processing information of existing structure detection, identification and reinforcement, which emphasizes data sharing and circulation in the process of structure detection, identification and subsequent construction, effectively reducing the frequency of repeated modeling and field operation. The outstanding feature of this scheme is to realize the unified management and display of detection information in multiple stages, but its core still focuses on the macro collection of structure information and the visualization of BIM, and does not involve the fine curved surface deviation calculation and the spatial fine alignment processing of point cloud and reverse model. At the same time, it also does not introduce multi-user interaction and spatial annotation mechanism, which has certain limitations for micro-analysis of curvature abnormalities and local deviations.

[0005] CN104137030A discloses a method for measuring a three-dimensional sample by a measuring device comprising a laser scanning microscope, and the measuring device, which realizes linkage mapping of the measurement space and the virtual space by interactively controlling the measurement action in the virtual space and visualizing the measurement result. Although the method is innovative in the field of microscopic scale measurement, its main target is the virtual control of operation and observation feedback, and it has not yet built a geometric difference expression mechanism between the point cloud and the model, nor has it established a multi-user collaborative annotation model at the spatial coordinate level, and it cannot meet the needs of multi-person joint decision-making, joint review and automatic report generation in industrial complex structures.

[0006] In summary, the existing curved surface detection technology generally has the following problems: first, there is a lack of fusion analysis mechanism of point cloud data and reverse modeling, which cannot realize accurate spatial alignment and deviation quantization; second, there is a lack of multi-user collaboration mechanism and annotation index structure, which makes it difficult to meet the requirements of collaborative review and dynamic interaction; third, the visualization means is relatively simple, and the deviation field has not been fully utilized in the VR / AR environment to realize real-time interactive presentation. SUMMARY

[0007] In view of the problems existing in the existing curved surface detection technology, the present application is proposed.

[0008] Therefore, the problem to be solved by the present application is how to realize the integrated process of high-precision curved surface deviation detection, spatial annotation review and automatic report generation, and improve the intelligentization and collaboration level of complex structure detection.

[0009] To solve the above technical problems, the present application provides the following technical solutions:

[0010] In a first aspect, the present application provides a curved surface deviation detection method combining virtual and real superimposition and multi-user collaboration, which comprises: collecting point cloud data of a target physical object, calculating a curvature feature matrix from the point cloud data; simultaneously storing the point cloud data and the reverse modeling data in association; performing a six-degree-of-freedom alignment operation on the reverse modeling data and the physical point cloud data, based on the spatial coordinates after alignment, performing a curved surface deviation detection operation, encoding the detection result as a deviation field, and preliminarily visualizing the deviation field in a VR / AR environment in the form of color mapping and contour lines; multi-users perform collaborative review based on the deviation field, dynamically visualize the annotation spatial coordinates according to the curved surface curvature features, simultaneously encode the annotation spatial coordinates, user gesture trajectories and text keywords, establish a two-dimensional R-tree annotation index, spatially associate the annotation data and the deviation field result; dynamically update the visualization content and deviation correction according to the spatial association, extract the multi-user review data, and automatically generate a detection report.

[0011] As a preferred scheme of the surface deviation detection method for virtual-real superposition and multi-user cooperation provided in the application, the generation of the curvature feature matrix comprises: local neighborhood screening of the point cloud data, selection of a plurality of neighbor points for each data point by using a certain neighborhood radius; least square plane fitting of the selected local neighborhood to obtain local fitting plane parameters of each neighborhood point; calculation of the curvature tensor of each data point based on the local fitting plane, and extraction of the principal curvature value of each point; calculation of the Gaussian curvature and the average curvature of each data point by using the principal curvature value of the curvature tensor, and construction of the curvature feature matrix according to the spatial position of the points.

[0012] As a preferred scheme of the surface deviation detection method for virtual-real superposition and multi-user cooperation provided in the application, the six-degree-of-freedom alignment operation comprises: extraction of feature points from the reverse modeling data and the physical point cloud data, and matching between the feature points; six-degree-of-freedom alignment of the reverse modeling data and the physical point cloud data by using the matching result of the feature points.

[0013] As a preferred scheme of the surface deviation detection method for virtual-real superposition and multi-user cooperation provided in the application, the execution of the surface deviation detection operation comprises: calculation of the surface deviation between the reverse modeling data and the physical point cloud data based on the aligned spatial coordinates; and generation of a deviation field containing spatial positions and deviation values by analyzing the distance error of each feature point.

[0014] As a preferred scheme of the surface deviation detection method for virtual-real superposition and multi-user cooperation provided in the application, the dynamic visualization projection comprises: when the user makes a comment, judging the spatial coordinate point of the comment by using the surface curvature feature matrix, judging whether the comment coordinate point is located in a visual blind area according to the position and shape of the surface, projecting the comment spatial coordinate to a visible area along the normal direction of the surface when the comment is located in the surface blind area, and generating a dynamic guide line to guide the user's line of sight to the actual position of the comment.

[0015] As a preferred scheme of the surface deviation detection method for virtual-real superposition and multi-user cooperation provided in the application, the spatial correlation of the comment data and the deviation field result comprises: joint coding of the spatial coordinates, user gesture trajectories and text keywords of each comment, and establishment of a spatial index of the comment data by using a two-dimensional R-tree structure; fast query and matching of the spatial coordinates in the comment data and the deviation field data in the detection result by using the established two-dimensional R-tree index; confirmation of the specific detection result position corresponding to each comment according to the spatial matching result of the comment spatial coordinates and the deviation field; and dynamic adjustment of the spatial matching of the comment and the deviation field and real-time update of the spatial correlation data according to the R-tree index when a new user comment is added or modified.

[0016] As a preferred scheme of the surface deviation detection method for virtual-real superposition and multi-user cooperation provided in the application, the dynamic updating of the visual content and the deviation correction comprises: querying the spatial correlation between the current batch comment space coordinates and the deviation values in the deviation field based on the two-dimensional R-tree index, and filtering out the deviation field data associated with the comment area according to the principal curvature values of the corresponding positions in the curvature feature matrix; and performing gradient descent optimization on the associated deviation field data according to the user comment, adjusting the spatial distribution of the deviation field, and updating the deviation values in the visual content based on color mapping.

[0017] As a preferred scheme of the surface deviation detection method for virtual-real superposition and multi-user cooperation provided in the application, the filtering out of the deviation field data associated with the comment area comprises: extracting the user comment coordinate point set based on the two-dimensional R-tree index, and traversing the spatial coordinates of all data points in the deviation field; for each comment coordinate point, locating the same coordinate point in the curvature feature matrix according to the spatial mapping relationship after six-degree-of-freedom alignment, and extracting the principal curvature value; and calculating the difference between the principal curvature of the data point in the deviation field and the principal curvature of the corresponding comment area, and removing the deviation field data points with a difference less than a preset curvature tolerance threshold.

[0018] In a second aspect, the application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the computer program instructions are executed by the processor to implement the steps of the surface deviation detection method for virtual-real superposition and multi-user cooperation according to the first aspect of the application.

[0019] In a third aspect, the application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program instructions are executed by the processor to implement the steps of the surface deviation detection method for virtual-real superposition and multi-user cooperation according to the first aspect of the application.

[0020] The application has the following beneficial effects: the application constructs a surface deviation detection method for virtual-real superposition and multi-user cooperation, realizes high-precision detection and efficient collaborative review of complex curved surface structures, and significantly improves the intelligentization and practicality level of deviation detection. By introducing the curvature feature matrix, the six-degree-of-freedom alignment mechanism and the deviation field coding, the geometric accuracy of the detection result and the completeness of the spatial expression can be effectively improved, and the recognition and analysis capability for complex curved surface errors can be enhanced; with the aid of the visual mapping mode of the VR / AR environment, the intuitiveness and interactivity of the detection result are improved, which helps users quickly understand and locate the deviation area; in addition, through the multi-user collaborative comment and dynamic visual projection mechanism, multi-angle review and intelligent guidance are realized, and the risk of misjudgment and missed detection is greatly reduced. The spatial index and dynamic updating mechanism between the comment data and the deviation information further support the real-time optimization and correction of the detection result, which is conducive to forming high-quality and structured detection records.

[0021] In summary, the present application effectively solves the problems of inaccurate alignment, low review efficiency, and difficult coverage of blind spots in curved surface deviation detection, and improves the accuracy, interpretability, and team collaboration efficiency of the detection results. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0023] Figure 1 Flowchart of the curved surface deviation detection method of virtual-real superposition and multi-user collaboration.

[0024] Figure 2 Flowchart of the generation of the curvature feature matrix in the curved surface deviation detection method of virtual-real superposition and multi-user collaboration. DETAILED DESCRIPTION

[0025] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0026] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from the description, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0027] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0028] As described in the above background, the existing curved surface detection technology generally has the following problems: first, there is a lack of fusion analysis mechanism of point cloud data and reverse modeling, which cannot realize accurate spatial alignment and deviation quantization; second, there is a lack of multi-user collaboration mechanism and annotation index structure, which makes it difficult to meet the requirements of collaborative review and dynamic interaction; third, the visualization means is relatively simple, and the real-time interactive presentation of the deviation field has not been fully utilized in the VR / AR environment. Therefore, a curved surface deviation detection scheme of virtual-real superposition and multi-user collaboration is needed.

[0029] Figure 1A flow chart of the virtual-real superimposition and multi-user cooperation curved surface deviation detection method according to an embodiment of the present application. As shown in Figure 1 The virtual-real superimposition and multi-user cooperation curved surface deviation detection method comprises the following steps.

[0030] S1: Collecting point cloud data of a target real object, and calculating a curvature feature matrix from the point cloud data; and simultaneously storing the point cloud data and reverse model data in association.

[0031] First, point cloud data of a target real object is collected by a handheld three-dimensional scanner, which can be a structured light scanner, a laser scanner or a portable photogrammetry device, and has the functions of multi-view synchronous acquisition, sub-millimeter level ranging accuracy and real-time data preprocessing. In order to maintain structural consistency between the collected point cloud data and the CAD model or the reverse model based on the NURBS (Non-Uniform Rational B-Spline) curved surface during the analysis process, the model data related to the target real object in the CAD / NURBS model database is automatically called during collection. The database supports efficient indexing and loading of models, and is bound to the real object through a unique identification ID, such as an RFID tag, a two-dimensional code mark or a digital twin mapping method, to ensure one-to-one correspondence between the point cloud and the model.

[0032] During the association process, an optical Tracker device is used to perform spatial calibration on the collected point cloud data and model data, and a unified coordinate frame is established to enable the point cloud data and the model data to be stored in association under the same spatial reference system, thereby supporting subsequent virtual-real superimposition comparison and analysis.

[0033] After the point cloud data collection and model data pairing storage are completed, as shown in Figure 2 The curvature feature extraction algorithm is performed on the point cloud data to calculate the principal curvature value and the curvature direction of each point, and a curvature feature matrix is obtained, wherein each data point contains Gaussian curvature and average curvature. The specific generation process includes:

[0034] a. Local neighborhood screening is performed on the point cloud data, and a certain neighborhood radius (such as k-nearest neighbors or spherical neighborhood) is used to select a number of neighbor points around each data point.

[0035] It should be noted that the number and distribution density of neighborhood points have a significant impact on the curvature calculation result. If the number of neighborhood points is too small, the curvature calculation will be unstable; if the number of neighborhood points is too large, the small curvature changes will be hidden, reducing the curvature resolution. It is necessary to ensure that each point cloud data has sufficient neighborhood information for subsequent curvature calculation.

[0036] b. Least squares plane fitting is performed on the selected local neighborhood to obtain the fitting plane parameters of each neighborhood point, and the principal direction of the local area where each data point is located is determined to provide a plane reference for curvature calculation.

[0037] The specific operation is: a minimum mean square error plane of all points in the neighborhood of the target point is constructed, and a normal vector thereof and a center point position are solved.

[0038] The fitting plane not only reflects the general trend of the local surface, but also provides a reference surface for curvature calculation, and helps to determine the curvature direction and the Gaussian curvature sign. Compared with the traditional global fitting method, the local plane fitting adopted by the application can better restore the surface morphological characteristics while maintaining the local details, and enhance the geometric interpretability of the fitting results.

[0039] c. Based on the local fitting plane, the curvature tensor of each data point is calculated, and the principal curvature value and the principal curvature direction of each point are extracted through the tensor. Among them, the principal curvature value represents the bending degree of the surface at the point, and the principal curvature direction determines the curvature direction, which can define the shape characteristics of the local surface.

[0040] In the specific calculation process, first, the local covariance matrix is constructed, and eigenvalue decomposition is performed thereon to obtain the principal curvature value and the corresponding direction.

[0041] d. The Gaussian curvature and the average curvature of each data point are calculated respectively by using the principal curvature value of the curvature tensor.

[0042] Among them, the Gaussian curvature is obtained by the product of the principal curvature, which represents the overall bending degree of the surface at the point, and is an important basis for judging whether the point is a "mountain point, valley point, saddle point or plane point"; the average curvature is the arithmetic mean of the principal curvature, which reflects the local smoothness of the surface, and is an important index for describing the surface tension and fitting error. The two curvature values respectively describe the local morphological characteristics of the surface and the average degree of curvature, and by simultaneously obtaining the two curvature parameters, the shape characteristics of the target surface can be fully described.

[0043] e. The Gaussian curvature and the average curvature of each data point are constructed into a curvature feature matrix according to the spatial position of the point, each row corresponds to a point cloud data point, and the column vector is sequentially: spatial coordinates, principal curvature and average curvature, and is stored in the database.

[0044] Optionally, according to the normal consistency and neighborhood density of each point in the curvature feature matrix, low-confidence point cloud data is removed, and the curvature feature matrix is updated, so that the surface characteristics are more stable.

[0045] The optimized point cloud data, CAD / NURBS reverse modeling data, curvature feature matrix and spatial index structure are encapsulated as metadata set and stored in structured database, so as to realize efficient retrieval and calling.

[0046] It should be noted that the curvature feature matrix is read-only data, and a copy is created for subsequent use to avoid concurrent modification conflicts.

[0047] It can be seen that the present application not only improves the accuracy of point cloud data in shape restoration, but also realizes the integrated management of the spatial structure between the point cloud data and the reverse modeling, providing solid data support for virtual-real superimposed alignment, multi-user collaborative operation and complex curved surface deviation visualization analysis.

[0048] S2: Perform six-degree-of-freedom alignment operation on the reverse modeling data and the physical point cloud data, based on the aligned spatial coordinates, perform curved surface deviation detection operation, encode the detection results as deviation field, and preliminarily visualize the results in VR / AR environment in color mapping and contour line way.

[0049] S2.1: Extract feature points from CAD / NURBS reverse modeling data and physical point cloud data.

[0050] For CAD / NURBS reverse modeling data, key points with significant geometric features are selected; for physical point cloud data, curvature features are used as auxiliary conditions to extract a set of feature points matching the CAD / NURBS reverse modeling.

[0051] It should be noted that since the geometric information of the reverse modeling has high regularity, the feature points are mainly selected from the geometric boundary points, curvature mutation points, intersection line points or hole edge points of the model, which have obvious geometric distinguishability and can maintain uniqueness in the global model, thereby enhancing the reliability of subsequent matching.

[0052] The physical point cloud data needs to use the curvature feature matrix K to set the threshold range of principal curvature value and Gaussian curvature, and filter out the representative data points in the shape mutation area as physical feature points, so as to ensure that they have corresponding relationship with the key geometric structures in the CAD model.

[0053] S2.2: Based on the feature points in the CAD / NURBS reverse modeling data and the physical point cloud data, match the feature points.

[0054] The specific operation includes: using Euclidean space distance as the initial screening condition, performing preliminary nearest neighbor search of the feature points in each point cloud data with the model feature points in space, and calculating the curvature feature difference between them; in order to enhance the geometric robustness of matching, the neighborhood curvature, normal vector direction and density information of the feature points are encoded into local descriptors, and descriptor similarity calculation is performed between all feature points.

[0055] It is worth emphasizing that in order to avoid the amplification of registration error, the curvature feature matrix is introduced as a mismatch elimination constraint in the matching process.

[0056] In a specific operation, by setting a distance function threshold between the first-order curvature and the second-order curvature, only when the curvature change trend between the point pairs is within the allowed range, it is considered to have legal geometric consistency, thereby ensuring the final established point pair relationship set.

[0057] S2.3: Through the feature point matching result, the CAD / NURBS reverse modeling data and the physical point cloud data are aligned in six degrees of freedom.

[0058] For example, first, the transformation matrix is initialized as an identity matrix, and each point pair in the point pair relationship set is taken as input data, and a classic iterative closest point (ICP) algorithm is used for preliminary registration. In the ICP algorithm, the nearest neighbor search and rigid body transformation estimation between the source point set and the target point set are repeatedly performed until the convergence condition is met.

[0059] To further optimize the registration accuracy, the application introduces a weighted least squares processing mechanism in the ICP iteration process, taking the Gaussian curvature of each point in the curvature feature matrix as a weight factor to control the contribution of each point in the error function. That is, points with high weights (usually located at geometric mutations) will have higher matching priority, thereby strengthening the alignment effect of the feature area.

[0060] S2.4: Based on the aligned spatial coordinates, the surface deviation between the CAD / NURBS reverse modeling data and the physical point cloud data is calculated; by analyzing the distance error of each point, a deviation field containing spatial position and deviation value is generated.

[0061] Specifically, in the operation, first, each point in the CAD model is selected, and the k-d tree data structure is used to search for its nearest neighbor point in the physical point cloud, and the Euclidean distance is calculated. The distance is the geometric deviation value of the point in the three-dimensional space.

[0062] S2.5: The generated deviation field data is preliminarily visualized by color mapping and contour lines. In the VR / AR environment, the deviation value is displayed by color gradient, and contour lines are generated according to different error ranges to provide visual feedback and help users quickly identify areas with large deviations.

[0063] Specifically, the application first takes the deviation value as the main mapping dimension and performs color mapping through the HSV color space to map different amplitude deviation values to different hue regions. High deviation value regions can be marked with red or warm colors, and low deviation value regions can be marked with blue or cold colors, thereby constructing a color gradient distribution map.

[0064] Meanwhile, to enhance the user's perception of the error boundary of the spatial structure, the application further generates deviation contour lines. In the contour extraction process, the Marching Cubes algorithm is used to extract the contour surface of the same error value in the deviation field as a wireframe set, which is superimposed on the model surface in the VR / AR environment.

[0065] By rendering high-density contour layers at different deviation levels, the user can clearly identify the regional boundary and variation trend of the structural deviation.

[0066] Finally, combined with a head-mounted display device or a three-dimensional projection system, the user can intuitively and interactively view the structural deviation of the target object in an immersive environment.

[0067] S3: Multi-user collaborative review based on the deviation field, dynamically visualizing the projection of the annotation spatial coordinates according to the surface curvature characteristics, and jointly encoding the annotation spatial coordinates, user gesture trajectories, and text keywords to establish a two-dimensional R-tree annotation index, and spatially associating the annotation data with the deviation field results.

[0068] S3.1: The dynamic visualization projection includes the following operation steps:

[0069] First, when the user makes an annotation, the spatial coordinate point of the annotation is determined by the surface curvature characteristic matrix, and according to the position and shape of the surface, it is determined whether the annotation coordinate point is located in the visual blind area.

[0070] For example, the surface curvature characteristic matrix is obtained wherein, and are the maximum and minimum curvature values of the spatial point in the principal direction. The local surface normal vector is calculated by the surface curvature characteristic matrix and the angle between the normal and the line of sight is analyzed in combination with the user's view vector If wherein, is a set visibility threshold; for example, if the set visibility threshold is 80°, it can be preliminarily determined that the annotation drop point is already in the current user's visual range, i.e., it falls into the visual blind area.

[0071] This process automatically completes the blind area judgment with the help of surface geometric characteristics, without user intervention, significantly improving the automation and spatial cognition accuracy of the annotation operation.

[0072] Secondly, when the annotation is located in the surface blind area, the annotation spatial coordinates are projected along the surface normal to the visible area, and a dynamic guide line is generated to guide the user's line of sight to the actual position of the annotation.

[0073] Exemplarily, let the original annotation point be P, and the normal vector of P be , find the visible point in the field of view that is most coincident with the normal direction , and pass the formula: , wherein is the projection step, which is dynamically adjusted according to the surface occlusion information to ensure that the projection landing point is in the current visible area of the user.

[0074] At the same time, a dynamic guide line is generated , connecting the original annotation point P and the projection point in a time parameterized curve manner , which is defined as: , .

[0075] The guide line is rendered in real time in the VR / AR space with a gradient color and an arrow indicating the direction, ensuring that the user can accurately perceive the real position and spatial semantics of the blind area annotation in the annotation view, improving the interaction understanding efficiency and positioning accuracy in multi-user collaboration.

[0076] S3.2: Spatially associating the annotation data with the bias field result includes the following operation steps:

[0077] First, the spatial coordinates of each annotation, the user gesture trajectory and the text keyword are combined for joint coding, and the spatial index of the annotation data is established through a two-dimensional R-tree structure.

[0078] It should be noted that in order to systematically manage and quickly retrieve multi-user annotation information, the present application proposes to jointly encode the annotation spatial coordinates, the user gesture trajectory and the text keyword vector, wherein the annotation spatial coordinates are mapped to a two-dimensional plane through a Z-order space filling curve to generate a two-dimensional encoding value; the user gesture trajectory is analyzed as a time-space sequence of key points; and the text keyword vector extracts a semantic vector through TF-IDF or BERT embedding, which is not uniquely limited in this embodiment.

[0079] On this basis, a two-dimensional R-tree structure is constructed as a spatial indexing mechanism, and the index dimension is mainly selected , and other dimensions are used as index extension attributes to realize efficient spatial organization.

[0080] Through the joint coding strategy, not only is the spatial coordinates indexed quickly, but also the user interaction behavior and semantic information are integrated, providing data support for subsequent matching and visualization analysis, and enhancing the multi-dimensional expression ability and correlation retrieval efficiency of the annotation data in the collaborative review scenario.

[0081] Secondly, through the established two-dimensional R-tree index, the spatial coordinates in the annotation data are quickly queried and matched with the deviation field data in the detection results; according to the spatial matching results of the annotation spatial coordinates and the deviation field, the specific detection result position corresponding to each annotation is confirmed.

[0082] Specifically, for any annotation point, the corresponding area of the annotation point in the deviation field is quickly queried through the R-tree, and a set of adjacent deviation points within a threshold range of spatial error is searched. If the matching is successful, a one-to-one correspondence is established between the annotation data and the matched deviation point. Through this mechanism, each annotation can be accurately associated with a specific error area in the deviation field, thereby realizing the efficient combination of annotation semantics and detection data.

[0083] Thirdly, when new user annotations are added or modified, the spatial matching of annotations and deviation fields is dynamically adjusted according to the R-tree index, and the spatial association data is updated in real time.

[0084] It should be noted that in the multi-user collaborative review scenario, the annotation behavior has significant dynamics and real-time performance, and users can add, delete or modify existing annotation content at any time. Therefore, in order to ensure the response efficiency and spatial consistency of annotation data, when a new annotation is added or an existing annotation is modified, the joint vector is recalculated according to the encoding result, and the joint vector is inserted into the R-tree index structure.

[0085] At the same time, if the annotation affects the spatial region boundary, a local R-tree node reconstruction operation is automatically triggered to maintain the optimality of the index structure. In addition, annotation modification will also trigger incremental update of the deviation field matching process, and only the affected area is re-matched using the aforementioned matching function, which greatly reduces the global computational burden.

[0086] When a user initiates annotation modification, the hash value of the spatial block to which the annotation belongs is calculated based on the annotation coordinates, and a distributed lock of the block is tried to be obtained; if the lock is already occupied, the user is prompted that the region is being edited by other users, please try later;

[0087] After successfully locking, the current operation version number is recorded, and the version consistency is checked when submitting the modification. If the version conflicts (such as other users have modified the same region), the two versions are automatically retained, and a conflict report is generated for the administrator to arbitrate.

[0088] S4: According to the dynamic update of spatial association, the visualization content and deviation correction are updated, the multi-user review data is extracted, and the detection report is automatically generated.

[0089] S4.1: Based on the two-dimensional R-tree index, the spatial association relationship between the current annotation spatial coordinates and the deviation value in the deviation field is queried, and the deviation field data associated with the annotation area is selected according to the principal curvature value of the corresponding position in the curvature feature matrix.

[0090] First, all the user comment information of the established spatial correlation relationship is dynamically extracted, specifically including the spatial coordinates of the comment, the gesture trajectory, the text keyword, and the corresponding deviation field region number.

[0091] Further, the deviation field data associated with the comment region is screened out, including:

[0092] Based on the two-dimensional R-tree index, the user comment coordinate point set is extracted, and the spatial coordinates of all data points in the deviation field are traversed. The principle is to detect the overlap based on the minimum distance domain and judge the spatial inclusion relationship to ensure that the comment point and the spatial position of the deviation data are highly consistent or approximate;

[0093] For each comment coordinate point, according to the spatial mapping relationship after six-degree-of-freedom alignment, the same coordinate point is located in the curvature feature matrix, and the principal curvature value is extracted;

[0094] The difference between the principal curvature of the data point in the deviation field and the principal curvature of the corresponding comment region is calculated, and the data point of the deviation field whose difference is less than the preset curvature tolerance threshold is removed. It should be noted that a curvature tolerance threshold needs to be set to eliminate those points with small difference and insufficient deformation to constitute effective deviation.

[0095] It should be noted that when traversing the deviation field data points, it is preferred to check whether it is marked as a user correction lock state. If it is in the locked state, skip the curvature tolerance screening. For non-locked points, calculate the difference between the principal curvature and the principal curvature of the comment region. If the difference is less than the preset threshold, it is removed. If there is a user correction operation in a comment region, the curvature tolerance threshold of the region is automatically expanded to 1.5 times the original value, which lasts until the next global deviation field update.

[0096] S4.2: According to the correction parameters in the user comment, the associated deviation field data is optimized by gradient descent, the spatial distribution of the deviation field is adjusted, and the deviation value in the visualized content is updated based on the color mapping rule.

[0097] In the present application, the correction parameters mainly include the expected value, the calibration value or the correction instruction actively input by the user during the comment process, such as "the offset should be 0, adjust 5mm to the left" and the like. The correction target is extracted from the user text and standardized as a constraint condition of the deviation optimization objective function.

[0098] For the above correction target, a nonlinear optimization strategy based on gradient descent method is used to adjust the spatial distribution of the deviation field.

[0099] It should be noted that during the deviation field optimization process, if it is detected that there is an unresolved version conflict in a certain region, the optimization calculation of the region is suspended until the conflict is resolved.

[0100] In the present application, the objective function is constructed as the sum of square errors between the current bias values and the user's correction target, and the adjustable parameters are the bias vector values of each spatial point in the bias field.

[0101] In each iteration, the gradient direction and step factor of all points are calculated respectively, and the bias values are updated according to the set learning rate parameter. This method can realize spatial optimization correction under the guidance of user's intention while ensuring the stability of global convergence, and has strong numerical stability and anti-interference ability.

[0102] After completing the bias optimization, the visual image content will be dynamically updated according to the color mapping, wherein the color mapping rule is to realize the visualization of the bias by mapping the bias values to different colors according to certain segmentation or continuous function.

[0103] This mechanism not only intuitively presents the spatial distribution changes before and after the bias correction, but also provides real-time visual feedback for multi-user interaction, which helps to quickly locate the problem area and confirm the correction effect.

[0104] After completing the bias field optimization, the point cloud data and CAD model data of the correction area are automatically extracted, the local six-degree-of-freedom alignment is performed (only for the correction area, not for the whole), and the bias field of the area is recalculated; the updated bias field data will overwrite the original results and trigger the real-time refresh of the visualization content.

[0105] S4.3: Generate a detection report containing the bias correction process, associated annotation positions and final detection results.

[0106] After all the steps of bias correction and visualization update are completed, a complete detection report is automatically generated, which is not only used for result archiving, but also serves as a key basis for quality tracing, process optimization and multi-user review.

[0107] In particular, the report also records the interaction log of each user correction, including correction time, operator, input parameters, optimization iteration rounds and other detailed information.

[0108] The embodiment also provides a computer device suitable for the virtual-real superimposed and multi-user cooperative curved surface bias detection method, which comprises a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the virtual-real superimposed and multi-user cooperative curved surface bias detection method proposed in the above embodiment.

[0109] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved by WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0110] The embodiment also provides a storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the curved surface deviation detection method for virtual-real superimposition and multi-user cooperation as described in the above embodiment.

[0111] To sum up, the curved surface deviation detection method for virtual-real superimposition and multi-user cooperation is constructed, high-precision detection and efficient collaborative review of complex curved surface structures are achieved, and the intelligentization and practicality level of deviation detection is significantly improved. By introducing the curvature feature matrix, the six-degree-of-freedom alignment mechanism and the deviation field coding, the geometric accuracy of the detection result and the completeness of the spatial expression can be effectively improved, and the identification and analysis capability of the complex curved surface error is enhanced. With the aid of the visual mapping mode of the VR / AR environment, the intuitiveness and interactivity of the detection result are improved, which helps users quickly understand and locate the deviation area. In addition, through the multi-user collaborative annotation and dynamic visual projection mechanism, multi-angle review and intelligent guidance are achieved, and the risk of misjudgment and missed detection is greatly reduced. The spatial index and dynamic update mechanism between the annotation data and the deviation information further support the real-time optimization and correction of the detection result, which is conducive to forming a high-quality and structured detection record.

[0112] To sum up, the curved surface deviation detection method for virtual-real superimposition and multi-user cooperation is constructed, high-precision detection and efficient collaborative review of complex curved surface structures are achieved, and the intelligentization and practicality level of deviation detection is significantly improved. By introducing the curvature feature matrix, the six-degree-of-freedom alignment mechanism and the deviation field coding, the geometric accuracy of the detection result and the completeness of the spatial expression can be effectively improved, and the identification and analysis capability of the complex curved surface error is enhanced. With the aid of the visual mapping mode of the VR / AR environment, the intuitiveness and interactivity of the detection result are improved, which helps users quickly understand and locate the deviation area. In addition, through the multi-user collaborative annotation and dynamic visual projection mechanism, multi-angle review and intelligent guidance are achieved, and the risk of misjudgment and missed detection is greatly reduced. The spatial index and dynamic update mechanism between the annotation data and the deviation information further support the real-time optimization and correction of the detection result, which is conducive to forming a high-quality and structured detection record.

[0113] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application rather than limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and all modifications or equivalent replacements should be included in the scope of the claims of the present application.

Claims

1. A surface deviation detection method based on virtual-real superposition and multi-user collaboration, characterized by: include: Collect point cloud data of the target object and calculate the curvature feature matrix of the point cloud data; At the same time, the point cloud data and the inverse model data are associated and stored; Complete the six-degree-of-freedom alignment operation between the inverse model data and the physical point cloud data. Based on the aligned spatial coordinates, perform surface deviation detection operations, encode the detection results into a deviation field, and perform preliminary visualization in the VR / AR environment using color mapping and contour lines. Multiple users conduct collaborative review based on the deviation field, dynamically visualize and project the annotation space coordinates according to the surface curvature characteristics, and jointly encode the annotation space coordinates, user gesture trajectories, and text keywords to establish a two-dimensional R-tree annotation index, spatially correlating the annotation data with the deviation field results. Dynamically update visualization content and deviation correction based on spatial correlation, extract multi-user review data, and automatically generate inspection reports; The dynamic updating of visualization content and deviation correction includes: querying the spatial correlation between the current annotation space coordinates and the deviation value in the deviation field based on the two-dimensional R-tree index, screening the deviation field data associated with the annotation area according to the principal curvature value of the corresponding position in the curvature feature matrix; performing gradient descent optimization on the associated deviation field data according to the user annotation, adjusting the spatial distribution of the deviation field, and updating the deviation value in the visualization content based on color mapping; The method of screening out the deviation field data associated with the annotation area includes: extracting the user annotation coordinate point set based on the two-dimensional R-tree index, and traversing the spatial coordinates of all data points in the deviation field; for each annotation coordinate point, locating the same coordinate point in the curvature feature matrix according to the spatial mapping relationship after six-degree-of-freedom alignment, and extracting the principal curvature value; calculating the difference between the principal curvature of the data point in the deviation field and the principal curvature of the corresponding annotation area, and removing the deviation field data points whose difference is less than a preset curvature tolerance threshold.

2. The surface deviation detection method for virtual-real superposition and multi-user collaboration according to claim 1, characterized in that: The generation of the curvature characteristic matrix includes: The point cloud data is screened in a local neighborhood, and several neighboring points are selected for each data point using a certain neighborhood radius. The selected local neighborhood is fitted with a least squares plane to obtain the local fitting plane parameters of each neighborhood point. Based on the local fitting plane, the curvature tensor of each data point is calculated, and the principal curvature value of each point is extracted. The Gaussian curvature and mean curvature of each data point are calculated using the principal curvature values ​​of the curvature tensor, and a curvature feature matrix is ​​constructed according to the spatial position of the point.

3. The surface deviation detection method for virtual-real superposition and multi-user collaboration according to claim 1, characterized in that: The six-degree-of-freedom alignment operation includes: Extract feature points from the inverse model data and physical point cloud data, and match the feature points; Based on the feature point matching results, the inverse model data and the physical point cloud data are aligned with six degrees of freedom.

4. The surface deviation detection method for virtual-real superposition and multi-user collaboration according to claim 3, characterized in that: The performing of the curved surface deviation detection operation comprises: Based on the aligned spatial coordinates, the surface deviation between the inverse model data and the physical point cloud data is calculated; by analyzing the distance error of each feature point, a deviation field containing the spatial position and deviation value is generated.

5. The surface deviation detection method for virtual-real superposition and multi-user collaboration according to claim 1, characterized in that: The dynamic visualization projection includes: When the user makes an annotation, the spatial coordinate point of the annotation is determined by the surface curvature feature matrix, and whether the annotation coordinate point is located in the blind spot of the field of view is determined according to the position and shape of the surface; When the annotation is located in the blind spot of the surface, it is projected to the visible area along the surface normal based on the annotation space coordinates, and a dynamic guide line is generated to guide the user's line of sight to the actual location of the annotation.

6. The method for detecting surface deviations by combining virtual and real objects and multi-user collaboration according to claim 5, wherein: The spatial association of the annotation data with the deviation field result includes: The spatial coordinates, user gesture trajectory and text keywords of each annotation are combined for joint encoding, and a spatial index of the annotation data is established through a two-dimensional R-tree structure; Through the established two-dimensional R-tree index, the spatial coordinates in the annotation data and the deviation field data in the detection results can be quickly queried and matched; According to the spatial matching results of the annotation space coordinates and the deviation field, the specific detection result location corresponding to each annotation is confirmed; When new user annotations are added or modified, the spatial matching between the annotations and the deviation field is dynamically adjusted according to the R-tree index, and the spatial association data is updated in real time.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the surface deviation detection method for virtual-real superposition and multi-user collaboration described in any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the surface deviation detection method for virtual-real superposition and multi-user collaboration described in any one of claims 1 to 6 are implemented.

Images