Drawing labeling collaboration method and system for multi-terminal mobile equipment

Abstract

The invention discloses a drawing annotation collaboration method and system for a multi-terminal mobile device, and relates to the technical field related to data processing, and the method comprises the steps: connecting the multi-terminal mobile device, and obtaining a plurality of drawing files; performing three-dimensional reconstruction on the two-dimensional plane drawing and the environment perception data, and the point cloud data and the environment perception data; performing space alignment on the first basic twin model and the second basic twin model, and marking a space alignment area; collecting error measurement data in a filling measurement mode; and checking and determining a marking deviation coordinate, and carrying out drawing marking collaborative management through the multi-terminal mobile equipment. The technical problems that in the prior art, due to multi-user labeling conflicts, drawing labeling error data are difficult to accurately recognize and correct, the accuracy and quality of drawing labeling are insufficient, and the collaborative management efficiency of drawing labeling is poor are solved. The technical effects of improving the accuracy of drawing labeling and improving the collaborative management efficiency of drawing labeling are achieved.

Description

Technical Field

[0001] The present application relates to the technical field related to data processing, and specifically to a collaborative method and system for drawing annotation of multiple mobile devices. Background Art

[0002] In complex engineering scenarios, design teams usually need to deal with multiple heterogeneous data sources, such as two-dimensional drawings, three-dimensional point cloud scanning data, and environmental perception information (such as temperature, lighting, spatial layout, etc.). Traditional drawing annotation methods mainly rely on a single data source (such as two-dimensional drawings or independent three-dimensional models), and lack the ability to dynamically integrate multimodal data. For example, two-dimensional drawings cannot intuitively reflect the spatial changes of the actual environment, and static three-dimensional point cloud data is difficult to capture dynamic environmental parameters in real time, resulting in deviations between the design model and the real scene; in addition, collaborative annotation on multiple devices (such as mobile terminals, tablets, AR / VR devices) often faces problems such as data asynchrony and version conflicts. Especially in the process of spatial alignment and error verification, manual comparison or offline processing is often used, which is inefficient and prone to human errors, thus affecting the accuracy and reliability of drawing annotation.

[0003] Therefore, in the current relevant technologies, there are technical problems such as multi-user annotation conflicts, difficulty in accurately identifying and correcting drawing annotation error data, which in turn leads to insufficient accuracy and quality of drawing annotation and poor efficiency of collaborative management of drawing annotation. Summary of the invention

[0004] The present application solves the technical problems existing in the prior art of multi-user annotation conflicts, difficulty in accurately identifying and correcting drawing annotation error data, and resulting in insufficient accuracy and quality of drawing annotation and poor efficiency of collaborative management of drawing annotation by providing a drawing annotation collaborative method and system for multi-terminal mobile devices, thereby achieving the technical effect of improving the accuracy of drawing annotation and improving the efficiency of collaborative management of drawing annotation.

[0005] The present application provides a drawing annotation collaborative method for multi-terminal mobile devices, the method comprising: connecting multi-terminal mobile devices to obtain multiple drawing files, the multiple drawing files comprising point cloud data, two-dimensional plane drawings, and environmental perception data; performing three-dimensional reconstruction through the two-dimensional plane drawings and environmental perception data, point cloud data and environmental perception data in the multiple drawing files, respectively, and setting a first basic twin model and a first drawing annotation set, a second basic twin model and a second drawing annotation set; according to the first drawing annotation set and the second drawing annotation set, spatially aligning the first basic twin model and the second basic twin model, and marking the spatial alignment area; at the same time, based on the spatial alignment area, collecting error measurement data in a filling measurement manner, the error measurement data comprising geometric error data and assembly error data; based on the error measurement data, verifying the first drawing annotation set and the second drawing annotation set, determining the annotation deviation coordinates, and performing drawing annotation collaborative management through the multi-terminal mobile devices.

[0006] In a possible implementation, the drawing annotation collaborative method for multiple mobile devices also performs the following processing: vectorizing the two-dimensional plane drawings in the multiple drawing files and extracting the drawing structured information; setting a first unidirectional feature queue based on the drawing structured information; performing feature matching according to the first unidirectional feature queue, and performing multi-perspective geometric reconstruction, and setting a first base model architecture.

[0007] In a possible implementation, the collaborative method for drawing annotation of multiple mobile devices also performs the following processing: denoising the environmental perception data in the multiple drawing files, and then obtaining environmental structured information through scale-invariant feature transformation; setting a second unidirectional feature queue based on the environmental structured information; performing feature matching according to the second unidirectional feature queue, and performing multi-perspective geometric reconstruction, and setting a second base model architecture; wherein the first base model architecture and the second base model architecture are pre-trained models of the first basic twin model.

[0008] In a possible implementation, the drawing annotation collaborative method for multiple mobile devices further performs the following processing: performing multi-frame registration processing on the point cloud data in the multiple drawing files, configuring initialization registration parameters and an initial transformation matrix; based on the initialization registration parameters and the initial transformation matrix, extracting point cloud structured information, and setting a third unidirectional feature queue.

[0009] In a possible implementation, the collaborative method for drawing annotation of a multi-terminal mobile device further performs the following processing: performing feature matching using a third unidirectional feature queue and a second unidirectional feature queue to locate feature matching points; determining a second drawing annotation set based on a third base model architecture corresponding to the third unidirectional feature queue and a second base model architecture corresponding to the second unidirectional feature queue, combined with the distance difference and angle difference between the feature matching points.

[0010] In a possible implementation, the drawing annotation collaborative method for multiple mobile devices further performs the following processing: the assembly error data includes gap deviation data and misalignment deviation data; the assembly error data is stored in a cloud database, and the multiple mobile devices are used for multi-terminal synchronous display.

[0011] In a possible implementation, the method for collaborative drawing annotation for multiple mobile devices further performs the following processing: generating a deviation correction scheme based on the error measurement data, and synchronizing the drawing annotation data and the deviation correction scheme in real time through a cloud server; at the same time, connecting to the cloud database to record the historical correction operations and timestamps of the annotation deviations; setting annotation permissions and an annotation locking mechanism based on the historical correction operations and timestamps in the cloud database, wherein the annotation permissions are used to support collaborative annotation for multiple devices.

[0012] In a possible implementation, the method for collaboratively annotating drawings of multiple mobile devices further performs the following processing: defining the annotation deviation vector D as the first drawing annotation set and the second drawing annotation set The difference between ,……, }, where the deviation of each annotation point is = - , and Mark the first drawing separately and the second drawing annotation set The coordinates of the jth annotation point in , where N is the total number of annotation points; based on the spatial alignment area, the weight of each annotation point is calculated ,in, Used to quantify the influence of different annotation points on drawing accuracy and assembly quality in overall deviation verification. is the distance between the jth annotation point and the boundary of the spatial alignment region.

[0013] In a possible implementation, the method for collaboratively annotating drawings of multiple mobile devices further performs the following processing: the verification index calculation formula is E= ; The verification index E is used to quantify the degree of deviation between the two annotation sets. represents the Euclidean norm corresponding to the jth annotation point, is the weight of the jth annotation point.

[0014] The present application also provides a drawing annotation collaborative system for multi-terminal mobile devices, including: a drawing file acquisition module, used to connect multi-terminal mobile devices and acquire multiple drawing files, wherein the multiple drawing files include point cloud data, two-dimensional plane drawings, and environmental perception data; a three-dimensional reconstruction module, used to perform three-dimensional reconstruction through the two-dimensional plane drawings and environmental perception data, point cloud data and environmental perception data in the multiple drawing files, respectively, and set a first basic twin model and a first drawing annotation set, a second basic twin model and a second drawing annotation set; a spatial alignment module, used to spatially align the first basic twin model and the second basic twin model according to the first drawing annotation set and the second drawing annotation set, and mark the spatial alignment area; an error measurement data acquisition module, used to simultaneously, based on the spatial alignment area, collect error measurement data in a filling measurement manner, wherein the error measurement data includes geometric error data and assembly error data; an annotation deviation coordinate determination module, used to verify the first drawing annotation set and the second drawing annotation set based on the error measurement data, determine the annotation deviation coordinates, and perform drawing annotation collaborative management through the multi-terminal mobile devices.

[0015] The collaborative drawing annotation method and system for multi-terminal mobile devices proposed in this application is intended to connect multi-terminal mobile devices and obtain multiple drawing files; perform three-dimensional reconstruction through two-dimensional plane drawings and environmental perception data, point cloud data and environmental perception data respectively; spatially align the first basic twin model and the second basic twin model, and mark the spatial alignment area; collect error measurement data in a filling measurement manner; verify and determine the annotation deviation coordinates, and perform collaborative management of drawing annotation through multi-terminal mobile devices. It solves the technical problems existing in the prior art of multi-user annotation conflicts, difficulty in accurately identifying and correcting drawing annotation error data, and thus leads to insufficient accuracy and quality of drawing annotation and poor efficiency of collaborative management of drawing annotation, and achieves the technical effect of improving the accuracy of drawing annotation and improving the efficiency of collaborative management of drawing annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the embodiment of the present disclosure, the accompanying drawings of the embodiment of the present disclosure will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0017] Figure 1 A schematic flow chart of a collaborative method for annotating drawings using multiple mobile devices is provided in accordance with an embodiment of the present application.

[0018] Figure 2 A schematic diagram of the structure of a drawing annotation collaborative system for multiple mobile devices provided in an embodiment of the present application.

[0019] Explanation of the reference numerals: drawing file acquisition module 10 , three-dimensional reconstruction module 20 , space alignment module 30 , error measurement data acquisition module 40 , annotation deviation coordinate determination module 50 . DETAILED DESCRIPTION

[0020] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0021] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0022] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0023] The present application embodiment provides a collaborative method for marking drawings on multiple mobile devices, such as Figure 1 As shown, the method includes:

[0024] Step S100, connecting multiple mobile devices to obtain multiple drawing files, wherein the multiple drawing files include point cloud data, two-dimensional plane drawings, and environmental perception data.

[0025] Preferably, by connecting multiple mobile devices (such as smart phones, tablets, computers, AR / VR devices, etc.), multiple types of drawing files are obtained from these devices, and the multiple drawing files specifically include point cloud data, two-dimensional plane drawings, and environmental perception data. Among them, multiple mobile devices are used for data collection, transmission and collaborative processing. For example, two-dimensional drawings or point cloud data are obtained through smart phones or tablets, and environmental perception data are collected through AR / VR devices or devices equipped with sensors. Specifically, point cloud data is obtained through laser scanners, photogrammetry equipment or depth cameras (such as LiDAR, Kinect). The data set used to represent the geometric shape of the surface of an object consists of a large number of three-dimensional points, each of which contains spatial coordinates ( X, Y, Z), and sometimes may also contain information such as color and reflection intensity, and are usually used to build high-precision three-dimensional models; two-dimensional plane drawings are traditional design drawings, usually in CAD (computer-aided design) file formats (such as DWG, DXF, etc.) or image formats (such as PDF, PNG, etc.), containing plan views, cross-sections, elevations, etc., and are used to describe the design details of engineering projects such as buildings and machinery; environmental perception data is dynamic information related to the surrounding environment collected by sensors (such as temperature sensors, humidity sensors, light sensors, cameras, radars, etc.), which may include ambient temperature, humidity, lighting conditions, spatial layout, obstacle locations, etc. Environmental perception data is usually used to enhance the understanding of actual scenes.

[0026] Step S200, three-dimensional reconstruction is performed respectively through the two-dimensional plane drawings and environmental perception data, point cloud data and environmental perception data in the multiple drawing files, and a first basic twin model and a first drawing annotation set, a second basic twin model and a second drawing annotation set are set.

[0027] Preferably, the two-dimensional plane drawings and environmental perception data in multiple drawing files are 3D reconstructed, that is, the two-dimensional plane drawings are combined with the environmental perception data, and a preliminary 3D model is generated by using an algorithm (such as rule-based 3D modeling, parametric modeling, etc.). Specifically, the two-dimensional plane drawings are first parsed and feature extracted, that is, the two-dimensional drawing files are read using a CAD parsing tool to extract geometric features (such as lines, polygons, annotations, etc.), and then the environmental perception data from different sensors are fused, for example, the image data collected by the camera is registered with the point cloud data collected by the lidar to generate a comprehensive data set containing geometric and environmental information, and then mapped to the coordinate system of the two-dimensional plane drawing through feature matching or coordinate transformation to integrate the sensor data with the drawing. finally, according to the geometric information and environmental perception data (such as spatial layout, obstacle location, etc.) in the two-dimensional plane drawing, a preliminary three-dimensional model is generated using a rule engine or parametric modeling tool (such as Revit, SketchUp, etc.), and the dynamic information in the environmental perception data (such as light, temperature, humidity, etc.) is added to the three-dimensional model, so as to construct a first basic twin model, which is used to simulate and reflect the scene after the two-dimensional drawing is combined with the environmental perception data. It not only contains geometric information, but also incorporates environmental perception data, which can reflect the dynamic characteristics of the actual scene. The design parameters, environmental characteristics, key point marks, etc. generated on the basis of the first basic twin model constitute the corresponding first drawing annotation set, which is used for subsequent collaborative management and error verification.

[0028] Preferably, the point cloud data in multiple drawing files are reconstructed in three dimensions with the environmental perception data, that is, the point cloud data is combined with the environmental perception data, and a high-precision three-dimensional model is generated using a point cloud processing algorithm (such as point cloud registration, surface reconstruction, etc.). Specifically, the point cloud data is preprocessed first, that is, the original point cloud data is denoised using statistical filtering, radius filtering, etc. to remove outliers and noise, and then the point cloud data from multiple scanning devices are registered using ICP (iterative closest point algorithm) and feature matching (such as SIFT, SURF), that is, aligned to the same coordinates, and then the point cloud data is divided into different regions or pairs through region growing method, clustering algorithm (such as DBSCAN, K-Means), etc. Similarly, the environmental perception data from different sensors are fused to obtain a comprehensive data set, and then the sensor data is aligned with the corresponding position in the point cloud through feature matching or coordinate transformation, and then the object surface model is generated through the point cloud data, and then the dynamic information in the environmental perception data (such as temperature, humidity, light, etc.) is added to the three-dimensional model, so as to obtain the second basic twin model, which is used to simulate and reflect the scene after the point cloud data and environmental perception data are combined. It not only contains geometric details, but also incorporates environmental perception data, which can more realistically reflect the actual scene. The geometric features, environmental parameters, error marks, etc. generated on the basis of the second basic twin model constitute the corresponding second drawing annotation set. The first basic twin model and the second basic twin model reflect the same scene features from different angles (design intent and high-precision geometry), and then through alignment verification, the consistency between the design model and the actual scene is achieved, thereby improving the accuracy of the drawing annotation.

[0029] Furthermore, step S200 also includes step S210, vectorizing the two-dimensional plane drawings in the multiple drawing files to extract the drawing structured information; step S220, setting a first unidirectional feature queue based on the drawing structured information; step S230, performing feature matching according to the first unidirectional feature queue, and performing multi-perspective geometric reconstruction to set a first base model architecture.

[0030] Preferably, two-dimensional plane drawings in multiple drawing files are vectorized, that is, two-dimensional plane drawings (usually bitmaps or scanned images) are converted into vector formats (such as DXF, DWG, SVG, etc.) to extract structural information. Specifically, the drawing images are subjected to denoising, binarization, edge detection and other operations to improve vectorization accuracy, and geometric elements (such as lines, polygons, and text) in the images are converted into vector data using vectorization tools or algorithms (such as Hough transform and contour extraction). Then, the drawing structural information is extracted from the vector data, including using a rule engine or a machine learning model (such as OCR technology) to identify and extract geometric elements and annotation information in the drawings; then, a first unidirectional feature queue is generated based on the drawing structural information to describe the geometric and annotation information in the drawings. Specifically, key features are extracted from structured information, the extracted features are arranged into a one-way queue according to certain rules (such as spatial order, order of importance), and the feature queue is stored as a computer-processable data structure (such as an array, a linked list) to obtain a first one-way feature queue; finally, a feature matching algorithm (such as SIFT, SURF) is used to match the features in the first one-way feature queue with features in other data sources (such as point cloud data, multi-view images), and based on the feature matching results, a multi-view geometry algorithm (such as triangulation, bundle adjustment) is used to generate a three-dimensional geometric model, that is, data of drawings or scenes (such as multi-view images, point cloud data) are obtained from different viewpoints, and corresponding points in the multi-view data are found through feature matching and the three-dimensional coordinates are calculated, thereby obtaining the first base model architecture.

[0031] Furthermore, step S200 also includes step S240, denoising the environmental perception data in the multiple drawing files, and then obtaining environmental structured information through scale-invariant feature transformation; step S250, setting a second unidirectional feature queue based on the environmental structured information; step S260, performing feature matching according to the second unidirectional feature queue, and performing multi-perspective geometric reconstruction, and setting a second base model architecture; step S270, wherein the first base model architecture and the second base model architecture are pre-trained models of the first basic twin model.

[0032] Preferably, the environmental perception data in multiple drawing files are denoised, that is, the data is smoothed using algorithms such as Kalman filtering and low-pass filtering to remove noise and outliers and improve data quality. Then, key feature information with clear semantics is extracted from the environmental perception data through scale-invariant feature transform (SIFT) to obtain environmental structured information, including spatial layout and dynamic information. Similarly, a second unidirectional feature queue is generated based on the extracted environmental structured information to describe the key features in the environmental perception data, and the features in the second unidirectional feature queue are matched with features in other data sources (such as point cloud data and multi-view images), that is, feature matching algorithms (such as SIFT, SURF, and ORB) are used to find corresponding points between environmental features and features of other data sources. Finally, based on the feature matching results, a three-dimensional geometric model is generated using a multi-view geometry algorithm (such as triangulation and bundle adjustment), that is, environmental perception data (such as multi-view images and point cloud data) is obtained from different viewpoints, corresponding points in the multi-view data are found through feature matching and the coordinates of these points are calculated, thereby obtaining a second base model architecture. Finally, pre-training is performed based on the first base model architecture and the second base model architecture, and annotation information and environmental data are added to obtain the first basic twin model.

[0033] Furthermore, step S200 also includes step S280, performing multi-frame registration processing on the point cloud data in the multiple drawing files, configuring initialization registration parameters and initial transformation matrix; step S290, based on the initialization registration parameters and initial transformation matrix, extracting point cloud structured information, and setting a third unidirectional feature queue.

[0034] Preferably, the point cloud data in multiple drawing files contain geometric information of the object surface (such as point coordinates X, Y, Z, and may also include color, reflection intensity, etc.), and multi-frame registration processing is performed on them, that is, multi-frame point cloud data (i.e., point clouds collected from different perspectives or time points) are aligned to the same coordinate system. Through registration, the relative position and posture differences between multi-frame point clouds are eliminated, and initialization registration parameters are configured at the same time, that is, appropriate initialization registration parameters are set according to the characteristics of point cloud data (such as density, noise level), usually including sampling rate (controlling the sampling density of point cloud data and reducing the amount of calculation), feature extraction method (using SIFT, SURF and other algorithms), matching threshold (threshold for judging whether feature points match), and use ICP (iterative closest point algorithm) or NDT (normal distribution transformation) for fine registration optimization. In addition, the initial transformation matrix is ​​configured in the multi-frame registration process. Matrix, which is used to transform one frame of point cloud data into another frame of point cloud data coordinate system, usually includes rotation matrix (describing rotation relationship) and translation vector (describing translation relationship). For example, the corresponding points between the two frames of point cloud are found through feature matching (such as SIFT, SURF), and the least squares method is used to calculate the rotation matrix and translation vector; then, according to the configured initialization alignment parameters and initial transformation matrix, the point cloud structured information (including geometric information and feature information) is extracted, and finally, based on the point cloud structured information, a third unidirectional feature queue is generated to describe the key features in the point cloud data, that is, the key features are extracted from the point cloud structured information, the extracted features are arranged into a unidirectional queue according to certain rules (such as spatial order, order of importance) and the feature queue is stored as a computer-processable data structure (such as an array, a linked list), thereby improving the utilization efficiency and modeling accuracy of the point cloud data.

[0035] Furthermore, step S200 also includes step A, using the third unidirectional feature queue and the second unidirectional feature queue to perform feature matching and locate feature matching points; step B, based on the third base model architecture corresponding to the third unidirectional feature queue and the second base model architecture corresponding to the second unidirectional feature queue, combined with the distance difference and angle difference between the feature matching points, determining the second drawing annotation set.

[0036] Preferably, the features in the third unidirectional feature queue and the second unidirectional feature queue are matched. Specifically, a feature matching algorithm (such as SIFT, SURF, ORB) is used to compare the features of the two feature queues, identify and locate feature matching points, that is, the corresponding key points in the third unidirectional feature queue and the second unidirectional feature queue, and then align the third base model architecture corresponding to the third unidirectional feature queue and the second base model architecture corresponding to the second unidirectional feature queue in the spatial coordinate system to ensure that different features of the same scene can be accurately reflected, and then calculate the Euclidean distance difference of the feature matching points in the spatial coordinate system, and calculate the directional angle difference of the two feature matching points in the spatial coordinate system; then combine the distance difference and angle difference between the feature matching points, analyze the geometric error and assembly error between the two model architectures, generate multiple drawing annotation information, and constitute a second drawing annotation set, which is used to describe the difference between the two model architectures, thereby improving the consistency and accuracy of the drawing annotation.

[0037] Step S300: spatially align the first basic twin model and the second basic twin model according to the first drawing annotation set and the second drawing annotation set, and mark the spatial alignment area.

[0038] Preferably, by comparing the first drawing annotation set and the second drawing annotation set, common features or areas (such as key points, geometric shapes, environmental parameters, etc.) are extracted, and a feature matching algorithm (such as SIFT, SURF, ORB, etc.) or a geometric matching algorithm (such as ICP iterative closest point algorithm) is used to find the correspondence between the two models; then, based on the matching results, the spatial transformation parameters (such as translation, rotation, scaling, etc.) between the two models are calculated, and the first basic twin model or the second basic twin model is coordinate transformed so that they are aligned in the same spatial coordinate system, and then the accuracy of the alignment result is verified by visualization or numerical calculation, for example, checking whether the alignment error of the key points is within the allowable range, and finally marking the alignment area (that is, the area covered by the two models), which is helpful for subsequent collaborative management and error verification, thereby improving the accuracy of the drawing annotation.

[0039] Step S400: At the same time, based on the spatial alignment area, error measurement data is collected in a filling measurement manner, and the error measurement data includes geometric error data and assembly error data.

[0040] Preferably, based on the spatial alignment area, error measurement data is collected in a filling measurement manner, which means that in the spatial alignment area, the error measurement data of the model is obtained by dense sampling or interpolation. For example, in the spatial alignment area, the point cloud data of the design model and the actual model are compared point by point, and the deviation of each point is calculated; the design model and the actual model are converted into a mesh representation, and the geometric error is calculated by comparing the mesh vertices or faces; the design model and the actual model are converted into a voxel representation, and the error distribution is calculated by comparing the voxels; in the filling measurement process, error measurement data, including geometric error data, are collected. The geometric error refers to the difference in geometric shape between the design model and the actual model. Specifically, the Euclidean distance between the design model and the actual model is calculated through point cloud comparison or grid comparison, and quantified using algorithms (such as Hausdorff distance and mean square error); the assembly error refers to the difference in assembly relationship between the design model and the actual model, such as the deviation of equipment installation position, the misalignment of pipeline connection points, etc. Specifically, the deviation of the design model and the actual model at key assembly points is calculated through feature point matching, and the assembly error is quantified using algorithms (such as ICP iterative closest point algorithm). Through geometric error data and assembly error data, the differences are analyzed and error correction is performed to ensure the accuracy and consistency of drawing annotations.

[0041] Preferably, step S400 also includes step S410, wherein the assembly error data includes gap deviation data and misalignment deviation data; and step S420, wherein the assembly error data is stored in a cloud database, and the multi-terminal mobile device is used for multi-terminal synchronous display.

[0042] Preferably, the assembly error data includes gap deviation data and misalignment deviation data. Specifically, the gap deviation data refers to the deviation between the gap between two assembly components and the design value, and the misalignment deviation data refers to the deviation between the relative position between two assembly components and the design value. The assembly error data (such as gap deviation and misalignment deviation) are converted into a structured data format (such as JSON, XML) and uploaded to a cloud database via the network to achieve centralized data management and multi-terminal access; using multi-terminal mobile devices for multi-terminal synchronous display refers to synchronizing the assembly error data in the cloud database to multi-terminal mobile devices for display, ensuring that all team members can access the latest data in real time, and supporting users to view, edit and annotate error data on multi-terminal devices.

[0043] Step S500: Based on the error measurement data, the first drawing annotation set and the second drawing annotation set are verified to determine the annotation deviation coordinates, and the drawing annotation collaborative management is performed through the multi-terminal mobile device.

[0044] Preferably, according to the collected error measurement data, the first drawing annotation set and the second drawing annotation set are verified to find the deviation between them, that is, to determine the annotation deviation coordinates. Specifically, the annotation information (such as design parameters, key point marks) in the first drawing annotation set is compared with the annotation information (such as geometric features, key point marks) in the second drawing annotation set. According to the error measurement data, the deviation between the two annotation sets is calculated, for example, the coordinate deviation between the key points in the design model and the key points in the actual model is calculated, and the calculated deviation is mapped to the spatial coordinate system to determine the specific position coordinates of the deviation (that is, the annotation deviation coordinates); then, the drawing annotation is collaboratively managed through multi-terminal mobile devices, including synchronizing the annotation deviation coordinates and the error measurement data to the multi-terminal mobile devices, and updating the drawing annotation information, such as marking the deviation area in the design model, and supporting multiple teams to collaboratively edit the drawing annotation information on the multi-terminal devices. For example, the design team can adjust the design parameters on the tablet computer, and the construction team can view the adjusted annotation information on the AR device; the error correction status is fed back in real time by the multi-terminal devices, thereby significantly improving the consistency of design and construction and the accuracy and reliability of the drawing annotation, and improving the efficiency of the collaborative management of drawing annotation.

[0045] Furthermore, step S500 also includes step S510, generating a deviation correction plan based on the error measurement data, and synchronizing the drawing annotation data and the deviation correction plan in real time through the cloud server; step S520, at the same time, connecting to the cloud database to record the historical correction operations and timestamps of the annotation deviation; step S530, setting annotation permissions and annotation locking mechanisms based on the historical correction operations and timestamps in the cloud database, and the annotation permissions are used to support multi-terminal collaborative annotation.

[0046] Preferably, the error source and impact range are analyzed based on the geometric error data and assembly error data (such as gap deviation and misalignment deviation), and a correction suggestion, i.e., a deviation correction plan, is generated using a rule engine or a machine learning model. For example, if the misalignment deviation of the equipment installation exceeds a threshold, it is recommended to reinstall the equipment. The deviation correction plan is then stored in a structured data format (such as JSON, XML), and the drawing annotation data and the deviation correction plan are synchronized to multiple mobile devices in real time through a cloud server. For example, the data is synchronized to multiple mobile devices through a real-time database (such as Firebase) or a message queue (such as MQTT). All team members can view the latest drawing annotation data and deviation correction plan in real time to ensure information consistency. The cloud database is then connected to record the historical correction operations and timestamps of the annotation deviations. That is, each time a correction operation occurs, the operation details (such as operation type, operation parameters) are recorded in the cloud database, and a correction plan is generated for each correction. The operation adds a timestamp to record the time when the operation occurred; finally, based on the historical revision operations and timestamps in the cloud database, set the annotation permissions and annotation locking mechanism, where the annotation permissions refer to the control of the operation permissions of different users or devices in multi-terminal collaborative annotation, and the annotation locking mechanism refers to preventing multiple users from editing the same annotation at the same time in multi-terminal collaborative annotation. For example, when a user edits a certain annotation, the annotation will be locked, and other users can only view it but not edit it; specifically, different annotation permissions are set according to user roles (such as administrator, ordinary user) and historical revision operation records, or when a user starts to edit a certain annotation, the system marks the annotation as "locked" in the cloud database. When other users try to edit the annotation, the system will prompt "The annotation has been locked". After the editing is completed, the system will release the lock and update the historical revision operation records, thereby realizing efficient multi-terminal collaborative annotation and management, and improving the accuracy of drawing annotation.

[0047] Further, step S500 further includes step S501, defining the annotation deviation vector D as the first drawing annotation set and the second drawing annotation set The difference between ,……, }, where the deviation of each annotation point is = - , and Mark the first drawing separately and the second drawing annotation set The coordinates of the jth annotation point in , where N is the total number of annotation points; Step S502, calculating the weight of each annotation point based on the spatial alignment area ,in, Used to quantify the influence of different annotation points on drawing accuracy and assembly quality in overall deviation verification. is the distance between the jth annotation point and the boundary of the spatial alignment region.

[0048] Preferably, the annotation deviation vector D includes the first drawing annotation set and the second drawing annotation set The multiple differences between the annotation points and the boundary of the spatial alignment area The smaller the weight The larger it is, the greater the impact of the annotation point on the drawing accuracy and assembly quality in the deviation verification.

[0049] Furthermore, step S500 also includes: the verification index calculation formula is E= ; The verification index E is used to quantify the degree of deviation between the two annotation sets. represents the Euclidean norm corresponding to the jth annotation point, is the weight of the jth annotation point.

[0050] Preferably, is the deviation of the marked point A measurement method that reflects the deviation of the annotation point in space, which is squared and multiplied by the weight , then sum all the labeled points, and finally take the square root and divide by , a comprehensive verification index E is obtained, which is used to quantify the degree of deviation between the two annotation sets. Through the verification index, we can intuitively understand the difference between the first drawing annotation set and the second drawing annotation set, which helps to evaluate the consistency of different annotation sets and then judge the accuracy and reliability of the drawing annotation.

[0051] In the above, refer to Figure 1 A method for collaboratively annotating drawings using multiple mobile devices according to an embodiment of the present invention is described in detail. Figure 2 A drawing annotation collaborative system for multiple mobile devices according to an embodiment of the present invention is described.

[0052] A collaborative drawing annotation system for multiple mobile devices according to an embodiment of the present invention is used to solve the technical problems existing in the prior art of multi-user annotation conflicts, difficulty in accurately identifying and correcting drawing annotation error data, which leads to insufficient accuracy and quality of drawing annotations, and poor collaborative management efficiency of drawing annotations, thereby achieving the technical effect of improving the accuracy of drawing annotations and improving the collaborative management efficiency of drawing annotations. Figure 2As shown, a drawing annotation collaborative system for multiple mobile devices includes: a drawing file acquisition module 10, a three-dimensional reconstruction module 20, a space alignment module 30, an error measurement data acquisition module 40, and an annotation deviation coordinate determination module 50.

[0053] A drawing file acquisition module 10 is used to connect multiple mobile devices and acquire multiple drawing files, wherein the multiple drawing files include point cloud data, two-dimensional plane drawings, and environmental perception data; a three-dimensional reconstruction module 20 is used to perform three-dimensional reconstruction through the two-dimensional plane drawings and environmental perception data, point cloud data and environmental perception data in the multiple drawing files, respectively, and set a first basic twin model and a first drawing annotation set, a second basic twin model and a second drawing annotation set; a spatial alignment module 30 is used to spatially align the first basic twin model and the second basic twin model according to the first drawing annotation set and the second drawing annotation set, and mark the spatial alignment area; an error measurement data acquisition module 40 is used to simultaneously, based on the spatial alignment area, collect error measurement data in a filling measurement manner, wherein the error measurement data includes geometric error data and assembly error data; a marking deviation coordinate determination module 50 is used to verify the first drawing annotation set and the second drawing annotation set based on the error measurement data, determine the marking deviation coordinates, and perform drawing annotation collaborative management through the multi-terminal mobile devices.

[0054] The specific configuration of the three-dimensional reconstruction module 20 will be described in detail below. The three-dimensional reconstruction module 20 further includes: performing vectorization processing on the two-dimensional plane drawings in the multiple drawing files to extract the drawing structured information; setting a first unidirectional feature queue based on the drawing structured information; performing feature matching according to the first unidirectional feature queue, performing multi-view geometric reconstruction, and setting a first base model architecture.

[0055] The specific configuration of the three-dimensional reconstruction module 20 will be described in detail below. The three-dimensional reconstruction module 20 further includes: denoising the environmental perception data in the multiple drawing files, and then obtaining environmental structured information through scale-invariant feature transformation; setting a second unidirectional feature queue based on the environmental structured information; performing feature matching according to the second unidirectional feature queue, and performing multi-view geometric reconstruction, and setting a second base model architecture; wherein the first base model architecture and the second base model architecture are pre-trained models of the first basic twin model.

[0056] The specific configuration of the three-dimensional reconstruction module 20 will be described in detail below. The three-dimensional reconstruction module 20 further includes: performing multi-frame registration processing on the point cloud data in the multiple drawing files, configuring initialization registration parameters and initial transformation matrix; extracting point cloud structured information based on the initialization registration parameters and initial transformation matrix, and setting a third unidirectional feature queue.

[0057] The specific configuration of the three-dimensional reconstruction module 20 will be described in detail below. The three-dimensional reconstruction module 20 further includes: using the third unidirectional feature queue and the second unidirectional feature queue to perform feature matching and locate feature matching points; based on the third base model architecture corresponding to the third unidirectional feature queue and the second base model architecture corresponding to the second unidirectional feature queue, combined with the distance difference and angle difference between the feature matching points, determining the second drawing annotation set.

[0058] The specific configuration of the error measurement data acquisition module 40 will be described in detail below. The error measurement data acquisition module 40 further includes: the assembly error data includes gap deviation data and misalignment deviation data; the assembly error data is stored in a cloud database, and the multi-terminal mobile device is used for multi-terminal synchronous display.

[0059] The specific configuration of the annotation deviation coordinate determination module 50 will be described in detail below. The annotation deviation coordinate determination module 50 further includes: generating a deviation correction scheme based on the error measurement data, synchronizing the drawing annotation data and the deviation correction scheme in real time through the cloud server; at the same time, connecting to the cloud database to record the historical correction operation and timestamp of the annotation deviation; setting annotation permissions and annotation locking mechanisms based on the historical correction operations and timestamps in the cloud database, and the annotation permissions are used to support multi-terminal collaborative annotation.

[0060] The specific configuration of the annotation deviation coordinate determination module 50 will be described in detail below. The annotation deviation coordinate determination module 50 further includes: defining the annotation deviation vector D as the first drawing annotation set and the second drawing annotation set The difference between ,……, }, where the deviation of each annotation point is = - , and Mark the first drawing separately and the second drawing annotation set The coordinates of the jth annotation point in , where N is the total number of annotation points; based on the spatial alignment area, the weight of each annotation point is calculated ,in, Used to quantify the influence of different annotation points on drawing accuracy and assembly quality in overall deviation verification. is the distance between the jth annotation point and the boundary of the spatial alignment region.

[0061] The specific configuration of the marking deviation coordinate determination module 50 will be described in detail below. The marking deviation coordinate determination module 50 further includes: the calibration index calculation formula is E= ; The verification index E is used to quantify the degree of deviation between the two annotation sets. represents the Euclidean norm corresponding to the jth annotation point, is the weight of the jth annotation point.

[0062] A drawing annotation collaboration system for multiple mobile devices provided in an embodiment of the present invention can execute a drawing annotation collaboration method for multiple mobile devices provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0063] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules may be used and run on the user terminal and / or server, and the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0064] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A collaborative method for drawing annotation of multiple mobile devices, characterized in that: The method comprises: Connect multiple mobile devices to obtain multiple drawing files, including point cloud data, two-dimensional plane drawings, and environmental perception data; Reconstructing three-dimensionally respectively the two-dimensional plane drawings and the environmental perception data, the point cloud data and the environmental perception data in the plurality of drawing files, and setting a first basic twin model and a first drawing annotation set, a second basic twin model and a second drawing annotation set; According to the first drawing annotation set and the second drawing annotation set, spatially align the first basic twin model and the second basic twin model, and mark the spatial alignment area; At the same time, based on the spatial alignment area, error measurement data is collected in a filling measurement manner, and the error measurement data includes geometric error data and assembly error data; Based on the error measurement data, the first drawing annotation set and the second drawing annotation set are verified to determine the annotation deviation coordinates, and the drawing annotation collaborative management is performed through the multi-terminal mobile device.

2. A collaborative drawing annotation method for multiple mobile devices as claimed in claim 1, characterized in that: Performing three-dimensional reconstruction through two-dimensional plane drawings in the plurality of drawing files and environmental perception data, setting a first basic twin model and a first drawing annotation set, the method comprising: Performing vectorization processing on the two-dimensional plane drawings in the plurality of drawing files to extract drawing structural information; Based on the drawing structured information, a first unidirectional feature queue is set; Feature matching is performed according to the first unidirectional feature queue, and multi-view geometric reconstruction is performed to set a first base model architecture.

3. A collaborative drawing annotation method for multiple mobile devices as claimed in claim 2, characterized in that: The method further comprises: De-noising the environmental perception data in the plurality of drawing files, and then obtaining environmental structured information through scale-invariant feature transformation; Based on the environmental structured information, setting a second unidirectional feature queue; Perform feature matching according to the second unidirectional feature queue, perform multi-view geometry reconstruction, and set a second base model architecture; Among them, the first base model architecture and the second base model architecture are pre-trained models of the first basic twin model.

4. The method for collaboratively annotating drawings of multiple mobile devices as claimed in claim 2, characterized in that: Reconstructing three-dimensionally respectively the point cloud data and the environment perception data in the plurality of drawing files, setting a second basic twin model and a second drawing annotation set, the method further comprising: Perform multi-frame registration processing on the point cloud data in the plurality of drawing files, and configure initialization registration parameters and an initial transformation matrix; Based on the initial registration parameters and the initial transformation matrix, point cloud structured information is extracted, and a third unidirectional feature queue is set.

5. A collaborative drawing annotation method for multiple mobile devices as claimed in claim 4, characterized in that: Perform feature matching using the third unidirectional feature queue and the second unidirectional feature queue to locate feature matching points; Based on the third base model architecture corresponding to the third unidirectional feature queue and the second base model architecture corresponding to the second unidirectional feature queue, combined with the distance difference and angle difference between feature matching points, a second drawing annotation set is determined.

6. A collaborative drawing annotation method for multiple mobile devices as claimed in claim 5, characterized in that: The assembly error data includes gap deviation data and misalignment deviation data; The assembly error data is stored in a cloud database, and is displayed synchronously on multiple terminals using the multiple mobile devices.

7. A collaborative drawing annotation method for multiple mobile devices as claimed in claim 6, characterized in that: The method further comprises: performing collaborative management of drawing annotations through the multi-terminal mobile devices. Based on the error measurement data, a deviation correction plan is generated, and the drawing annotation data and the deviation correction plan are synchronized in real time through a cloud server; At the same time, the cloud database is connected to record the historical correction operations and timestamps of the marked deviations; Based on the historical correction operations and timestamps in the cloud database, annotation permissions and annotation locking mechanisms are set, and the annotation permissions are used to support multi-terminal collaborative annotation.

8. The method for collaboratively annotating drawings of multiple mobile devices as claimed in claim 1, characterized in that: Based on the error measurement data, the first drawing annotation set and the second drawing annotation set are verified to determine the annotation deviation coordinates, the method comprising: Define the annotation deviation vector D as the first drawing annotation set and the second drawing annotation set The difference between ,……, }, where the deviation of each annotation point is = - , and Mark the first drawing separately and the second drawing annotation set The coordinates of the jth annotation point in , N is the total number of annotation points; Based on the spatial alignment area, the weight of each annotation point is calculated ,in, Used to quantify the influence of different annotation points on drawing accuracy and assembly quality in overall deviation verification. is the distance between the jth annotation point and the boundary of the spatial alignment region.

9. A collaborative drawing annotation method for multiple mobile devices as claimed in claim 8, characterized in that: The calculation formula for the calibration index is E= ; The verification index E is used to quantify the degree of deviation between two annotation sets. represents the Euclidean norm corresponding to the jth annotation point, is the weight of the jth annotation point.

10. A collaborative drawing annotation system for multiple mobile devices, characterized in that: The system is used to implement a collaborative drawing annotation method for multiple mobile devices according to any one of claims 1 to 9, and the system includes: A drawing file acquisition module is used to connect multiple mobile devices to acquire multiple drawing files, wherein the multiple drawing files include point cloud data, two-dimensional plane drawings, and environmental perception data; A three-dimensional reconstruction module, used to perform three-dimensional reconstruction respectively through the two-dimensional plane drawings and environmental perception data, point cloud data and environmental perception data in the multiple drawing files, and set a first basic twin model and a first drawing annotation set, a second basic twin model and a second drawing annotation set; A spatial alignment module, configured to spatially align the first basic twin model and the second basic twin model according to the first drawing annotation set and the second drawing annotation set, and mark a spatial alignment area; An error measurement data acquisition module is used to simultaneously, based on the spatial alignment area, acquire error measurement data in a filling measurement manner, wherein the error measurement data includes geometric error data and assembly error data; The annotation deviation coordinate determination module is used to verify the first drawing annotation set and the second drawing annotation set based on the error measurement data, determine the annotation deviation coordinates, and perform collaborative management of drawing annotations through the multi-terminal mobile device.

Images