Cargo packaging and labeling remote verification device based on VR interaction

Abstract

The present invention discloses a device for remotely verifying cargo packaging and labeling based on VR interaction, relating to the field of virtual reality technology. The device comprises: a three-dimensional environment construction unit, a VR device, a permeation rotation link fusion unit, and a result generation unit. The three-dimensional environment construction unit is used to generate a three-dimensional interactive model containing a complete set of normal lines and establish a unified coordinate reference space within the three-dimensional interactive model. The VR device is used to receive the three-dimensional interactive model and then package it into a section to be verified. The permeation rotation link fusion unit is used to arrange a permeation probe grid for each section to be verified and record a list of slip paths on the surface of the three-dimensional interactive model in a spiral propulsion manner. The result generation unit is used to complete remote verification of the target package. The present invention effectively improves the spatial accuracy, interaction efficiency, and judgment reliability of remote packaging quality verification.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to a device for remotely verifying cargo packaging and identification based on VR interaction. Background Art

[0002] In the current logistics and manufacturing industries, quality verification of cargo packaging and labeling has become a crucial component in ensuring product delivery integrity, symbol compliance, and anti-counterfeiting and traceability. This is particularly true in cross-regional, multi-link supply chains, where traditional quality control models relying on manual visual inspection and spot checks are no longer sufficient to meet the demands of digital, standardized, and high-throughput quality regulation. Consequently, technologies such as 3D visual perception, virtual reality, robotics, and automated testing are being increasingly introduced in the field of cargo packaging quality verification, both domestically and internationally, in an effort to effectively integrate information perception and remote collaboration.

[0003] Most existing 3D vision inspection systems use monocular or binocular cameras in conjunction with structured light or laser scanners to capture and reconstruct point clouds of the packaging's exterior surface. Some systems incorporate industrial robots or automated rotating platforms to capture the packaging from multiple angles. After acquiring sufficient visual information, they generate a preliminary 3D model using a depth recovery algorithm based on image matching or a 3D reconstruction method based on laser echo intensity. These systems then compare the 3D model with a CAD template or pre-qualified samples to identify surface defects or structural deviations. Some systems also incorporate machine learning methods, training classifiers to automatically identify common surface defects such as scratches, dents, and cracks. While this technical approach is theoretically feasible, it faces numerous key limitations in practical industrial deployment.

[0004] First, most existing technologies use a frame-by-frame stitching method to process point clouds during the 3D reconstruction stage, which is easily affected by small posture disturbances, lighting changes, and inconsistent surface reflectivity during the image acquisition cycle. Because these systems lack the ability to deeply integrate with inertial measurement units (IMUs), they rely solely on feature matching between image sequences to estimate relative pose. This can lead to feature loss, mismatches, and even drift accumulation in areas with sparse textures or repeated textures, thus affecting the overall accuracy and consistency of the model. Although some systems using structured light have improved reconstruction resolution, they have poor adaptability to the complexity of packaging shapes and reflective conditions. This is especially true for packaging materials with metallic reflective coatings or transparent film layers. The projection of laser and structured light often leads to depth distortion, which in turn misleads the subsequent defect determination process. Summary of the Invention

[0005] The purpose of the present invention is to provide a remote verification device for cargo packaging and labeling based on VR interaction, which uses binocular vision and inertial measurement components to synchronously obtain the surface texture and posture data of the packaging body, construct a three-dimensional interactive model under a unified coordinate system, and realize immersive interactive selection of suspicious areas by remote inspectors through VR virtual reality devices. The system further arranges a penetration probe grid in the selected area, collects normal vector information along the spiral sliding of the surface, generates sliding path records and constructs a state matrix to achieve quantitative evaluation of local roughness. Finally, unqualified sections are formed through abnormal superposition and clustering, and a defect list is automatically output and highlighted in the model. The invention effectively improves the spatial accuracy, interaction efficiency and judgment reliability of remote packaging quality verification, and has the practical application advantages of traceability, structurability and scalability.

[0006] To address the above technical problems, the present invention provides a device for remote verification of cargo packaging and labeling based on VR interaction. The device comprises: a three-dimensional environment construction unit, a VR device, a penetration, rotation, link, and fusion unit, and a result generation unit. The three-dimensional environment construction unit comprises a binocular vision acquisition component and an inertial measurement component, which are used to synchronously acquire surface texture data and pose data of a target package, generate a three-dimensional interactive model containing a complete set of normals, and establish a unified coordinate reference space within the three-dimensional interactive model. The VR device is used to receive the three-dimensional interactive model and present it to a remote inspector. The remote inspector uses the VR device to select suspicious locations in the three-dimensional interactive model and then packages them into a section to be verified. The penetration, rotation, link, and fusion unit is used to arrange a grid of penetration probes for each section to be verified and record a list of slip paths on the surface of the three-dimensional interactive model in a spiral propulsion manner. The result generation unit is used to construct a state matrix based on the slip path list, generate unqualified section entries using a cascaded anomaly overlap cataloger, automatically output a list of unqualified items, and highlight unqualified marks in the VR device, thereby completing remote verification of the target package.

[0007] Furthermore, the binocular vision acquisition component uses a global shutter method to simultaneously expose the left-view image frame and the right-view image frame within a single scanning cycle, and outputs surface texture data of the target package body containing original brightness information; the inertial measurement component outputs three-axis angular velocity and three-axis acceleration at a beat exactly the same as the single scanning cycle, and obtains corresponding posture data by first-order integration of the three-axis angular velocity and second-order integration of the three-axis acceleration.

[0008] Furthermore, the three-dimensional environment construction unit performs geometric correction on the left-view image frame and the right-view image frame according to the posture data; calculates the disparity value of the corrected left-view image frame and the right-view image frame pixel by pixel, adopts a segmented weighted window search to avoid error accumulation in the low-texture area, and maps the obtained disparity value to an initial depth map; according to the initial depth map and the posture data, the depth value is solved into the initial point cloud coordinate according to the three-dimensional imaging geometric relationship, and the initial point cloud coordinates of multiple consecutive frames are spatially aligned and redundantly eliminated in the coordinate system to generate dense point cloud data; performs hierarchical voxel grid filtering on the dense point cloud data, divides the dense point cloud data into multiple voxel grid levels, and performs iterative averaging operations from coarse to fine between each level to eliminate random noise and fill point cloud holes, and outputs a smooth point cloud volume; applies a triangle partitioning strategy to the smooth point cloud volume, generates a vertex set, an edge set and a face set triplet while maintaining topological continuity, and obtains a three-dimensional interaction model.

[0009] Furthermore, the three-dimensional environment construction unit performs the following operations on each triangular face of the three-dimensional interactive model: calculates the area vector of the triangular face; normalizes the area vector to a unit normal vector; adds the unit normal vector to the normal vector accumulation pool of the corresponding mesh vertex; after all triangular faces are traversed, averages and normalizes the normal vector accumulation pool in each mesh vertex to obtain the final unit normal vector of the mesh vertex; and records the final unit normal vector set of all mesh vertices as a complete normal line set.

[0010] Furthermore, when the remote inspector selects a suspicious location in the VR virtual reality device through rays or gestures, the VR virtual reality device obtains the three-dimensional coordinates of the suspicious location and the posture number at the selected moment, and selects the mesh vertex connected to the seed coordinates from the complete normal set of the three-dimensional interactive model. If the angle between the unit normal vector of the adjacent mesh vertex and the unit normal vector of the vertex where the seed coordinate is located is less than a preset threshold, the adjacent mesh vertex is added to the extended candidate set; otherwise, it is removed; the recursive search in the extended candidate set is continued until no new vertices that meet the threshold condition appear, and a normal consistency candidate set is obtained; the outer bounding box of the normal consistency candidate set is calculated and a continuous face patch is generated; a unique to-be-checked segment identification number and a vertex set are assigned to the continuous face patch, and then it is packaged into a to-be-checked segment.

[0011] Furthermore, the penetration rotation link fusion unit retrieves the compliance template identifier from the enterprise database, arranges the penetration probe grid for each section to be checked in the unified coordinate reference space, and the penetration probe grid slides along the surface of the three-dimensional interactive model in a spiral propulsion manner, and records the sliding path in real time; reads the vertex set for each section to be checked, and calculates the outer bounding box; takes the geometric center of the outer bounding box as the starting anchor point, and selects the unit normal vector of the anchor point in the complete normal set as the local normal vector; generates a square grid unit in the plane perpendicular to the local normal vector, and the side length of the grid unit is equal to one tenth of the shortest side of the outer bounding box; takes the grid unit as the Based on the foundation, copy outward in a circular manner to form concentric rings, and copy three times in total to obtain a five-by-five penetration probe grid; for each grid node of the penetration probe grid, the grid node is located at the vertex position of all grid units of the penetration probe grid, and the complete normal set is called to query the unit normal vector of the nearest vertex of the corresponding grid node; if the angle between the queried unit normal vector and the local normal vector is less than thirty degrees, the grid node is kept in place; if the angle is not less than thirty degrees, the grid node is adjusted along the direction of the unit normal vector so that the difference in side length between it and the adjacent grid node does not exceed ten percent; after completion, the three-dimensional coordinates of the current grid node set are saved as the aligned node coordinate set.

[0012] Furthermore, the penetration rotation link fusion unit takes the starting anchor point as the pole, plans an Archimedean spiral in the aligned node coordinate set, and the spiral step length is equal to the grid unit side length; the central node of the penetration probe grid is gradually moved along the spiral path, and each time the central node moves one step, the entire penetration probe grid slides in a synchronous translation manner; for each slip, the following information is recorded to obtain a slip path record: timestamp of the slip moment; three-dimensional coordinates of the central node; the number of grid nodes covered by the penetration probe grid; the average value of the angles between all unit normal vectors and local normal vectors in the coverage area; when the spiral radius is greater than half of the longest side of the outer bounding box, or the number of slip steps reaches fifty, the spiral advancement is terminated; all slip path records are formed into a slip path list in chronological order.

[0013] Furthermore, the result generation unit traverses the slip path list, and for each slip path record, randomly selects grid vertices that are no less than 10% of the number of grid nodes in the slip path record in the complete normal set as sampling grid vertices; calculates the average angle between these sampling grid vertices and the unit normal vectors of their adjacent vertices to obtain the baseline roughness; multiplies the baseline roughness by a coefficient of 1.5 to set it as the unqualified threshold, and multiplies it by a coefficient of 1.2 to set it as the review threshold; creates a matrix with the number of rows equal to the total number of path numbers and the number of columns fixed to 4, and writes the following content row by row to obtain a state matrix: column 1: path number, column 2: number of covered grid nodes, column 3: average angle, and column 4: state mark; if the average angle is greater than the unqualified threshold, the state mark is set to 3; if the review threshold is less than the average angle ≤ the unqualified threshold, the state mark is set to 2; if the average angle ≤ the review threshold, the state mark is set to 1.

[0014] Furthermore, the cascaded exception overlap cataloger scans the rows of status mark 3 in the status matrix; based on the unified coordinate reference space coordinates, the path numbers with spatial proximity ≤ 0.2 times the longest side of the outer bounding box are aggregated into a single exception cluster; an unqualified segment entry is generated for each exception cluster; all unqualified segment entries are arranged in ascending order according to the unqualified segment identification number, and a list of unqualified items is automatically output, and the list of unqualified items is synchronously written into the enterprise database for traceability; the outer bounding box of the unqualified segment entry is mapped back to the three-dimensional interactive model, and the unqualified mark is highlighted in the VR virtual reality device, thereby completing the remote verification of the target package body.

[0015] The VR-interactive cargo packaging and labeling remote verification device of the present invention has the following beneficial effects:

[0016] By combining the three-dimensional environment construction unit with the binocular vision acquisition component and the inertial measurement component, the synchronous acquisition of images and postures can be achieved within a single scanning cycle, ensuring that the three-dimensional interactive model has high reliability in terms of geometric accuracy and coordinate consistency.

[0017] Secondly, the introduction of VR virtual reality devices enables remote inspectors to complete three-dimensional observation and interactive operations of the packaging body in an immersive scene, and achieve accurate and efficient selection of suspicious areas, thus breaking through the limitations of traditional detection methods on spatial perception and operation dimensions.

[0018] Furthermore, the present invention constructs a mesh of penetrating probes by linking and fusing units through penetration rotation. This mesh then slides in a spiral pattern along the surface of a 3D model, enabling high-density, continuous recording of surface information and effectively detecting local roughness and abnormal deformation. This process, combined with analysis of the unit normal vector field and the average angle of the sliding path, ensures that every potential defect has complete and quantifiable data support.

[0019] Finally, the result generation unit works in conjunction with the cascading anomaly overlay cataloger to automatically identify, number, classify, and highlight unqualified sections, while synchronously writing the verification results into the database to support subsequent traceability and batch management.

[0020] Overall, the present invention opens up a complete inspection process from spatial data acquisition, virtual interactive operation, regional-level probe slippage, roughness assessment to structured result output, significantly improving the intelligence level, inspection coverage density and decision reliability of remote quality verification, and is particularly suitable for multi-location, multi-batch, high-throughput industrial packaging inspection application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of the structure of a device for remotely verifying cargo packaging and identification based on VR interaction provided by an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of the operation of a remote inspector in a remote verification device for cargo packaging and labeling based on VR interaction provided by an embodiment of the present invention;

[0023] Figure 3 Schematic diagram of state matrix construction and unqualified section identification in a remote verification device for cargo packaging and labeling based on VR interaction provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0025] refer to Figure 1A device for remotely verifying cargo packaging and labeling based on VR interaction is disclosed. The device comprises: a three-dimensional environment construction unit, a VR device, a penetration rotation link fusion unit, and a result generation unit. The three-dimensional environment construction unit comprises a binocular vision acquisition component and an inertial measurement component, which are used to synchronously acquire surface texture data and posture data of a target package, generate a three-dimensional interactive model containing a complete set of normals, and establish a unified coordinate reference space within the three-dimensional interactive model. The VR device is used to receive the three-dimensional interactive model and display it to a remote inspector. The remote inspector uses the VR device to select suspicious locations in the three-dimensional interactive model and then packages them into a section to be verified. The penetration rotation link fusion unit is used to arrange a penetration probe grid for each section to be verified and record a list of slip paths on the surface of the three-dimensional interactive model in a spiral propulsion manner. The result generation unit is used to construct a state matrix based on the slip path list, generate unqualified section entries using a cascaded anomaly overlap cataloger, automatically output a list of unqualified items, and highlight unqualified marks in the VR device, thereby completing remote verification of the target package.

[0026] Specifically, the 3D environment construction unit synchronously maps the target packaging's real-world geometry and surface texture into digital space, instantly generating a unified coordinate reference space. To achieve this, the binocular vision acquisition component captures left and right view image frames simultaneously, while the inertial measurement component outputs angular velocity and acceleration at a completely synchronized pace. The system integrates this inertial information into a pose, which describes the camera's true pose at the moment of exposure for each frame. The pose links image pixels to world coordinates, allowing the two images to be geometrically aligned. The system then uses the parallax method to convert the corresponding pixel differences between the left and right images into a depth estimate, providing distance information along the spatial coordinate axes for each pixel. The depth map is then back-calculated into an initial point cloud, and the multiple point clouds are spatially aligned after pose correction. Redundant points are removed, random noise is smoothed using voxel grid filtering, and point cloud holes are filled, ultimately resulting in a smooth point cloud with continuous structure and stable normal vectors. The system uses a triangulation strategy on this point cloud, transforming the point set into a triplet of vertex, edge, and face sets. The area vectors are normalized and added to the mesh vertex normal pool, and a global average is performed to obtain the complete normal set. The significance of a unified coordinate reference space is that all subsequent interactions, detection, and analysis can be described using the same set of coordinates, reducing error propagation.

[0027] The VR device serves as a bridge between humans and machines. Within this device's technical approach, it not only renders the 3D interactive model but, more importantly, provides an intuitive 3D pointing method. The remote inspector sees a 3D interactive model that matches the size and texture of the actual package. When the inspector selects a seemingly unusual spot on the model's surface using a ray or hand gesture, the VR device immediately reads the 3D coordinates and current pose number of that point and submits them as seed coordinates to the backend algorithm. This is where the complete normal set comes into play: the algorithm compares the angle between the unit normal vector of the vertex at the seed coordinate and the unit normal vectors of adjacent vertices. If the angle does not exceed a preset limit, the adjacent vertices are added to the expanded candidate set, and the search is recursively repeated within the candidate set until convergence. The resulting patch is not only continuous but also curvature-consistent. The system then calculates an outer bounding box, generating a continuous patch and assigning a unique identification number to the segment to be verified. The principle behind this process is "normal vector consistency infers similar surfaces," transforming 3D curvature continuity, difficult for the human eye to directly determine, into a programmable threshold judgment, automatically outlining suspicious areas.

[0028] The penetration rotation link fusion unit combines probe grid placement with adaptive surface alignment. The system first retrieves compliance template identifiers from the enterprise database for reference during subsequent verification. Next, a penetration probe grid is placed for each section to be verified in a unified coordinate reference space. The placement process begins with the geometric center of the bounding box as the anchor point. The unit normal of the vertex corresponding to this anchor point is used as the local normal. Square grid cells are then generated in a plane perpendicular to the local normal. These grid cells are replicated in a circular pattern to form a five-by-five probe distribution. The grid node positions are fine-tuned based on the direction of the complete normal set feedback to ensure that the probes are as parallel as possible to the surface normal and that the edge lengths between nodes are as consistent as possible. The probe grid's construction relies on the "local plane assumption": at small scales, a surface can be approximated as a plane. Therefore, the probe grid only needs to be arranged in the plane perpendicular to the normal vector to maximize local surface coverage. This fine-tuning mechanism further ensures that the probe nodes follow the actual surface details, thereby improving subsequent measurement accuracy. Once the probe grid is aligned, the system plans an Archimedean spiral trajectory around the anchor point. The spiral step size is set to be equal to the side length of the grid unit, so that the probe can cover the surface area as evenly as possible when sliding. When the center node moves one step along the spiral trajectory, the entire probe grid will be translated one step synchronously; each translation records the timestamp, center node coordinates, number of covered nodes, and the average value of the angle between all unit normal vectors and local normal vectors within the coverage range. The core principle of recording is to use "local normal vector perturbation" to infer the surface roughness: the larger the average angle, the more drastic the change in the surface normal vector in this area, that is, the rougher the surface, the higher the probability of potential defects; conversely, the surface is approximately flat. When the spiral radius grows to a certain range or the number of steps reaches the upper limit, the advancement stops, and all records are sorted by time to form a list of sliding paths.

[0029] The result generation unit receives the slip path list and begins the statistical inference process. First, a random sample is taken from the complete normal set, selecting vertices where each record covers at least one-tenth of the number of nodes. The average angle between these vertices and their neighboring vertices is calculated to obtain the baseline roughness. The baseline roughness is then amplified to a rejection threshold and a recheck threshold, which together classify the average angle. Random sampling is used to reduce the risk of misclassification due to extreme local texture. Using mean curvature as a roughness measure, "normal vector consistency" directly reflects the surface flatness. The system then creates a state matrix, with each row corresponding to a slip path record, containing the path number, number of nodes covered, average angle, and a state flag. The state flag is essentially a multi-threshold classification result: an average angle above the rejection threshold is assigned the highest state, while an angle below the recheck threshold is assigned the lowest state. The intermediate range is assigned the recheck state, facilitating further manual verification. The cascaded anomaly superposition cataloger scans the rows with high state flags in the state matrix and groups spatially adjacent path numbers into anomaly clusters based on coordinates in a unified coordinate reference space. The principle of this merging operation is the "spatial locality assumption": abnormal records caused by the same defect tend to cluster in locations close to each other in physical space; therefore, as long as the central node positions corresponding to the path numbers are close enough in space, they can be considered to originate from the same defect. After the aggregation is completed, the system generates an unqualified segment entry for each abnormal cluster, and outputs all entries into a list of unqualified items in ascending order of identification numbers, and simultaneously writes them into the enterprise database. The database writing action ensures a complete closed loop for subsequent traceability and quality statistics, and also allows the system to be integrated into the company's existing quality management process. The last step is to map the outsourced boundary box of the unqualified segment back to the three-dimensional interactive model and highlight it in the VR virtual reality device. The highlighted display uses a unified coordinate reference space to achieve a one-to-one correspondence between the interface layer and the data layer. The inspector can see the bright frame line covering the defective area when he looks up, realizing intuitive and easy-to-understand remote verification.

[0030] Furthermore, the binocular vision acquisition component and the inertial measurement unit are forcibly bound to the same "single scan cycle," which can be understood as the minimum time slice required for the system to acquire a complete set of raw data and immediately submit it to the reconstruction algorithm for processing. To eliminate the potential damage to stereo matching accuracy caused by relative motion and illumination fluctuations within the same cycle, the binocular vision acquisition component uses a global shutter method, exposing the left and right image frames simultaneously at the same microsecond time point. The advantages of a global shutter method are twofold: First, there is no row-level time difference between the left and right image frames. Therefore, even if the target package vibrates slightly at the moment of exposure or the conveyor belt experiences momentary displacement, there is no temporal shift within the frame; all pixels correspond to the same real-time scene. Second, the diffuse brightness of the measured surface is instantaneously "frozen," and the brightness distributions of the two frames are naturally conformal to each other. Subsequent parallax calculations can be strictly aligned at the sub-pixel level without the need for additional temporal compensation or optical flow correction. Looking at the entire 3D reconstruction chain, this operation directly improves the robustness of depth estimation, because the disparity algorithm relies on the assumption of brightness consistency in areas with mixed high-frequency textures and low-textures. If there is a slight misalignment in the sampling moment, the assumption will be destroyed, resulting in amplified depth variance.

[0031] Simultaneously, the inertial measurement unit (IMU) outputs triaxial angular velocity and acceleration at a rhythm perfectly aligned with this scan cycle. "Perfectly aligned" here refers not only to matching average output frequencies but also to clock synchronization of external hardware trigger lines and even the internal FIFO buffer, ensuring that each IMU frame is timestamped with the same timestamp as the camera exposure marker. The system first performs a first-order integration of the triaxial angular velocity to obtain the rotation vector. It then subtracts gravity from the triaxial acceleration and performs a second-order integration to obtain the displacement vector. These two are then combined to form the pose data. Because the camera optical center and the IMU center of mass are extrinsically calibrated during assembly, the pose data resides directly in a unified coordinate reference space, seamlessly driving forward and reverse reprojection. More importantly, the IMU integrated output trajectory provides a secure initial value for subsequent dense point cloud registration. The registration algorithm only requires sub-pixel local optimization to achieve spatial alignment of multiple point clouds, avoiding local extrema caused by random drift.

[0032] Furthermore, after receiving the left-view and right-view image frames and their corresponding pose data, the 3D environment construction unit first performs geometric correction on the two image frames using the rotation and translation information contained in the pose data. During the correction process, the camera's intrinsic calibration matrix and extrinsic pose matrix work together to remap the light corresponding to each pixel back to a unified coordinate reference space, thereby eliminating the aberrations and projection distortions caused by lens distortion and pose changes. After completing the geometric correction, the system performs pixel-by-pixel disparity calculations for the left-view and right-view image frames at the pixel level. The core algorithm uses a segmented weighted window search within the sliding matching window: in texture-rich areas, each pixel in the window is assigned a uniform weight, making full use of grayscale gradients to maintain depth sensitivity; in low-texture or repetitive texture areas, the window is divided into several subintervals, and the weight of each subinterval is dynamically adjusted based on the local gradient and brightness change level, using weight differences to suppress the accumulation of false matches caused by texture loss. After the disparity calculation is completed, the disparity value is mapped into an initial depth map through imaging geometry. The greater the depth, the farther the observation point is from the camera. Combining the camera posture and projection center in the pose data, the system resolves each depth value into a three-dimensional coordinate in a unified coordinate reference space, thereby obtaining the initial point cloud coordinates.

[0033] The initial point cloud coordinates are continuously generated across multiple scanning cycles as the conveyor line moves. The system uses the pose information of each frame to spatially align multiple point clouds, allowing the same spatial physical points to aggregate in the coordinate system. Next, overlapping and redundant points are removed by statistically analyzing local density and normal vector consistency, retaining samples with the most significant changes in surface features to compress the data size, while still preserving details with high curvature, thereby forming dense point cloud data. The dense point cloud data then enters a hierarchical voxel grid filtering process: the system constructs a multi-level voxel hierarchy from large to small according to the voxel grid size, performs mean or median iterative averaging on the point set within each level of voxel grid, and propagates the results to the next layer of finer voxel grids. This coarse-to-fine iterative averaging quickly balances random noise with large voxel grids, while compensating for holes in low sampling density at the fine voxel grid level, ultimately deriving a smooth point cloud volume. After obtaining a smooth point cloud, the system uses a triangulation strategy to perform surface reconstruction while maintaining topological continuity: the algorithm is based on the local Delaunay principle, and searches for the optimal three-point group in the point cloud to establish triangulated facets, which are gradually expanded to the global level. All generated triangulated facets share a vertex set and an edge set to ensure geometric consistency between facets on common edges. If the difference in normal vectors of adjacent facets exceeds a set threshold, the system inserts additional vertices to refine the mesh, thereby avoiding sharp folding. The final output of the vertex set, edge set, and face set triplet constitutes a three-dimensional interactive model. The mesh resolution of this model maintains the same depth as the original dense point cloud, but the surface noise is significantly suppressed, and the topological structure is continuous and closed, without self-intersection or cracks. At the same time, the triangulation strategy naturally maintains the consistency of the normal vector, which facilitates the subsequent precise mounting of the penetration probe mesh in the penetration rotation link fusion unit.

[0034] Furthermore, in the mesh structure of the three-dimensional interactive model, each triangular face is composed of three vertices in counterclockwise order. The system first obtains two edge vectors using the coordinate difference of these three vertices, and performs a vector cross product on the two to obtain an area vector that contains both direction and area size. The modulus of the area vector is proportional to the actual area of ​​the facet, and its direction is the same as the facet normal vector. Therefore, when the system normalizes the area vector to a unit normal vector, it obtains a facet normal vector that purely describes the direction and has a constant length of one. It can be used for continuous surface curvature inference and to avoid unnecessary bias in subsequent statistics caused by the area size. Subsequently, the system uses the facet vertex as an index and adds the unit normal vector just obtained to the normal vector accumulation pool corresponding to the three vertices respectively; the core idea of ​​this is to use the facet normal vector to collectively reflect the average direction of the small surfaces around the vertex.

[0035] After all the faces have been traversed, the normal vector accumulation pool of a certain vertex will save the directional contributions of all the adjacent faces of the vertex. The system then averages and normalizes the vectors in the pool to obtain the final unit normal vector of the vertex. This final unit normal vector not only provides the best approximation of curvature continuity at the vertex, but also balances the impact of unequal area faces on directional statistics. Therefore, it is more stable than simply taking the arbitrary normal vectors of adjacent faces. After completing the same operation for all vertices in the model, the system records the final unit normal vector set of all vertices as a complete normal set. This set is the core geometric feature library that subsequent algorithms rely on: for example, the steps of performing normal vector angle threshold expansion in VR virtual reality devices, calculating local normal vectors in the penetration rotation link fusion unit, and evaluating surface roughness in the result generation unit all directly reference the data in the complete normal set, thereby ensuring the consistency and traceability of the entire remote verification process at the normal vector level.

[0036] Furthermore, when a remote inspector uses a raycast or gesture to touch a model surface within a VR device, the system immediately maps this interaction event to the specific mesh topology within the 3D interactive model. First, through optical tracking and pose synchronization, the 3D position of the "seed coordinates" and the corresponding pose number at the triggering moment are calculated, ensuring that the interaction is precisely locked into a unified coordinate reference space. The system then searches the complete set of normals for the mesh vertex closest to the seed coordinates and uses this vertex as the starting point for region growing. The core logic behind this is recursive expansion based on the normal vector angle: the system visits the neighboring vertices of the seed vertex layer by layer along the mesh topology. For each neighboring vertex, the angle between its unit normal vector and the starting point's unit normal vector is calculated. If the angle is less than a preset threshold, the local orientation of the surfaces on which the two vertex are located is considered to be converging. The vertex is then added to the expanded candidate set and recursively tested as a new inspection source. If the angle does not meet the threshold, it indicates that the local geometry of the surface on which the vertex is located has changed significantly, and the algorithm removes it from the expanded candidate set. As the recursion progresses, the expanded candidate set gradually grows into a cluster of vertices that are highly continuous in terms of curvature. The system names this cluster the "normal-consistent candidate set." Because the algorithm consistently relies on the topological adjacency of the mesh to search, it inherently ensures that the vertices within the set are geometrically connected, avoiding the mis-aggregation of spatially dispersed points with similar orientations.

[0037] When no new qualified vertices are found during the recursion, the normal consistency candidate set is fully converged. The system then calculates an outer bounding box for the set, describing the minimum footprint of the surface in 3D coordinates. The outer bounding box not only provides size and position data but also clarifies the orientation of the set in space, which is crucial for subsequent mesh alignment and penetration probe placement. The algorithm then regenerates a topologically continuous mesh segment based on the triangles contained in the set, called a continuous patch. This reconstruction ensures that all patches are completely closed in terms of edge sharing, facilitating consistent segment operations in subsequent steps. After patch reconstruction, the system assigns a unique segment identification number to be verified and packages the topologically continuous vertex set, the outer bounding box, and the identification number to form a segment object to be verified. This unique identification number enables a one-to-one mapping between the database, interaction, and analysis layers throughout the remote verification process, facilitating the loading of probe meshes in the subsequent penetration, rotation, link, and fusion units, and facilitating rapid location and archiving of the result generation unit in the state matrix. In this way, the inspector's single click is quickly converted into a section to be checked with geometric completeness, topological connectivity and data traceability, laying a solid geometric foundation for subsequent in-depth detection and quality judgment.

[0038] Furthermore, the infiltration rotation link fusion unit calls the database interface to retrieve the compliance template identification for each section to be checked. The core meaning of this step is to load the qualified surface samples and identification information of the current product batch into a unified coordinate reference space. After the loading is completed, the unit performs a fast bounding volume analysis on the vertex set of the section to be checked, calculates the outer bounding box and extracts the geometric center. The center usually falls in the area of ​​smooth curvature, which can reduce the initial calibration error. The system retrieves the unit normal vector of the vertex corresponding to the geometric center in the complete normal set and defines it as the local normal vector. This vector determines the orientation of the local tangent plane and provides a normal vector orthogonal basis for the subsequent probe mesh generation. In the plane perpendicular to the local normal vector, the system establishes the first square grid unit based on one-tenth of the shortest side of the outer bounding box; the spatial sampling spacing of the three-dimensional reconstruction link at the probe level determines the Nyquist limit of the normal vector roughness statistics. If the spacing is too large, micro defects will be missed. If it is too small, excessive data will be generated in the slip phase and the state matrix write rate will be slowed down. The algorithm then performs concentric ring amplification, achieving three rings of replication, and ultimately forming a five-by-five grid of penetration probes; this format means that the number of probes just covers the local surface radius interval increasing from the center outward, making the sampling density proportional to the importance of the target area.

[0039] After mesh generation, the system enters the "node fitting iteration" phase: for each mesh node, the system calls the complete normal set to query the unit normal vector of the nearest vertex. If the angle between this vector and the local normal vector is less than 30 degrees, the node remains in place; otherwise, the system nudges it in the direction of the queried unit normal vector until the difference in side length with the adjacent node does not exceed 10%. The reason for setting 30 degrees as the threshold is that in the maximum angle design specifications for conventional packaging materials (cardboard, plastic film, composite coating), the bending angle of the surface generally does not exceed 30 degrees. Therefore, if the angle is greater than this threshold, it means that the node is crossing a sudden change in curvature and needs to be nudged to the actual corresponding point on the surface to prevent the probe mesh from being suspended or interpenetrating. As for the 10% side length difference limit, it is to maintain mesh isomorphism, keep the step size in the spiral advancement step locally constant, and avoid unstable data sampling rhythm due to sudden changes in mesh density.

[0040] The formula for adaptive projection of mesh nodes along the normal vector is:

[0041] ;

[0042] in, For nodes The initial 3D coordinates in the unified coordinate reference space. For nodes The adjusted 3D coordinates. For the node The unit normal vector (from the complete normal set) corresponding to the nearest mesh vertex. For the probe grid and the node The set of adjacent nodes that are directly connected by mesh edges. For adjacent nodes The initial three-dimensional coordinates of . For nodes Point to adjacent nodes The signed distance of the vector in the direction of the normal vector (positive value indicates the direction of the direction, negative values ​​indicate reverse direction). For nodes With node For each node , first make a weighted average of the inverse square of the normal distance of all adjacent nodes, and then calculate the distance along the local normal vector Translate the corresponding displacement to achieve fine alignment of the node with the surface normal field, and ensure that the edge length difference between adjacent nodes remains limited (the weight decays rapidly with increasing distance).

[0043] Nudges aren't performed all at once, but rather in a layered, iterative process. The system first adjusts nodes with significantly out-of-range angles at a coarse-scale level, then fine-tunes local geometric details at finer scales until the overall mesh is coherent and the edge length errors meet constraints. Once all nodes are adjusted, the 3D coordinates are solidified into the aligned node coordinate set, representing the zero-time state of the mesh. This set ensures optimal coupling between probe spacing, node topology, and surface normal distribution, providing a precise benchmark for the spiral propulsion phase. The unit then plans an Archimedean spiral trajectory within the aligned node coordinate set: the spiral equation has its vertex at the geometric center, and the step length is equal to the mesh cell edge length. This consistent step length ensures a mathematically linear relationship between probe coverage density and sampling steps, facilitating back-end writes to the state matrix at a constant frame rate. The system then moves the central node along the spiral trajectory. With each step, the remaining nodes follow suit with a rigid-body translation, resulting in a synchronized sliding of the penetration probe grid across the surface. The system records four core metrics in real time during the sliding process: first, the timestamp, which is used to maintain subsequent timing mapping with the enterprise's MES system; second, the 3D coordinates of the central node, which are used to plot the sliding path and trigger spatial proximity aggregation; third, the number of covered grid nodes, which measures the real-time contact area between the probe and the surface; and fourth, the average angle between all unit normal vectors and the local normal vectors in the covered area, which directly reflects the surface roughness. Each record is written to the memory queue and then stored in a compressed cache. The system uses sliding window differential encoding to reduce I / O overhead and ensure that even long sliding operations on high-resolution surfaces do not block the main thread. The spiral advances continuously until the spiral radius exceeds half the longest side of the outer bounding box or the number of sliding steps reaches fifty. The "fifty steps" is based on the measured spiral coverage area and roughness uniformity curve. When the number of steps exceeds fifty, the coverage rate tends to saturate and the gain diminishes. At the end of the sliding process, the cache is decompressed and sorted in chronological order to form a sliding path list.

[0044] Furthermore, after completing the node alignment of the penetration probe grid, the penetration rotation link fusion unit uses the side length of the grid unit as the step length of the spiral advancement with the starting anchor point position as the center, so as to ensure that each sliding step can accurately cover adjacent but non-overlapping local areas. The Archimedean spiral trajectory is essentially a path that expands at constant linear intervals in a two-dimensional plane. Here, its advantage is that it can effectively control the probe movement density and avoid excessive repeated sampling of local areas, and ensure that the spiral radius grows steadily to cover the surface details of the section to be checked to the greatest extent. The smoothness of this trajectory and the consistency of the step length make the path planning of the penetration probe grid movement easier to implement and more in line with the requirements of spatial coverage uniformity.

[0045] The angle recursion formula for the Archimedean spiral step length to satisfy the fixed arc length constraint is:

[0046] ;

[0047] in, is the radial coefficient of the Archimedean spiral, satisfying the polar coordinate relationship , which is equivalent to the "radius increment per unit radian"; its value is set at mesh initialization as a constant multiple of the side length of a single mesh element and is independent of the surface. The predetermined spatial arc length step between two sliding passes of the penetration probe grid is equal to the side length of the grid cell to ensure a constant coverage density. The spiral path Polar angle at the end of the secondary slip (unit: radians). To satisfy the arc length constraint The next sliding target polar angle must be numerically solved by the above formula before it can be used for position update. The differential arc length of the Archimedean spiral in the local tangent plane is . Change the arc length from Points to And let the result be equal to the fixed step size , we can get the above formula. Solve Afterwards, Write the slip path record, where is the starting anchor point, are orthogonal basis vectors in the tangent plane, .

[0048] After determining the spiral trajectory, the system begins to perform the actual sliding action of the penetration probe grid. The specific implementation process of each slide includes the following key steps: First, the unit uses the current step size of the spiral path as a reference to push the center node of the penetration probe grid forward along the path; then, all nodes within the probe grid also move in a rigid body synchronous translation manner, keeping the relative positions of each node strictly fixed. This strict synchronous translation is the key to achieving data consistency. It ensures that the distance between nodes within the probe grid remains unchanged, so that the local surface information of each measurement is perfectly aligned with the results of the previous step in terms of spatial layout. In addition, the synchronous translation ensures the unity between the time dimension and the spatial dimension of the data, thereby avoiding spatial overlap or data omission.

[0049] After each probe grid slide, the system immediately records a detailed set of sliding path information for subsequent analysis and evaluation. This sliding path information specifically includes the following four core elements: The first core element is the timestamp of the sliding moment, which records the exact time of each sliding action. This timestamp not only facilitates accurate sorting and backtracking of measurement data during subsequent analysis, but also serves as a time reference for synchronization between data and external quality management systems (such as the enterprise's Manufacturing Execution System (MES), facilitating rapid location and timely response to anomalies. The second core element is the 3D coordinates of the probe grid's center node. This coordinate point clearly marks the current probe grid's specific position in the unified coordinate reference space. By continuously recording 3D coordinates, the system can map the precise spatial trajectory of the probe grid's sliding, which intuitively reflects the spatial coverage and trajectory characteristics of the entire sliding process. The third core element is the number of grid nodes covered by the current penetration probe grid. This number reflects the number of nodes that actually made effective contact or adhered to the surface during each sliding motion. Through the changing trend of the number of nodes, the system can directly judge the local geometric complexity of the surface. For example, a sudden decrease in the number of nodes may mean that the local surface has an obvious curvature mutation or defective area. The fourth core information is the average value of the angle between all unit normal vectors and the local normal vector in the coverage area. The average value of this angle is an important basis for evaluating the geometric consistency of the local surface. The larger the value, the more drastic the change in the surface curvature of the local area, and there may be obvious surface defects, wrinkles, folds or other unevenness; conversely, the smaller the average value of the angle, the smoother and more uniform the local surface geometry and the better the surface quality. Therefore, this indicator is an important original parameter directly used for quality judgment in the subsequent result generation unit.

[0050] The above sliding process is executed continuously, and the recorded path information is written to the memory buffer one by one and temporarily stored in chronological order. The system sets two constraints for the termination of the sliding action: first, when the radius of the spiral exceeds half of the longest side of the bounding box of the segment to be verified, it means that the sliding path has fully covered the majority of the target area in space; second, when the number of sliding steps reaches 50, the system believes that it has acquired a sufficient density of valid data, and further advancement may cause data redundancy and have little effect on accuracy improvement. If either of these two termination conditions is met first, the system stops the spiral advancement.

[0051] Furthermore, the result generation unit converts the slip path list output by the permeation, rotation, and link fusion unit into quantifiable, comparable, and traceable quality assessment results. These results are then encapsulated into a unified data structure and ultimately fed back to the VR device and enterprise database. To accomplish this, the system first performs a linear traversal of the slip path list. During this traversal, list elements are sequentially read according to their recorded timestamps and path numbers, ensuring consistency between the analysis process and data sampling timing, thereby maintaining repeatability in subsequent backtracking or replay scenarios. For each slip path record read, the system performs a random sample from the complete normal set. The minimum size of the random sample is set to 10% of the number of mesh nodes covered by the current path record. This ratio ensures statistical representativeness while avoiding the computational overhead of oversampling in high-resolution areas. The sampling is performed using a repeatable random seed strategy, hashing the current path number with a preset random seed to generate a subseed. This ensures that the same path record receives a consistent set of sampled vertices across different machines or run batches. This ensures that the quality judgment results are strictly consistent when compared across platforms and versions.

[0052] After selecting a sample set of mesh vertices, the system queries each vertex in the set for its first-order neighboring vertices in the mesh topology. The system then calculates the angle between the vertex's unit normal and the unit normals of its neighboring vertices. These angles are then averaged to produce a statistic known as the "baseline roughness." This statistic is designed to detect the overall trend of surface normal gradients within a local area, using the average rather than the extreme value to mitigate the influence of random noise or isolated outliers. Because vertex adjacency naturally reflects microscale variations in surface curvature, the baseline roughness serves as a reliable indicator of surface flatness. After calculating the baseline roughness, the system multiplies this value by fixed coefficients of 1.5 and 1.2, respectively, to determine the rejection and reconfirmation thresholds. These two thresholds are designed following a "screen first, then reconfirm" grading principle: multiplying by 1.5 creates a higher threshold, effectively rejecting significantly exceeding roughness standards; multiplying by 1.2 creates a slightly wider threshold, effectively marking areas that may be nearing the edge of quality but require manual or secondary verification by a secondary algorithm. The specific value of the coefficient is derived from historical data modeling. During offline verification of multiple batches and multi-material packaging, the system found that the misjudgment rate was the lowest within this multiplication range, and the missed judgment rate was also controlled at an acceptable level.

[0053] Next, the system writes a row of data into the status matrix for the current slip path record. The number of rows in the status matrix equals the total number of records in the slip path list, while the number of columns is fixed at four to maintain fast indexing in both the database and memory. The writing order is strictly based on the following: path number, number of covered nodes, average angle, and status flag. The first column, path number, is directly derived from the slip path record, allowing for quick association with other fields such as timestamp and center coordinates during external indexing. The second column, number of covered nodes, is derived from the slip path record itself and serves as a measure of local sampling density. The third column, average angle, represents the previously calculated baseline roughness, directly reflecting the surface flatness of the path segment. The fourth column, status flag, is assigned a value based on the comparison of the average angle with two thresholds: When the average angle is greater than the unacceptable threshold, the status flag is set to three, indicating a severe violation. When the average angle is between the recheck threshold and the unacceptable threshold, the status flag is set to two, indicating a pending recheck status. When the average angle is less than or equal to the recheck threshold, the status flag is set to one, indicating a pass or near pass status. The classification process is a simple conditional branch in the algorithm implementation, but because the noise has been fully suppressed by random sampling and averaging calculation in the early stage, the status mark can relatively reliably reflect the actual quality status of the local surface.

[0054] As the traversal progresses, the four columns of data in the state matrix are continuously filled, and the matrix is ​​added in row order in memory. At the same time, the system refreshes the lightweight visual summary in real time and sends the current statistical information of the matrix to the background thread of the VR virtual reality device so that the inspector can immediately view the current analysis progress when needed. After the traversal is completed, the state matrix will trigger a complete data lock to prevent concurrent write conflicts in the subsequent algorithm stage and database write-back process. The result generation unit then passes the locked state matrix to the cascading anomaly overlap cataloger for spatial aggregation and generation of unqualified segment entries; at the same time, the unit will also store the matrix in a high-availability columnar database with four metadata fields: version number, batch number, inspector ID, and algorithm model version, forming a complete traceability chain.

[0055] Furthermore, after receiving the locked state matrix, the cascaded anomaly superposition cataloger first creates a fast index table in memory, linking path numbers to spatial coordinates. This index table directly reuses the 3D coordinates of the central node stored in the previously saved slip path records. The state flag field in the fourth column of the state matrix is ​​also appended to the index structure, forming a composite entry with the path number as the primary key and spatial location and quality status as attributes. The system then scans the state matrix row by row, filtering out all rows with a state flag equal to 3 in a single-threaded traversal. These rows represent slip paths that significantly exceed the rejection threshold in terms of roughness and are therefore considered candidates for potential serious defects. The resulting set of path numbers typically exhibits a discrete distribution. The system spatially clusters these path numbers using 3D coordinates in a unified coordinate reference space to identify multiple paths that actually belong to the same defect. The clustering criterion is "spatial proximity ≤ 0.2 times the longest side of the bounding box." The 0.2 factor is derived from the statistical segmentation curve of multiple batches of experimental data. This effectively clusters multiple adjacent paths generated by spiraling under the same defect while avoiding excessive merging of different defects.

[0056] The clustering process employs a density-based, unidirectional neighborhood expansion strategy. The algorithm first sorts the set of path numbers in ascending spatial coordinate order. It then extracts the first number from the head of the set as the seed for a new cluster. It then calculates the Euclidean distance between the center coordinates of that path and the center coordinates of all subsequent unassigned paths. As long as the distance does not exceed 0.2 times the longest side of the outer bounding box, the path number is assigned to the current cluster. The algorithm then recursively checks the spatial neighborhood of the newly added path until no more path numbers meet the proximity requirement. Once the recursion completes, a cluster is fully converged, and the algorithm marks it as a clustering result. New seeds are then selected from the remaining unassigned path numbers, and the process repeats until all path numbers are assigned to an outlier cluster. This clustering strategy does not require a preset number of clusters and makes no assumptions about cluster geometry, adapting to the diverse shapes of defect features on packaging surfaces. After clustering, the system generates a failed segment entry for each outlier cluster. The entry includes the unqualified section ID, the ID of the section to be verified, a list of path numbers within the cluster, the number of paths within the cluster, the geometry of the outer bounding box, the average maximum and minimum angles, and the timestamp of the generation. The ID is generated by combining the section ID to be verified and the cluster number to ensure global uniqueness and natural ordering.

[0057] Once all abnormal cluster entries have been generated, the system sorts them in ascending order by the nonconforming segment identification number and merges all entries into a nonconforming item list. During sorting, if two entries originate from the same segment to be verified, the system prioritizes comparing the average values ​​of the maximum angle within the cluster and sorting them in descending order, allowing inspectors to see the most serious issues first when browsing the list. Once sorting is complete, the system immediately writes the nonconforming item list to the enterprise database via the database transaction interface. This write process utilizes an idempotent validation mechanism: if a record with the same key combination (batch number, product number, segment identification number) already exists in the database, the system performs an overwrite update rather than a duplicate insert, thus avoiding historical data redundancy. After a successful database table update, the system returns a version number and write timestamp to the cataloger, which appends this information to the metadata field of the nonconforming item list, allowing subsequent audits to precisely locate the timing of data generation and storage.

[0058] The cataloger then performs a coordinate transformation on the bounding box of the failed segment entry, mapping the box coordinates back to the rendering space of the 3D interactive model. It then pushes the eight vertex information for each box to the VR device's rendering thread via a message bus. Upon receiving the data, the rendering thread overlays a translucent highlight border within the inspector's field of view, using color coding consistent with the status markers. The box corresponding to marker 3 uses a highly saturated warning color with moderate transparency to avoid obscuring texture details. A text label automatically floats in the center of the box, displaying the failed segment identification number and the average maximum angle. The rendering effect dynamically follows the inspector's head movements or zooming, and depth testing locks it into the correct surface position to ensure visual consistency.

[0059] To ensure real-time rendering and database consistency in a multi-threaded environment, the cascading exception overlap cataloger initiates a bidirectional lock at the same time as clustering is completed. The uplink lock prevents new state matrices from being written, while the downlink lock protects the list of unqualified items from being modified during the write-to and render-to-pushing period. After the list is stored and a successful receipt is received, the cataloger releases the downlink lock; after the VR rendering thread confirms that all highlight borders have been loaded, the cataloger releases the uplink lock. This prevents the rendering thread from reading semi-finished data and ensures that highlighting is triggered only after the database is successfully written, avoiding false positives. The rendering thread also passes the count of successfully loaded highlight borders to the cataloger through the callback interface. The cataloger generates a quality event log based on this, writes it to the system log file, and synchronizes it with the enterprise log center for subsequent scheduling and statistical analysis.

[0060] The following is a complete and repeatable example to illustrate how to complete a remote verification of a cardboard box package. The package dimensions are set to 0.40 meters in length, 0.30 meters in width, and 0.25 meters in height. The focal length f of the binocular vision acquisition component camera is 0.016 meters, the pixel pitch p of the imaging chip is 3.45 microns, the baseline B of the left and right cameras is 0.120 meters, and the image resolution is 1920×1080. The single scan period t_c is 1 / 120 seconds, which is consistent with the IMU output frequency. Assuming that the difference d between the same-named pixel columns selected in the left and right views is 38 pixels, the depth Z is given by classical solid geometry:

[0061] ;

[0062] The image column u for this pixel is 960 pixels, and the row v is 540 pixels. Using the pinhole model, the camera coordinates (X, Y, Z) are ≈ (0.73, 0.41, 1.46) meters. The IMU outputs an angular velocity of (0.002, −0.001, 0.004) radians per second and an acceleration of (0.10, −0.08, −9.71) meters per second squared during this period. The first-order integration of the angular velocity yields a small rotation of ≈ (1.67 × 10-4, −8.33 × 10-5, 3.33 × 10-4) radians; the second-order integration of the gravity-compensated acceleration yields a translation of ≈ (3.47 × 10-5, −2.78 × 10-5, 0) meters. The rotation and translation form a 3×4 pose matrix, transforming the camera coordinates to a unified coordinate reference space to obtain (0.73003, 0.40997, 1.46) meters. This process reconstructs approximately 3.5×10^{6} point cloud samples for the entire image plane. After layered voxel filtering of the dense point cloud, 2.1×10^{6} points remain, with a mean normal vector stability variance of 0.8 degrees.

[0063] The remote inspector selects a corner of the box in the VR scene by ray clicking. The system locates the vertex number v_0 in the model and the three-dimensional coordinates =(0.12, 0.06, 0.24) meters, corresponding to the unit normal vector ≈(0,0,1). Recursively expanding the normal vector with a threshold of 15 degrees yields a set of 124 vertex normals with a bounding box size of 0.082 × 0.076 × 0.010 meters. The system assigns this set the pending verification segment identifier D17.

[0064] The shortest side of the outer bounding box is 0.076 meters, and the grid unit side length is The penetration probe grid is five by five, with a total of N = 25 nodes. The center node i = 13 is taken, and the initial coordinates are Meters, normal vector The coordinates of its four adjacent nodes are:

[0065] ;

[0066] Calculate the projection displacement of node 13. For any adjacent node j,

[0067] .

[0068] The directed distance between adjacent nodes in the normal direction ;in They are −0.001, 0.001, −0.001, 0.001 meters respectively, so:

[0069] ;

[0070] Therefore, the translation of node 13 is 0 and it remains in its original position; the rest of the nodes are fine-tuned according to the same logic, with the maximum translation not exceeding 0.0009 meters, which meets the side length difference limit. All node coordinates are written into the node coordinate set after alignment .

[0071] Spiral parameters Meters, predetermined arc length step Meters, starting polar angle . In formula 2, Substituting in, we get:

[0072] ;

[0073] Eliminate the coefficients and use Newton iteration to find Radians. So the first step is the polar coordinate radius of the center node Meter; tangent plane takes orthogonal basis , . The location of the spiral center node:

[0074] .

[0075] Continue with the same algorithm , the value is 1.737 radians, and we get . Iterate to Spiral radius in radians meters, which is more than 80% of the longest side of the outer bounding box (0.082 meters), which is 0.041 meters. At the same time, the number of steps has reached 12, which is still within the limit of 50 steps. The system continues to advance until radian, The radius exceeds 0.041 m, and the spiral propulsion is terminated. K = 16 sliding path records are obtained.

[0076] Each record stores: timestamp, for example, step k is t_k=t_0+kt_c; coordinates of the central node ; The number of covered nodes is always 25; The average angle Obtained by counting the angles between the unit normal vector and the local normal vector of 25 nodes. Example: Take the 7th path as an example. The number of covered nodes is 25, and the sampling ratio is 10%. Three nodes are selected after rounding: 3, 11, and 22. The angles between each node and the adjacent vertex normal vector are 3.8°, 4.1°, and 4.2° respectively. The average baseline roughness is:

[0077] ;

[0078] Failure threshold , review threshold The original average angle in the record , so the state is marked as 1. Repeat this process to fill the 16-row state matrix, where path 5 and path 9 appear , is assigned the tag 3.

[0079] The coordinates of the center nodes of paths 5 and 9 are (0.171, 0.110, 0.245) meters and (0.177, 0.113, 0.245) meters, respectively. The Euclidean distance is 0.007 meters, which is less than 0.0164 meters, or 0.2 times the longest side of the outer bounding box (0.082 meters), and therefore clusters them together. An unqualified segment entry, B17-1, is generated, and the eight-point coordinates of the outer bounding box are written back to the VR scene. The inspector immediately sees the patch highlighted with a translucent red frame, with the text "B17-1maxφ7.2°" hovering in the center of the frame. The inspector can further switch to a local view and confirm the defective texture. The system automatically writes the entry to the database, along with product batch P20250704 and model version v3.2.0, completing a traceable remote verification process.

[0080] Figure 2 This is a diagram of the operation of a remote inspector in a remote verification device for cargo packaging and labeling based on VR interaction. Figure 2As shown, the remote inspector wears a VR headset, which includes a VR helmet and hand controllers. The remote inspector receives a 3D interactive model generated by the 3D environment construction unit through the VR device. This model displays the complete 3D structure of the target package on the display screen. The target package has a surface texture mesh structure generated from surface texture data collected by the binocular vision acquisition component, establishing accurate spatial positioning within a unified coordinate reference space. The remote inspector interacts with the 3D interactive model through the VR device, selecting suspicious locations using rays or gestures. When the remote inspector selects a suspicious location, the VR device obtains the 3D coordinates and gesture number of the location at the selected moment. The system selects mesh vertices from the complete normal set of the 3D interactive model that connect to the seed coordinates and generates a segment to be verified through normal consistency. Each segment to be verified 9 has a unique segment identification number, such as "Segment 001" and "Segment 002" shown in the figure. The segment to be verified is identified by an elliptical bounding box that encompasses the bounding box of the candidate set of consistent normals. The system assigns a unique identification number and a corresponding vertex set to each section to be checked, completing the entire process from selecting suspicious locations to generating sections to be checked, and providing accurate input data for subsequent processing of the infiltration rotation link fusion unit.

[0081] Figure 3 This is a schematic diagram of the state matrix construction and unqualified section identification in the remote verification device for cargo packaging and labeling based on VR interaction. Figure 2As shown, the result generation unit first receives a slip path list from the infiltration rotation link fusion unit. This slip path list contains multiple slip path records, each of which includes key information such as the path number, the number of covered grid nodes, and the average angle. For example, path 001 has a covered number of nodes and an average angle of 0.8; path 002 has a covered number of nodes and an average angle of 1.2; and path 003 has a covered number of nodes and an average angle of 2.1. The result generation unit traverses the slip path list, processes each slip path record, and creates a state matrix with a number of rows equal to the total number of path numbers and a fixed number of columns of 4. The state matrix contains the following information per row: column 1 is the path number, column 2 is the number of covered grid nodes, column 3 is the average angle, and column 4 is the state flag. The system determines the status mark by comparing the average angle with a preset threshold: if the average angle is greater than the unqualified threshold, the status mark is set to 3; if the review threshold is less than the average angle and less than or equal to the unqualified threshold, the status mark is set to 2; if the average angle is less than or equal to the review threshold, the status mark is set to 1. The cascaded anomaly superposition cataloger scans the rows with status mark 3 in the status matrix, such as path 003 and path 005 shown in the figure. Based on the coordinates of the unified coordinate reference space, the cataloger aggregates the path numbers with spatial proximity less than or equal to 0.2 times the longest side of the outer bounding box into a single anomaly cluster, forming anomaly cluster 001. The system generates an unqualified segment entry for each anomaly cluster, which contains detailed information such as the path number, anomaly cluster identifier, outer bounding box coordinate range, and unqualified segment identification number. All unqualified segment entries are sorted in ascending order by unqualified segment identification number, and a list of unqualified items is automatically output and simultaneously written to the enterprise database for traceability. At the same time, the outer bounding box of the unqualified section entry is mapped back to the three-dimensional interactive model, the unqualified mark is highlighted in the VR virtual reality device, and the abnormal area is highlighted with a red mark, thereby completing the remote verification of the target packaging body.

[0082] The present invention has been described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only intended to help understand the method and core ideas of the present invention. It should be noted that, for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications may be made to the present invention, and such improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A remote verification device for cargo packaging and labeling based on VR interaction, characterized in that: The device comprises: a three-dimensional environment construction unit, a VR device, a penetration rotation link fusion unit, and a result generation unit; the three-dimensional environment construction unit comprises a binocular vision acquisition component and an inertial measurement component, and is used to synchronously acquire surface texture data and posture data of a target package, generate a three-dimensional interactive model containing a complete normal set, and establish a unified coordinate reference space within the three-dimensional interactive model; the VR device is used to receive the three-dimensional interactive model and present it to a remote inspector; the remote inspector uses the VR device to select a suspicious location in the three-dimensional interactive model and then packages it into a section to be verified; the penetration rotation link fusion unit is used to arrange a penetration probe grid for each section to be verified and record a list of slip paths on the surface of the three-dimensional interactive model in a spiral propulsion manner; the result generation unit is used to construct a state matrix based on the slip path list, generate unqualified section entries using a cascaded anomaly overlap cataloger, automatically output a list of unqualified items, and highlight unqualified marks in the VR device, thereby completing remote verification of the target package; The penetration rotation link fusion unit retrieves the compliance template identifier from the enterprise database, arranges the penetration probe grid for each section to be checked in the unified coordinate reference space, and the penetration probe grid slides along the surface of the three-dimensional interactive model in a spiral propulsion manner, and records the sliding path in real time; reads the vertex set for each section to be checked, and calculates the outer bounding box; takes the geometric center of the outer bounding box as the starting anchor point, and selects the unit normal vector of the anchor point in the complete normal set as the local normal vector; generates a square grid unit in the plane perpendicular to the local normal vector, and the grid unit side length is equal to one tenth of the shortest side of the outer bounding box; based on the grid unit , copy outward in a circular manner to form concentric rings, copy three times in total, and obtain a five-by-five penetration probe grid; for each grid node of the penetration probe grid, the grid node is located at the vertex position of all grid units of the penetration probe grid, and the complete normal set is called to query the unit normal vector of the nearest vertex of the corresponding grid node; if the angle between the queried unit normal vector and the local normal vector is less than thirty degrees, the grid node is kept in place; if the angle is not less than thirty degrees, the grid node is adjusted along the direction of the unit normal vector so that the difference in side length between it and the adjacent grid node does not exceed ten percent; after completion, the three-dimensional coordinates of the current grid node set are saved as the aligned node coordinate set; The penetration rotation link fusion unit takes the starting anchor point as the pole and plans an Archimedean spiral in the aligned node coordinate set. The spiral step length is equal to the grid unit side length. The central node of the penetration probe grid is gradually moved along the spiral path. Every time the central node moves one step, the entire penetration probe grid slides in a synchronous translation manner.

2. The device for remotely checking cargo packaging and labeling based on VR interaction according to claim 1, characterized in that: The binocular vision acquisition component uses a global shutter method to simultaneously expose the left-view image frame and the right-view image frame within a single scanning cycle, and outputs surface texture data of the target package body containing original brightness information; the inertial measurement component outputs three-axis angular velocity and three-axis acceleration at a beat exactly the same as the single scanning cycle, and obtains the corresponding posture data by first-order integration of the three-axis angular velocity and second-order integration of the three-axis acceleration.

3. The device for remotely checking cargo packaging and labeling based on VR interaction according to claim 2, characterized in that: The three-dimensional environment construction unit performs geometric correction on the left-view image frame and the right-view image frame according to the posture data; calculates the disparity value of the corrected left-view image frame and the right-view image frame pixel by pixel, adopts a segmented weighted window search to avoid error accumulation in the low-texture area, and maps the obtained disparity value to an initial depth map; according to the initial depth map and the posture data, the depth value is solved into the initial point cloud coordinate according to the three-dimensional imaging geometric relationship, and the initial point cloud coordinates of multiple consecutive frames are spatially aligned and redundantly eliminated in the coordinate system to generate dense point cloud data; performs hierarchical voxel grid filtering on the dense point cloud data, divides the dense point cloud data into multiple voxel grid levels, and performs iterative averaging operations from coarse to fine between each level to eliminate random noise and fill point cloud holes, and outputs a smooth point cloud volume; applies a triangle partitioning strategy to the smooth point cloud volume, generates a vertex set, an edge set and a face set triplet while maintaining topological continuity, and obtains a three-dimensional interaction model.

4. The device for remotely checking cargo packaging and labeling based on VR interaction according to claim 3, characterized in that: The 3D environment construction unit performs the following operations on each triangular face of the 3D interactive model: calculating the area vector of the triangular face; normalizing the area vector to a unit normal vector; adding the unit normal vector to the normal vector accumulation pool of the corresponding mesh vertex; After all triangles are traversed, the normal vector accumulation pool in each mesh vertex is averaged and normalized to obtain the final unit normal vector of the mesh vertex; Record the final set of unit normal vectors for all mesh vertices as the full normal set.

5. The device for remotely checking cargo packaging and labeling based on VR interaction according to claim 4, characterized in that: When the remote inspector selects a suspicious location in the VR device using a ray or gesture, the VR device obtains the 3D coordinates of the suspicious location and the posture number at the selected moment. The mesh vertex that is connected to the seed coordinate is selected from the complete normal set of the 3D interactive model. If the angle between the unit normal vector of the adjacent mesh vertex and the unit normal vector of the vertex where the seed coordinate is located is less than a preset threshold, the adjacent mesh vertex is added to the extended candidate set; otherwise, it is eliminated. Continue to search recursively in the extended candidate set until no new vertices that meet the threshold condition appear, and obtain the normal consistent candidate set; calculate the outer bounding box of the normal consistent candidate set and generate a continuous face patch; assign a unique to-be-checked segment identification number and vertex set to the continuous face patch, and then package it into a to-be-checked segment.

6. The device for remotely checking cargo packaging and labeling based on VR interaction according to claim 1, characterized in that: For each slip, the following information is recorded to obtain the slip path record: timestamp of the slip moment; three-dimensional coordinates of the central node; number of grid nodes covered by the penetration probe grid; average value of the angles between all unit normal vectors and local normal vectors in the coverage area; when the spiral radius is greater than half of the longest side of the outer bounding box, or when the number of slip steps reaches fifty, the spiral advancement is terminated; all slip path records are sorted in chronological order to form a slip path list.

7. The device for remotely checking cargo packaging and labeling based on VR interaction according to claim 6, characterized in that: The result generating unit traverses the sliding path list, and for each sliding path record, randomly selects mesh vertices that are no less than 10% of the number of mesh nodes of the sliding path record in the complete normal set as sampling mesh vertices; Calculate the average angle between these sampled mesh vertices and the unit normal vectors of their adjacent vertices to obtain the baseline roughness; multiply the baseline roughness by a coefficient of 1.5 to set it as the unqualified threshold, and multiply it by a coefficient of 1.2 to set it as the review threshold; create a matrix with the number of rows equal to the total number of path numbers and a fixed number of columns of 4, and write the following content row by row to obtain the status matrix: column 1: path number, column 2: number of covered mesh nodes, column 3: average angle, and column 4: status mark; if the average angle is greater than the unqualified threshold, the status mark is set to 3; if the review threshold is less than the average angle ≤ the unqualified threshold, the status mark is set to 2; if the average angle ≤ the review threshold, the status mark is set to 1.

8. The device for remotely checking cargo packaging and labeling based on VR interaction according to claim 7, characterized in that: The cascaded exception overlap cataloger scans the rows of status mark 3 in the status matrix; based on the unified coordinate reference space coordinates, the path numbers with spatial proximity ≤ 0.2 times the longest side of the outer bounding box are aggregated into a single exception cluster; an unqualified segment entry is generated for each exception cluster; all unqualified segment entries are sorted in ascending order according to the unqualified segment identification number, and a list of unqualified items is automatically output, and the list of unqualified items is synchronously written into the enterprise database for traceability; the outer bounding box of the unqualified segment entry is mapped back to the three-dimensional interactive model, and the unqualified mark is highlighted in the VR virtual reality device, thereby completing the remote verification of the target package body.

Images