A multi-camera cooperative bag three-dimensional pattern matching and cutting optimization method

Abstract

The application relates to a multi-camera cooperative vision and intelligent cutting path dynamic adjustment technology, which is mainly used for high-precision movement and boundary prediction of flexible material three-dimensional cutting in automatic processing. In view of the problem that the existing cutting path is insufficient in real-time response to material deformation, the application obtains multi-angle video streams through ring multi-camera collection, spatial registration and illumination normalization, extracts boundary features according to a hybrid model of depth separable convolution and graph neural network, and realizes abnormal path replacement in combination with Bezier curve re-planning. The system has the ability of adaptive adjustment of reasoning frequency and feature weighting strategy based on prediction error closed-loop feedback, so as to improve the adaptability and precision of the cutting path to dynamic deformation, significantly reduce the error and abnormality in the cutting process, and improve the stability and efficiency of the overall automatic cutting.

Description

Technical Field

[0001] This application relates to the fields of intelligent manufacturing and industrial vision technology, and in particular to a multi-camera collaborative method for 3D cutting and alignment optimization of bag panels. Background Technology

[0002] Currently, the demand for high-precision and high-efficiency cutting of flexible materials (such as those used for 3D cutting of bags) is growing in the fields of intelligent manufacturing and industrial vision technology. Especially on automated production lines, flexible cutting pieces with complex 3D / curved structures often exhibit large deformations and frequent dynamic changes in boundaries. Mainstream cutting path planning solutions in the industry largely rely on single-camera static vision alignment, preset template path tracking, or periodic manual correction. Specific methods include obtaining the outline of the cutting piece from a single viewpoint, obtaining the boundary points of the current frame through classic edge detection or morphological algorithms, and then driving a CNC robotic arm or cutting equipment to operate according to a preset CAD path. These methods are effective for certain regular 2D materials such as fabrics and leather, but are difficult to reliably apply to 3D, multi-degree-of-freedom moving bag cutting pieces.

[0003] The main development trends of traditional dynamic alignment schemes are reflected in two aspects: on the one hand, relying on high-resolution cameras and vision processing units to improve boundary detection accuracy; on the other hand, combining motion compensation and path fine-tuning techniques to minimize error details caused by material jitter, stretching, and mechanical disturbances. In recent years, some research has also attempted to use more advanced spatiotemporal feature modeling algorithms such as temporal convolutional networks and optical flow analysis to provide some theoretical support for dynamic boundary prediction. However, due to limitations such as blind spots in the field of view of a single camera, high computational latency, large model size, and strong dependence on hardware platforms, the practical engineering implementation effect of existing adaptive adjustment of clipping paths is still relatively limited. Some publicly available solutions use defect database scanning or template superposition to deal with abnormal deformation of soft materials, which not only increases system complexity but also reduces the smoothness of processing real-time dynamic scenes.

[0004] Typical existing technology scenarios include: using multiple static cameras to periodically sample and then fitting the overall cropped area through global reconstruction of the point cloud; or introducing complex acoustic sensing, stabilization devices to assist alignment, and even some quantum sensing or composite physical field data fusion technologies. These solutions often have significant shortcomings in terms of equipment cost, system scalability, and algorithm load, making it difficult to meet the needs of high-speed dynamic operation of production lines, frequent order changes, and continuous adaptation to flexible and diverse structural components.

[0005] Existing technologies have shortcomings and technical bottlenecks in the following key aspects: The contradiction between real-time dynamic accuracy and error: For three-dimensional cut pieces of bags that are in motion and undergo severe deformation, existing visual alignment and path planning methods either sacrifice boundary prediction accuracy in exchange for response speed, or are limited by large model volume and high computational latency, making it difficult to achieve end-to-end closed-loop compensation within 200~300ms, resulting in a decrease in cutting efficiency and pass rate.

[0006] Challenges in 3D coverage and cross-view consistency: Most solutions only acquire data from a single or limited number of viewpoints, failing to fully capture the boundary contours of complex 3D freeform surfaces, and are prone to positioning errors in blind spots or occluded areas. Even with multi-camera setups, the lack of efficient registration and cross-view feature fusion results in low utilization of 3D information.

[0007] Large model computing power consumption and system energy efficiency issues: Some dynamic prediction algorithms based on large-scale spatiotemporal networks (such as Transformer class) or 3D point cloud modeling, although theoretically able to improve the accuracy of boundary time series inference, cannot run smoothly in actual embedded or edge computing environments, seriously affecting the continuity of production cycle.

[0008] Untimely identification of local dynamic anomalies: Existing path replanning often relies on global analysis and lacks rapid capture and sensitive response to local small-scale abnormal behaviors of materials (such as instantaneous wrinkling, tensile slip). During operation, it often causes unnecessary large-scale path regeneration, resulting in resource waste and response lag.

[0009] The system has weak closed-loop adaptive capability: Most pruning systems lack a dynamic optimization mechanism that can continuously adjust itself based on actual operational feedback. They use static settings for parameters such as model inference frequency and visual acquisition density, which makes it impossible to achieve a flexible balance between resource utilization and accuracy. Summary of the Invention

[0010] This application provides a multi-camera collaborative method for aligning and cutting three-dimensional pieces of bags, aiming to solve one of the problems or issues of the prior art mentioned in the background.

[0011] This application provides a multi-camera collaborative method for aligning and cutting three-dimensional patterns of bags, specifically including: S1: Acquire multi-view image sequences of three-dimensional cut pieces of bags during the transmission and positioning process based on a multi-camera collaborative vision system; S2: Perform denoising and geometric correction processing on the multi-view image sequence to generate a standardized image sequence with consistent spatial registration; S3: Utilize depthwise separable convolution to extract local texture motion features from image sequences at various viewpoints, and combine sparse graph structures to model the topological evolution relationship between key points at the edge of the cropped piece, thus constructing the input feature tensor of the lightweight convolutional-graph neural network hybrid model L-KGNN; S4: Input the input feature tensor into a lightweight convolutional-graph neural network hybrid model L-KGNN trained by knowledge distillation, and output the predicted contour; S5: Based on the difference comparison results between the predicted contour and the original CAD cutting path, identify local abnormal segments with deviations exceeding a set threshold to generate a spatial index set. S6: For the set of spatial indexes, the incremental Bezier fitting algorithm is called in combination with the physical constraint rule library to generate a local alternative trajectory that satisfies the mechanical dynamics characteristics, forming an adaptively adjusted clipping path for optimization.

[0012] This application provides a multi-camera collaborative method for aligning and cutting three-dimensional patterns for bags, which has the following beneficial effects: (1) By deploying a circumferential vision array and a synchronously triggered multi-view image acquisition mechanism, combined with a lightweight convolutional-graph neural network hybrid model based on knowledge distillation (L-KGNN), high-precision, low-latency perception and motion trend prediction of the three-dimensional deformation state of bag three-dimensional cut pieces are realized. Compared with the problems of poor adaptability and response lag caused by traditional methods that rely on fixed template matching or static feature extraction, this scheme uses spatiotemporally aligned multi-view video streams as input, gives full play to the efficiency advantage of depthwise separable convolution in capturing local texture and dynamic changes, and introduces sparse graph structure to model the topological relationship between key edge points, which significantly improves the robustness and generalization ability of contour recognition under complex surface deformation; at the same time, by inheriting the temporal reasoning ability from the high-precision teacher model through soft label transfer learning, the number of model parameters is compressed to less than 30% of the original size while ensuring the prediction accuracy. This effectively overcomes the technical bottleneck that existing deep networks are difficult to deploy on embedded edge devices due to excessive computing resource requirements, enabling the system to run stably in a resource-constrained environment and providing a reliable data foundation for subsequent real-time path decision-making.

[0013] (2) Construct a dynamic path adaptive module and feedback verification mechanism to form a closed-loop control system of "perception-prediction-decision-execution-feedback" in the whole link, which significantly improves the response speed and execution reliability of path planning. This module, based on the short-term boundary prediction results output by L-KGNN, uses an incremental Bezier fitting algorithm to generate a smooth-transition expected trajectory. By comparing the trajectory with the original CAD design path, it accurately identifies local areas where deviations exceed limits. Local replanning is performed only for abnormal sections, and alternative paths conforming to mechanical dynamics characteristics are quickly generated by combining a preset physical constraint rule library. This avoids the high computational overhead and time delay caused by global path reconstruction in traditional methods, significantly reducing the system response cycle. Furthermore, a feedback-driven dynamic computing power adjustment strategy is introduced, which automatically adjusts the model inference frequency and sampling mode according to the trend of prediction error changes within a continuous period: when the prediction stability is high, the computational load is reduced to save energy; conversely, a high-fidelity compensation mechanism is activated to improve perception accuracy, thereby achieving intelligent allocation of computing resources and energy efficiency optimization. In addition, through an attention weight-guided spatial selection strategy, the data contribution of cameras from different perspectives is dynamically weighted according to the anisotropic characteristics of the fabric material, prioritizing the enhancement of observation confidence in deformation-sensitive areas. This effectively suppresses the risk of misjudgment caused by local occlusion, reflection, or material differences, significantly improving the system's adaptability and stability under multiple working conditions.

[0014] (3) The entire technical path focuses entirely on a vision-driven lightweight intelligent prediction and local adaptive correction mechanism. It does not rely on a pre-built defect database or fixed template matching logic, nor does it adopt non-universal cutting-edge sensing methods such as acoustic stabilization or quantum sensing. It has good engineering feasibility and production line compatibility. The proposed closed-loop control architecture can complete the entire process response from image acquisition to path update within 200ms. Compared with traditional methods based on global replanning or multi-stage offline processing, the efficiency is improved by more than 40%, which meets the real-time requirements of high-speed automated sewing scenarios. At the same time, since it eliminates the dependence on support for specific material sample libraries or expensive sensors, the system deployment cost is significantly reduced and the ease of maintenance is greatly improved. It is suitable for flexible production environments with frequent changes in various materials and styles. The overall solution not only realizes the paradigm shift from passive correction to active prediction, but also builds an intelligent path control system with self-learning, self-adjustment, and self-verification capabilities. While ensuring trajectory accuracy, it also takes into account diversity and flexibility, providing a replicable and scalable technical model for intelligent manufacturing in the fields of bags, clothing, and footwear. It is especially suitable for complex process links that require high-precision cutting piece positioning and dynamic path guidance, and has outstanding innovation and broad application prospects.

[0015] In summary, this solution achieves a synergistic improvement in efficiency, accuracy, and robustness of the pattern positioning and path guidance system through the deep integration of multi-view collaborative perception, lightweight intelligent prediction model, local dynamic replanning, and closed-loop feedback adjustment. Without the need for complex adjustments or additional hardware investment, it comprehensively outperforms existing traditional technical approaches based on static analysis or global optimization, demonstrating significant technological progress and practical application value. Attached Figure Description

[0016] Figure 1 The main flowchart of a multi-camera collaborative method for aligning and cutting three-dimensional pieces of bags is provided in a preferred embodiment of this application.

[0017] Figure 2 This is a sub-flowchart of a multi-camera collaborative method for 3D pattern alignment and cutting optimization of bags, provided as a preferred embodiment of this application.

[0018] Figure 3 Another sub-flowchart of a multi-camera collaborative method for aligning and cutting three-dimensional pieces of bags, provided as a preferred embodiment of this application. Detailed Implementation

[0019] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0020] The following disclosure provides many different implementations or examples for carrying out different structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various implementations and / or arrangements discussed.

[0021] like Figure 1 As shown, this application provides a multi-camera collaborative method for 3D pattern alignment and cutting optimization of bags, specifically including: S1: Based on a multi-camera collaborative vision system, a multi-view high frame rate image sequence of three-dimensional cut pieces of bags is acquired during the transmission and positioning process, and a spatiotemporally aligned video stream containing three-dimensional surface deformation features is obtained as raw perception data.

[0022] S2: Denoise and geometric correction are performed on the acquired multi-view image sequences to eliminate lens distortion and illumination fluctuation interference, and a standardized image sequence with consistent spatial registration is generated to support subsequent cross-view feature fusion.

[0023] S3: Utilize depthwise separable convolution to extract local texture motion features from image sequences at various viewpoints, and combine sparse graph structure to model the topological evolution relationship between key points on the edge of the cropped piece, thus constructing the input feature tensor of the lightweight convolutional-graph neural network hybrid model L-KGNN.

[0024] S4: Input the input feature tensor into the lightweight convolutional-graph neural network hybrid model L-KGNN trained by knowledge distillation, and output the predicted contour of the cut piece boundary within the next 50ms. The predicted contour is the time-series inference result of the dynamic boundary evolution trend.

[0025] S5: Based on the differential comparison results between the predicted contour and the original CAD cutting path, identify local abnormal segments with deviations exceeding the set threshold, and generate a spatial index set of the areas that need to be replanned.

[0026] S6: For the spatial index set of the local abnormal segment, call the incremental Bezier fitting algorithm combined with the physical constraint rule library to generate a local alternative trajectory that satisfies the mechanical dynamic characteristics, and form an adaptively adjusted clipping path.

[0027] S7: During the execution of the adjusted path, the system receives observation data from the multi-camera system in real time for the next cycle, performs feedback verification on the consistency between the predicted contour and the actual boundary, and generates a prediction error assessment value.

[0028] S8: Based on the prediction error evaluation value obtained from two consecutive feedback verifications, dynamically adjust the inference frequency and input weighting strategy of the L-KGNN model: if the error is lower than the tolerance, reduce the sampling rate to save computing power; if the error increases, increase the sampling rate and activate the high-fidelity compensation mode to achieve closed-loop adaptive optimization.

[0029] Step S1: Based on a multi-camera collaborative vision system, acquire multi-view high-frame-rate image sequences of the 3D cut piece of the bag during the transmission and positioning process, and obtain a spatiotemporally aligned video stream containing 3D surface deformation features as raw perceptual data. Specifically, this includes: S1.1: Deploy a circumferential vision array consisting of multiple wide-angle industrial cameras, which are distributed at different heights and orientations around the transport path of the bag 3D cut piece according to a preset spatial angle. The field of view of each camera covers all key areas of the 3D curved surface of the cut piece, so as to ensure blind-spot-free observation of edge contours and local deformation features, and generate a physical layout structure for multi-view synchronous imaging.

[0030] Based on the input transmission path geometric parameters and the three-dimensional size data of the bag 3D cut piece, a structured layout planning method (parameters: path length L, width W, cut piece thickness T) is adopted to realize the spatial distribution calculation of the circumferential array of multiple wide-angle industrial cameras, so as to ensure full coverage observation of the three-dimensional curved surface and edge contour.

[0031] Furthermore, by using a camera field of view coverage optimization algorithm (parameters: horizontal viewing angle α, vertical viewing angle β, minimum overlap rate Rmin), the optimal pitch and azimuth angles of each camera are set, and a spatial arrangement coordinate system covering all key areas of the cut piece surface is obtained to reduce blind spots.

[0032] Furthermore, a three-dimensional ray tracing simulation method (parameters: camera position Ci, cut mesh model M) is used to analyze the visibility of the curved surface at different heights and generate visibility matrix data to ensure the all-time observability of the local deformation area.

[0033] Furthermore, based on the parallax redundancy minimization algorithm (parameters: baseline length between cameras B, image resolution Res), the camera spacing is iteratively optimized, and a physical configuration table of camera mounting intervals is generated to reduce the cumulative error of geometric distortion in multi-view reconstruction.

[0034] By using array structure modeling and parameterized optimization, the spatial distribution calculation results from the previous step are transformed into a physical layout model for multi-view synchronous imaging, enabling precise deployment of the circumferential vision array at different heights and orientations, and achieving the expected technical effect of real-time observation of edge contours and local deformation features without blind spots.

[0035] For example, in a three-dimensional fabrication conveying system for bags, the conveying path length is set to 2.5m, the width to 0.8m, and the maximum fabric thickness to 0.05m. The spatial distribution of eight wide-angle industrial cameras in a circumferential vision array is planned. Each camera has a horizontal viewing angle of 90° and a vertical viewing angle of 60°, with a minimum field-of-view overlap rate of 0.15. The heights of each camera relative to the fabric centerline are calculated using a layout planning algorithm to be 0.3m, 0.6m, 0.9m, and 1.2m, respectively, and these cameras are evenly distributed along the conveying path. A visibility matrix generated using 3D ray tracing simulation shows that, with a 0.3m height camera at a 20° downward angle, the visibility of the fabric surface edge area reaches full frame coverage, and the number of observations of sensitive local deformation points is significantly increased per second. Through a parallax redundancy minimization algorithm, the camera spacing is optimized to 0.31m, reducing baseline mismatch errors in multi-view 3D reconstruction. With this configuration, the accuracy of the three-dimensional surface reconstruction of the measured pattern piece is significantly improved compared to the original layout. It also enables dynamic observation without blind spots throughout the high frame rate synchronous acquisition process. The output physical layout structure file is used for the actual installation and calibration process of the circumferential vision array.

[0036] S1.2: Perform hardware-level synchronous triggering configuration on each industrial camera in the circumferential vision array, control the image acquisition time based on the global unified clock signal, eliminate inter-frame time offset, and generate a multi-view high frame rate image sequence with strict spatiotemporal alignment as the basic input data to characterize the continuous motion and deformation dynamics of the cut piece.

[0037] For each industrial camera in the circumferential vision array, a hardware-level synchronous triggering method is introduced (parameters: global unified clock signal source frequency ≥1MHz, trigger signal delay ≤100ns) to achieve unified control of image acquisition time and ensure that imaging events from each viewpoint are strictly aligned in physical time.

[0038] Furthermore, by using clock distribution and phase-locked loop (PLL) frequency stabilization control methods (parameters: jitter <5ps, frequency stabilization error <10ppm), a low-latency distribution of a globally unified clock signal within the multi-camera system is achieved, and time consistency verification data of camera trigger signals is obtained.

[0039] Furthermore, a timestamp embedding and correction algorithm (parameter: nanosecond-level timestamp based on the IEEE 1588 precision clock protocol) is adopted to achieve accurate time domain marking of each camera's acquired frames and generate a time domain index sequence that eliminates inter-frame time offset.

[0040] Furthermore, by utilizing the linkage mechanism between the trigger signal and exposure control (parameters: exposure start delay <50μs, exposure time matching accuracy <0.1%), the exposure start and end synchronization of multiple cameras at the same physical moment is achieved, and a set of original image frames with strict spatiotemporal alignment is generated.

[0041] By using a hardware-level synchronous triggering and timestamp correction processing method driven by a globally unified clock signal, the physical layout structure of the previous step is transformed into a multi-view high frame rate image sequence that meets the spatiotemporal dual-domain alignment characteristics, thereby achieving high-precision characterization of the continuous movement and deformation dynamics of the cut pieces.

[0042] For example, on a processing line with a 2.5m conveying path and a conveying speed of 0.4m / s for three-dimensional cut pieces of luggage, eight wide-angle industrial cameras are deployed to form a circumferential vision array. The array surrounds the cut pieces at different heights and orientations, covering the entire three-dimensional curved surface area. The global unified clock signal source is set to a frequency of 2MHz, and the jitter value is measured to be 3.8ps after PLL frequency stabilization control. Each camera receives a unified TTL trigger signal, and the trigger delay is measured to be 85ns, the exposure delay is 42μs, and the exposure time deviation is controlled within 0.05%. Combined with the timestamp embedding of the IEEE 1588 protocol, the nanosecond-level marking accuracy, after comparison and correction, has a deviation of no more than 5ns. In the spatiotemporal alignment verification of the multi-view image sequences acquired after synchronous configuration, the time difference between corresponding frames of each viewpoint is less than 100ns, and the average spatial alignment residual is less than 0.02 pixels. This achieves high-precision dynamic capture of the movement trajectory and local deformation of the cut pieces, and the output video stream provides stable and consistent input conditions for subsequent texture motion feature extraction and topology modeling.

[0043] S1.3: Utilize high dynamic range imaging mode to capture texture details on the surface of the cut piece under complex lighting conditions, perform pixel-level gain compensation and non-uniformity correction on the original image to suppress brightness distortion caused by ambient light fluctuations, generate a multi-view video stream with enhanced lighting stability, and improve the signal-to-noise ratio of subsequent feature extraction.

[0044] Based on the original high frame rate image sequence of multi-view synchronous imaging, the high dynamic range (HDR) imaging mode built into the sensor hardware is selected and enabled. The exposure fusion parameters, including the ratio coefficient of short exposure time and long exposure time, are set to ensure that the texture information of bright and low bright areas are captured simultaneously within a single frame period, thereby enhancing the visibility of details under complex lighting conditions.

[0045] Furthermore, by using a pixel-level gain compensation algorithm (parameter: gain adjustment coefficient is set according to the reference grayscale field), the deviation between the actual brightness and the desired brightness of each pixel is calculated, and linear gain scaling and color balance adjustment are performed in the three channels (R, G, B) respectively to obtain a multi-channel pixel matrix after photometric consistency correction.

[0046] Furthermore, a non-uniformity correction method based on multi-scale Retinex is adopted (parameters: scale factor σ is set to 15, 90, 250), which estimates the illumination component and separates the reflection component in the local and global ranges respectively. Logarithmic domain compression and dynamic range stretching are performed on the reflection component to map the brightness values ​​of different regions to a uniform contrast distribution, thereby reducing the brightness distortion caused by the uneven spatial distribution of ambient light.

[0047] Furthermore, an adaptive gamma map (with the γ value automatically adjusted based on the local brightness mean square) is applied to the image sequence after illumination equalization to enhance the texture information in the intermediate brightness areas while maintaining the detail integrity of the dark and bright areas, thus forming intermediate video stream data with enhanced illumination stability.

[0048] By employing the aforementioned HDR imaging, pixel gain compensation, non-uniformity correction, and gamma mapping processing methods, the multi-view high frame rate image sequence from the previous step is transformed into a multi-view video stream with high signal-to-noise ratio and consistent brightness distribution under different lighting conditions, thereby improving signal quality and enhancing robustness in the subsequent cross-view feature extraction process.

[0049] For example, in a scenario where the conveyor belt for three-dimensional cutting of bags moves at a speed of 0.5 meters per second, eight wide-angle industrial cameras are arranged in a ring. Each camera is set to HDR mode, with a short exposure time of 2ms and a long exposure time of 8ms, and an exposure fusion coefficient of 0.6. The pixel gain compensation parameter is set based on a 24-level grayscale reference field and adjusted by 1.1, 1.0, and 0.95 times in the R, G, and B channels, respectively. The Retinex multi-scale parameter σ is set to 15, 90, and 250, respectively. After non-uniformity correction, the brightness variance of the images from each viewpoint is reduced to less than 40% of the original. The adaptive gamma mapping γ value is dynamically adjusted between 0.85 and 1.15. The processed video stream shows a signal-to-noise ratio improvement of more than twice that of the control group, as measured by statistics. The edge detection response in the subsequent feature extraction stage is enhanced, significantly reducing feature instability caused by illumination fluctuations. The final output multi-view video stream maintains cross-frame consistency in color and brightness in different illumination areas, meeting the high-quality requirements of the input conditions of the real-time dynamic boundary prediction model.

[0050] S1.4: Based on the pre-calibrated camera intrinsic and extrinsic rotation and translation matrices, epipolar geometric correction and depth map fusion processing are performed on multi-view image sequences to map heterogeneous images to a unified 3D point cloud coordinate system, generating a standardized multi-view video stream with consistent spatial registration, and providing geometric consistency guarantee for cross-view feature association.

[0051] The video stream with enhanced illumination stability based on the multi-camera collaborative vision system after processing by S1.3 is input to the multi-view image sequence acquired by each camera after pixel-level gain compensation and non-uniformity correction, as well as the corresponding pre-calibrated camera intrinsic parameter matrix and extrinsic parameter rotation and translation matrix.

[0052] An epipolar geometry correction method (parameters: camera intrinsic matrix, extrinsic rotation and translation matrix, feature matching threshold) is adopted to achieve epipolar constraint alignment of cross-view image pairs and eliminate feature point mapping errors caused by imaging geometry from different viewpoints.

[0053] Furthermore, a stereo matching algorithm (parameters: disparity search range, cost aggregation window size) is used to calculate pixel disparity between corresponding viewpoint images and generate dense disparity map data, providing input results for depth information derivation.

[0054] Furthermore, using a depth map fusion method (parameter: fusion weights are set based on the camera baseline length and the overlap area of ​​the field of view), the dense disparity maps of each viewpoint are converted into 3D point cloud coordinates using a triangulation formula, as follows:

[0055] in, The depth value of the target point. For camera focal length, For camera baseline length, This represents pixel parallax.

[0056] Furthermore, through a coordinate transformation algorithm (parameters: Euler angle decomposition of the extrinsic rotation and translation matrix and translation vector), the 3D point cloud output from multiple viewpoints is mapped to a unified world coordinate system, generating fused point cloud data with spatial registration consistency.

[0057] Furthermore, by using voxel meshing and point cloud interpolation methods (parameters: voxel side length, interpolation radius), spatial resampling and noise suppression of the fused point cloud are achieved. The resampled 3D point cloud and synchronous timestamp data are jointly encoded into a standardized multi-view video stream, thereby ensuring geometric consistency of cross-view feature association.

[0058] By using epipolar geometric correction and depth map fusion processing, the multi-view image sequence results from the previous step are transformed into a standardized video stream in a unified 3D point cloud coordinate system, thereby improving the reliability of spatial registration consistency and cross-view geometric constraints.

[0059] For example, the conveying speed of the three-dimensional cutting pattern for bags is 0.5 m / s, the number of circumferential vision array cameras is 8, the camera resolution is 1920×1080 pixels, and the focal length is... The value is 35 mm, and the baseline length is... Under the conditions of 120 mm, a disparity search range of 64 pixels, and a cost aggregation window size of 9×9 pixels, the epipolar geometry correction method was used to reduce the feature matching error of the left and right camera views from an average of 1.8 pixels to 0.2 pixels. A disparity map was generated, and the depth range was calculated to be between 0.3 m and 1.5 m. After transforming the depth maps of each view to a unified coordinate system using an extrinsic parameter matrix, spatial resampling with a voxel grid side length of 5 mm reduced the point cloud noise density to 0.01 points / mm³. The standardized multi-view video stream was output to S1.5, significantly improving cross-view geometric consistency and meeting the distortion-free input requirements for subsequent L-KGNN cross-view feature fusion.

[0060] S1.5: Pack the spatially registered multi-view high frame rate image sequence into a structured video data stream by timestamp, and attach metadata tags (including acquisition time, camera ID, pose information), and output it to the next processing module as raw perceptual data with spatiotemporal dual-domain alignment characteristics, for subsequent L-KGNN model to perform local texture motion feature extraction and topological evolution modeling.

[0061] Step S2: Denoising and geometric correction are performed on the acquired multi-view image sequence to eliminate lens distortion and illumination fluctuation interference, generating a standardized image sequence with consistent spatial registration to support subsequent cross-view feature fusion. Specifically, this includes: S2.1: Based on the original high frame rate image sequence output by the multi-camera collaborative vision system, obtain the RAW format pixel data of each frame image, and read the intrinsic parameter matrix (including focal length, principal point coordinates, distortion coefficients) and extrinsic parameter calibration parameters of the corresponding camera; use the camera calibration parameters to perform demosaic and radiometric correction processing on the original image to restore the true brightness distribution and generate preliminary uncompressed image data with consistent luminance.

[0062] Based on the original high frame rate image sequence output by the multi-camera collaborative vision system, the original pixel matrix of each frame is obtained using a RAW data reading interface (parameters: maximum bit depth 12 bits, color filter array mode Bayer RGGB), achieving lossless acquisition of the numerical mapping of electrical signals output by the photosensitive element. Through a module that reads the camera intrinsic parameter matrix (parameters: focal length fx, fy; principal point coordinates cx, cy; radial distortion coefficients k1, k2, k3; tangential distortion coefficients p1, p2) and extrinsic parameter calibration parameters (rotation matrix R, translation vector T), unified management of the imaging geometric description and spatial pose information of each camera is achieved, ensuring the accuracy of parameters in the subsequent geometric correction process.

[0063] Furthermore, a demosaic algorithm (using bidirectional linear interpolation with a 3×3 window size) is applied to the single-channel Bayer image matrix for color reconstruction, transforming the monochrome pixel array into a preliminary color image in three-channel RGB mode, thus achieving complete color space restoration. Further, a radiometric correction algorithm (parameters: illuminance response curve obtained from absolute exposure testing, black and white levels provided by the sensor specifications) is used to perform pixel-level brightness normalization on the demosaiced color image, mapping the light intensity values ​​to a dimensionless normalized range. This compensates for the grayscale differences caused by the nonlinear response of the imaging sensor.

[0064] Furthermore, the gain equalization module (parameter: the gain coefficient of each channel is calculated based on the white field calibration image) performs multiplicative adjustment on the radiometrically corrected RGB channels to achieve photometric consistency of the three channels, ensuring that the response values ​​of different channels to the same physical brightness are equal, and obtaining a preliminary photometric consistent uncompressed image data matrix.

[0065] By combining the above-mentioned demosaicing and radiometric correction processing, the RAW format raw data from the previous step is transformed into a color image matrix that is spatially geometrically uncorrected but has consistent luminance, thus achieving the expected technical effect of providing a high-fidelity luminance basis for lens distortion correction and spatial registration.

[0066] For example, in a multi-camera system for dynamic processing of bag cut pieces, an industrial camera with a resolution of 4096×2160 pixels and a bit depth of 12 bits is used to acquire RAW format image sequences. The intrinsic parameter matrix has focal lengths fx=3650.4 pixels and fy=3648.7 pixels, principal point coordinates cx=2048.0 pixels and cy=1080.0 pixels, radial distortion coefficients k1=-0.102, k2=0.014, k3=-0.004, and tangential distortion coefficients p1=0.0008 and p2=-0.0006. During demosaicing, 3×3 bidirectional linear interpolation is used to recover the RGB three channels. The radiometric correction curve is calculated from the pre-acquired absolute exposure response, and the normalization formula is:

[0067] in, The current pixel value. Black level. The image is set to white level. During gain equalization, the average value of each channel in the white field image is calculated, resulting in a gain coefficient of 1.02 for the R channel, 0.98 for the G channel, and 1.05 for the B channel. After multiplicative adjustment, the average values ​​of the three channels tend to be consistent. While the processed image still exhibits spatial geometric distortion under this configuration, its photometric consistency is significantly improved. The interference from illumination differences during subsequent lens distortion inverse transformation and spatial registration is effectively suppressed, enhancing the stability of cross-view feature fusion.

[0068] S2.2: Apply the inverse lens distortion transformation algorithm based on the radial-tangential model to the radiometrically corrected image data. Perform nonlinear mapping correction on the pixel coordinates according to the distortion coefficient set corresponding to each camera to compensate for barrel distortion caused by wide-angle imaging. After correction, use bilinear interpolation to reconstruct the pixel grid and generate a corrected image frame sequence with geometric distortion suppression as the reference input for subsequent spatial registration.

[0069] S2.3: Based on the global synchronization timestamp of the multi-camera system and the known extrinsic rotation and translation matrix, construct the three-dimensional reprojection relationship under the unified world coordinate system; backproject the feature point set in the corrected image frame of each viewpoint to the common three-dimensional space through the perspective projection equation, and reproject it to the virtual standard view plane, perform image registration operation based on Homography transformation, and generate a spatially normalized image sequence with viewpoint alignment.

[0070] S2.4: For the spatially registered multi-view image sequence, detect the illumination intensity drift between adjacent frames and calculate the gradient of the mean brightness change of each image block; perform nonlinear stretching adjustment of local contrast based on the adaptive histogram equalization (CLAHE) algorithm, and combine grayscale reference field modeling technology to perform piecewise linear fitting compensation of the overall illumination field, generating illumination-stability-enhanced image data free from illumination fluctuations.

[0071] For the spatially registered multi-view image sequence obtained after processing in step S2.3, an adjacent frame illumination drift detection algorithm (parameters: inter-frame time interval Δt, image block size M×N) is used to analyze the mean brightness difference of corresponding image blocks in consecutive frames and generate a block-by-block brightness change gradient matrix for identifying illumination fluctuation areas.

[0072] Furthermore, by statistically analyzing the difference in the grayscale mean sequence of each image block in adjacent frames, the brightness change gradient formula is used for calculation: in This represents the average gray value of the (i,j)th image block in the current frame. This represents the average grayscale value of the corresponding image block in the next adjacent frame. This represents the sampling time interval.

[0073] Furthermore, by using the adaptive histogram equalization (CLAHE) algorithm (parameters: contrast limit ClipLimit, grid size TilesX×TilesY), local nonlinear contrast stretching adjustment is performed on regions where the brightness change gradient exceeds the set threshold, generating grayscale corrected image sequences with enhanced texture details and alleviating the loss of details caused by uneven local illumination.

[0074] Furthermore, using grayscale reference field modeling technology (parameters: number of reference field sampling points K, fitting order p), an illuminance field distribution model based on brightness samples is established across the entire frame. A piecewise linear fitting function is then used to compensate for the overall illuminance trend in different regions, eliminating the influence of low-frequency gradual illumination drift. The piecewise linear fitting formula can be expressed as: in For the reference field position function, and These are the piecewise fitting coefficients.

[0075] By weighted fusion of the local contrast enhancement result output by CLAHE with the grayscale reference field compensation result (parameters: fusion weights α and β satisfy α+β=1), an image sequence with enhanced illumination stability and removal of illumination fluctuations is generated, which improves cross-frame illumination consistency and provides stable input for subsequent feature extraction.

[0076] Through the above fusion processing method, the spatial registration image results of the previous step are transformed into standardized image data with significantly enhanced illumination stability, thereby ensuring the brightness consistency of cross-view image sequences under complex illumination environments and significantly improving the success rate of cross-frame feature matching.

[0077] For example, in a dynamic transport scenario of 3D cut pieces for bags, with an adjacent frame sampling interval Δt = 0.05s and an image block size of 64×64 pixels, the maximum value of the brightness change gradient matrix is ​​statistically determined to be 12 gray levels. Setting the ClipLimit of the CLAHE algorithm to 3.0 and the grid size to 8×8, contrast stretching is performed in areas where the brightness change gradient exceeds 8, significantly improving the clarity of local details. The number of reference field modeling sampling points is set to 200, and the fitting order p = 2. By analyzing the fitting residuals of each segment, the low-frequency illumination drift amplitude is reduced from the original 15 gray levels to 3 gray levels. Weighted fusion is performed with fusion weights α = 0.6 and β = 0.4. The output image maintains a brightness difference of no more than 5 gray levels between different viewpoints, verifying that the processed illumination stability meets the engineering application requirements in the multi-view cropping system, ensuring the input stability of subsequent L-KGNN feature extraction.

[0078] S2.5: Multi-view image frames that have completed geometric correction, spatial registration, and illumination normalization are indexed and organized by timestamp and camera ID, and encapsulated into a standardized image tensor with uniform spatial resolution and color space standard (sRGB); the standardized image sequence is output as the input condition for the subsequent depthwise separable convolutional feature extraction module to ensure spatial semantic consistency in the cross-view feature fusion process.

[0079] like Figure 2As shown, step S3 involves: extracting local texture motion features from image sequences at various viewpoints using depthwise separable convolution, and modeling the topological evolution relationship between key points on the cropping edge using a sparse graph structure to construct the input feature tensor of a lightweight convolutional-graph neural network hybrid model, L-KGNN. Specifically, this includes: S3.1: Based on the geometrically corrected multi-view high frame rate image sequence, a channel-wise spatial convolution operation is performed on each frame image using a depthwise separable convolution kernel to extract local texture gradient and motion direction features, and generate a low-dimensional local feature map to reduce computational redundancy and retain texture dynamic information.

[0080] Based on geometrically corrected and spatially registered multi-view high frame rate image sequences, a depthwise separable convolution kernel (parameter: kernel size) is employed. × Step length (With zero padding) Spatial convolution is performed on each channel of each frame to enhance the directional response of local texture gradients and independently extract edge details.

[0081] Furthermore, by performing channel-wise convolution and element-wise dot product fusion calculation (corresponding to the pointwise stage of depth-separable convolution), the dynamic reorganization of texture features of different spectral channels under each viewpoint is realized, and a feature data matrix containing local motion direction vector fields is obtained.

[0082] Furthermore, a local gradient magnitude calculation algorithm (based on the output of dual-kernel convolution of the Sobel operator in the horizontal and vertical directions) is employed to obtain the texture gradient intensity map within the pixel neighborhood, while simultaneously calculating the gradient direction angle. To characterize the deformation trend.

[0083] Furthermore, local normalization is utilized (parameter: window size). × (Mean and variance normalization) suppresses the interference of brightness changes on texture gradient magnitude, and generates a low-dimensional local feature map with stable dynamic deformation response, thereby preserving key dynamic information under non-rigid motion on the surface of flexible materials.

[0084] By using the aforementioned depthwise separable convolution and normalization processing, the standardized image tensor from the previous step is transformed into a feature map containing local texture gradients and directional characteristics, and with reduced dimensions. This achieves the expected technical effect of reducing computational burden and enhancing the separability of deformable features.

[0085] For example, in a dynamic cutting scenario for 3D pattern pieces of bags, for a resolution of... × Frame rate A multi-view synchronized image sequence at fps, configured with a depthwise convolution kernel size of depthwise separable convolution. × The number of output channels is The pointwise convolution kernel size is × The number of output channels is During execution, the convolution operation time per frame on the GPU acceleration unit is... ms, Sobel gradient calculation window is × The formula for calculating the direction angle is: ,in and These are the gradient components in the horizontal and vertical directions, respectively. The local normalization window is fixed at [value missing]. × The normalization calculation formula is: ,in The mean of the window. The standard deviation of the window is used. The low-dimensional local feature map output by this processing module has a size that is a fraction of the original image. It effectively preserves the motion direction characteristics of the deformation of the cut piece edge, and significantly improves the response stability and spatial positioning accuracy of the feature points in subsequent key point detection tasks.

[0086] S3.2: Perform key point detection on the local feature map, use the Harris-Laplace operator to identify significant feature points with scale invariance in the edge region of the cut piece, and generate a set of spatial key points containing coordinate positions and local response intensities as the node basis of the graph structure.

[0087] S3.3: Based on the set of spatial key points, construct a sparse Euclidean distance adjacency matrix, set a dynamic threshold to connect spatially adjacent key points under adjacent viewpoints, and form a sparse graph structure that is connected across viewpoints to model the spatial topological relationship between edge points of the cut piece.

[0088] Based on the spatial keypoint set generated by keypoint detection, the Euclidean distance calculation method (parameter: 3D coordinates of each keypoint) is used to measure the spatial distance between any two keypoints. Furthermore, through a dynamic threshold setting algorithm (parameters: material type, deformation rate, sampling period), the filtering of neighboring keypoints for different camera perspectives is realized, and a keypoint pairing set that meets the spatial proximity requirement is obtained.

[0089] Furthermore, a sparse adjacency matrix construction method is adopted (parameters: total number of keypoints N, pairing set index) to represent the connectivity between keypoints in matrix form and generate an adjacency matrix data structure with adjustable sparsity to reduce storage and computational overhead. Furthermore, a cross-view connectivity strategy (parameters: view index, point cloud registration error threshold) is used to establish cross-camera connectivity relationships between keypoints from different views and generate a sparse graph node connection map with cross-view connectivity.

[0090] Furthermore, a topology consistency check algorithm (parameters: minimum loop length, maximum connectivity) is used to identify and remove redundant or erroneous connections that violate the connectivity logic of the fabric edge, forming a simplified sparse graph structure that satisfies topological constraints. Through the above graph structure construction, the spatial key point results of the previous step are transformed into sparse graph data containing cross-view connectivity relationships, realizing efficient modeling of the spatial topological relationships between edge points of flexible material fabric pieces.

[0091] For example, in a multi-camera collaborative vision system for 3D pattern cutting of bags, the Euclidean distance formula is used for 120 edge key points obtained by Harris-Laplace detection:

[0092] in, , , For the three-dimensional coordinate components of the key points, i , j Keypoint indexing was used. A dynamic threshold was set to 2.5mm based on the material being soft leather and a real-time deformation rate of 0.15mm / ms, selecting approximately 480 pairs of neighboring keypoints. A sparse adjacency matrix was constructed using N=120, with sparsity controlled above 92%. A cross-view connectivity strategy was used to select cross-camera connections with registration errors within 1mm, forming a sparse graph structure with 220 edges. After topological consistency checks, 24 erroneous edges were removed, resulting in a high-quality cross-view sparse graph with edge geometry as its skeleton. This graph significantly improved the stability and fast response capability of edge dynamic change prediction during the L-KGNN input generation stage.

[0093] S3.4: Assign the feature vectors in the local feature map to each node of the sparse graph structure according to their corresponding positions, construct a graph signal input with attribute features, and realize the fusion expression of local texture motion features and global topological relationships.

[0094] S3.5: Perform time-dimensional stacking processing on the graph signal input, integrate multi-frame graph structure data within a continuous time window, and generate a four-dimensional feature tensor (number of viewpoints × number of nodes × feature dimension × time step), which serves as the standardized input feature tensor for the lightweight convolutional-graph neural network hybrid model L-KGNN.

[0095] Based on the sparse graph structure data with node attribute features output by S3.4, the time sliding window length and step size parameters are set to control the stacking range and update frequency of consecutive frames, ensuring the capture of short-term dynamic features of edge deformation. A time-dimensional stacking method (parameters: window length L, step size Δt) is used to match the multi-frame image signals within adjacent time windows according to node indices, ensuring spatial consistency of the time series. Furthermore, a sequence alignment algorithm (parameters: timestamp constraint τ, synchronization tolerance ε) is used to perform alignment correction on the time axis of each frame image signal, eliminating time drift caused by acquisition delay and obtaining a viewpoint-node aligned temporal feature matrix. Finally, vector concatenation operations (parameters: feature dimension d) are used to sequentially arrange the attribute feature vectors of the same node in consecutive time steps, forming a node-level temporal feature sequence, realizing the embedded expression of the time evolution pattern. Furthermore, a multidimensional tensor reconstruction operation is performed (parameters: number of views v, number of nodes n, feature dimension d, time step t), recombining the temporal feature sequences of all views and all nodes into a four-dimensional feature tensor according to a preset dimensional order. The tensor structure is as follows: This achieves unified encapsulation of the spatial and temporal domains. Through the stacking and reconstruction processing described above, the local texture motion features and topological relationships from the previous step are transformed into a standardized four-dimensional feature tensor that can be directly input into the lightweight convolutional-graph neural network hybrid model L-KGNN. This enables joint feature representation across perspectives and time, improving the completeness and temporal continuity of the input data for dynamic boundary prediction.

[0096] For example, in a dynamic cutting scenario for 3D pattern making of bags, the time sliding window length L is set to 8 frames, the step size Δt is 2 frames, the sampling time interval is 25ms, and the time window covers a dynamic process of 200ms. The number of viewpoints v is 6 industrial cameras, the number of nodes n is 120 edge keypoints, and the feature dimension d is 16-dimensional local texture motion features. For each keypoint within 8 consecutive frames, after aligning according to the timestamp, they are stitched together to form a time feature sequence of length 8. After using vector concatenation, the tensor reconstruction function is called to sort all the data according to... The structural arrangement generates a four-dimensional input tensor of size 92160. After this tensor is input into the L-KGNN, the model can significantly improve its ability to capture short-term deformation trends during the prediction stage. The consistency between the output predicted contour and the actual boundary is greatly improved in the test set validation, and low latency characteristics are maintained on embedded inference hardware.

[0097] like Figure 3As shown, step S4 involves inputting the input feature tensor into a lightweight convolutional-graph neural network hybrid model L-KGNN trained by knowledge distillation, and outputting the predicted contour of the cutout boundary within the next 50ms. This predicted contour is a temporal inference result of the dynamic boundary evolution trend. Specifically, it includes: S4.1: Based on the spatially registered and standardized image sequence generated in the previous steps, a depthwise separable convolutional layer is used to extract spatiotemporal features of local texture motion features from each viewpoint to obtain a high-dimensional sparse feature response map, so as to capture pixel-level displacement patterns caused by minute deformations on the surface of flexible materials.

[0098] Based on the spatially registered standardized image sequence generated in the preceding steps, a depthwise separable convolutional layer (parameters: the kernel size for each channel is set to 3×3, and the kernel size for each cross-channel convolutional layer is set to 1×1) is used to extract the spatial features of each viewpoint image frame, thereby achieving the separation and encoding of local texture gradient and subtle motion direction information.

[0099] Furthermore, by using the temporal convolution expansion method (parameters: the length of the one-dimensional convolution window is set to 5, and the stride is set to 1), the texture and displacement patterns of consecutive frames under the same viewpoint are aggregated along the time dimension to achieve dynamic modeling of the local deformation of the flexible material surface over time, and to obtain a feature response matrix containing temporal variation information.

[0100] Furthermore, a feature sparsification algorithm is adopted (parameters: response intensity threshold is set to 0.05, and the sparsity ratio is controlled to retain 10% of the high-intensity region) to select high-value feature pixels that are sensitive to deformation and have been noise-suppressed from the feature response matrix, thereby realizing the construction of a high-dimensional sparse feature map and significantly reducing the computational redundancy of subsequent graph structure modeling.

[0101] Furthermore, the pixel intensity of the sparse feature map is mapped to a uniform scale by using response normalization operation (parameter: normalization interval is set to [0,1], and minimum-maximum normalization algorithm is used), which ensures that features from different perspectives have equivalent weights during fusion calculation and generates a high-dimensional sparse feature response map with enhanced cross-perspective feature consistency.

[0102] By combining depthwise separable convolution and temporal convolution, the standardized image sequence from the previous step is transformed into a high-dimensional sparse feature response map that can be directly input into the graph neural network module, thereby achieving accurate capture and expression of pixel-level displacement patterns caused by minute deformations on the surface of flexible materials.

[0103] For example, during the transport of three-dimensional cut pieces for bags, the spatially registered standardized image sequence acquired by the multi-camera circumferential vision array has a resolution of 1920×1080 and a frame rate of 120fps. In the depth-separable convolutional layer, the per-channel convolutional kernel size is set to 3×3, the cross-channel convolutional kernel size is 1×1, and the number of channels is 64, extracting a texture gradient feature map containing 64 channels. In the temporal convolutional expansion, a convolutional kernel with a window length of 5 is used to perform one-dimensional convolution operation on each channel in the time dimension to obtain the motion direction pattern that changes along time. When performing feature sparsification, the response intensity threshold is set to 0.05, and only the 10% of feature pixels with the highest intensity are retained to form a high-dimensional sparse feature map. The normalization step uses the min-max normalization algorithm to map all feature values ​​to between 0 and 1, forming a sparse feature response map of uniform scale. In this scenario, the final output high-dimensional sparse feature response map can clearly depict the small displacement trend of the cut piece surface between consecutive frames, ensuring that the subsequent S4.2 cut piece edge key point recognition and sparse map structure construction process has high signal-to-noise ratio input conditions, thereby significantly improving the accuracy and real-time performance of dynamic boundary prediction.

[0104] S4.2: Based on the high-dimensional sparse feature response map obtained in S4.1, identify the set of key points on the edge of the cut piece and construct a sparse graph structure, where nodes represent the position and normal information of key edge points, and edges represent the topological connection relationship and relative motion vector between adjacent points, generating an initial graph data representation.

[0105] Based on the high-dimensional sparse feature response map obtained from S4.1, a method based on gradient orientation histogram (HOG) combined with scale invariant feature transform (SIFT) (parameters: feature scale range 1.6~4.0, number of orientation histogram segments 8) is adopted to realize multi-scale feature point recognition in the edge region of the cut piece, and extract the sub-pixel level coordinate position and local normal information of each feature point.

[0106] Furthermore, a contour thinning algorithm based on Canny edge detection (parameters: high threshold 0.2, low threshold 0.05) is used to merge the edge segments of multi-scale feature points, fusing isolated feature points into a set of edge segments with continuous curvature, thus obtaining a preliminary set of boundary points and their direction vectors.

[0107] Furthermore, a K-nearest neighbor search algorithm with Euclidean distance weighting (parameter: neighborhood threshold is twice the camera's imaging resolution, corresponding to a physical size of approximately 1.2 mm) is used to achieve cross-viewpoint keypoint matching and redundancy removal, and to calculate the relative motion vector between adjacent keypoints. The vector is defined as the difference in position between two key points within the current time window divided by the time interval.

[0108] Furthermore, using an adjacency matrix construction algorithm (parameters: maximum adjacency degree 5, connection weight is the product of distance weight and motion similarity), each keypoint is treated as a node. Key pairs that meet the adjacency criteria are used as edges. The relative motion vector and direction angle information are stored in the edge attributes to construct a sparse graph structure with edge weights.

[0109] By assigning attribute values, the feature vectors of the corresponding node positions in the high-dimensional sparse feature response graph are mapped to the graph node attributes, generating an initial graph data representation that includes node position coordinates, normal information, local texture features, and edge topology. This achieves a structured expression of the input features and provides the basic data for geometrically and semantically consistent message passing in the graph neural network in S4.3.

[0110] For example, in a production scenario where the width of a 3D cut piece for a bag is 450mm and the height is 300mm, the high-dimensional sparse feature response map is output by four cameras at a resolution of 1920×1080 and a frame rate of 120FPS. HOG+SIFT detection yields an average of 540 edge feature points per frame. After Canny refinement, the number of feature points is reduced to approximately 420, with an average edge segment length of 12.5mm. When using K-nearest neighbor search with a neighborhood threshold of 1.2mm, the cross-view matching rate reaches over 93%, and the formula for calculating the relative motion vector of each pair of matching points is generated.

[0111] in, Let be the position vector of the key point in the unified machine coordinate system. and The timestamp is used. After the adjacency matrix is ​​constructed, the total number of nodes is approximately 400, the average adjacency degree is 4.2, the edge weights follow a normal distribution, and the standard deviation is 0.15. The resulting initial graph data indicates that the construction process can be completed within 50ms when inputting into a subsequent graph neural network, significantly improving the stability and accuracy of the predicted contours.

[0112] S4.3: Input the initial graph data representation generated in S4.2 into the message passing layer of the graph neural network, perform multiple rounds of node feature aggregation and update operations, use the neighborhood weighted average mechanism to fuse context deformation information, and output an enhanced edge state embedding vector sequence containing global consistency constraints.

[0113] S4.4: Based on the enhanced edge state embedding vector sequence output by S4.3, combined with the time sliding window mechanism, it is input into the lightweight temporal inference module. The gated recurrent unit (GRU) architecture is used to model the state evolution path between consecutive frames, predict the spatial distribution trend of the cut piece boundary points at the next moment, and generate a short temporal dynamic contour sequence.

[0114] Based on the enhanced edge state embedding vector sequence output by S4.3, a time sliding window mechanism (parameters: window length L=5 frames, step size Δt=10ms) is used to input continuous time series data into the lightweight time series inference module in batches to achieve stable modeling of deformation state in a short time.

[0115] Furthermore, a gated recurrent unit (GRU) architecture (parameters: number of hidden layer nodes h=64, update gate weight initialization range [-0.08,0.08]) is used to recursively process each batch of temporal embedding vectors, capture the dependencies between sequences and realize state memory and update, and output the hidden state vector sequence at each time step.

[0116] Furthermore, a context fusion method based on bidirectional GRU (parameter: concatenation dimension of forward and backward hidden layer outputs = 128) is adopted to simultaneously model the edge evolution trends of the past and future, improve the temporal consistency of boundary point prediction, and obtain the fused global temporal feature vector set.

[0117] Furthermore, the global temporal feature vector set is projected onto the spatial coordinate domain through a linear mapping layer (parameters: weight matrix dimension = 128×2, initial bias value = 0), generating a two-dimensional or three-dimensional position prediction vector for each boundary key point at the future prediction time t+Δt.

[0118] Furthermore, a short-time-domain coordinate smoothing algorithm (parameter: exponential smoothing coefficient α=0.3) is used to smooth the coordinate set at the same prediction time, reducing the impact of high-frequency fluctuations on the tracking accuracy of the clipping path, and generating a continuous short-time-domain dynamic contour sequence.

[0119] By using a gated loop unit architecture and context fusion processing, the enhanced edge state embedding vector sequence from the previous step is transformed into future single-step boundary position prediction data, achieving a dynamic and smooth tracking effect for the deformation of flexible materials.

[0120] For example, in a 3D cutting scenario for bags, the time sliding window is set to 5 frames with a step size of 10ms. Each frame contains a state embedding vector of 32 edge keypoints with a dimension of 64. This vector is input into a bidirectional GRU module with 64 nodes in both the forward and backward hidden layers, resulting in a 128-dimensional global temporal feature vector after concatenation. This vector is then linearly mapped with a weight matrix of size 128×2 to generate two-dimensional coordinate predictions in millimeters. A future prediction time Δt = 50ms is set to obtain the predicted spatial position of each keypoint at that time. An exponential smoothing algorithm with a smoothing coefficient α = 0.3 is used to smooth the predicted coordinates, removing high-frequency jitter and obtaining a smoothed short-term dynamic contour sequence. Testing shows that this contour sequence, after subsequent Bezier fitting, can form a continuous cutting path that meets mechanical execution requirements. Under high-speed dynamic cutting conditions, the contour prediction error is significantly reduced, effectively improving cutting accuracy and finished product quality.

[0121] S4.5: Performs Bezier curve fitting and geometric smoothing on the short-time dynamic contour sequence output by S4.4 to eliminate high-frequency jitter noise and generate a continuous and physically realizable cut piece boundary prediction contour within the next 50ms, which serves as the input condition for the path adaptive adjustment module.

[0122] Step S5: Based on the difference comparison results between the predicted contour and the original CAD clipping path, identify local abnormal segments where the deviation exceeds a set threshold, and generate a spatial index set of areas requiring path replanning. Specifically, this includes: S5.1: Obtain the predicted contour point cloud data of the cut piece boundary within the next 50ms output by the lightweight convolutional-graph neural network hybrid model L-KGNN, as well as the vector curve data of the original CAD cutting path that has been pre-imported, as input objects for differential comparison; use the spatial coordinate unified alignment algorithm to map the two types of path data to the same mechanical coordinate system, eliminate the registration error caused by the viewpoint transformation, and generate a spatiotemporally aligned dual-path comparison benchmark.

[0123] Based on the prediction output of the lightweight convolutional-graph neural network hybrid model L-KGNN, the predicted contour point cloud data of the cut piece boundary within the next 50ms is obtained, and the pre-imported original CAD cutting path vector curve data is read to form the input object for differential comparison.

[0124] A unified spatial coordinate alignment algorithm (parameters: 3D rigid body transformation matrix, coordinate reference origin setting) is adopted to realize the coordinate system mapping function between the predicted contour point cloud and the CAD vector curve, and to convert the two types of path data into the same mechanical coordinate system.

[0125] Furthermore, by using an external parameter matrix optimization method (parameters: rotation vector, translation vector), the viewpoint transformation residual is iteratively minimized, and optimized transformation matrix data with registration error less than a set threshold is obtained.

[0126] Furthermore, the Iterative Closest Point (ICP) algorithm (parameters: maximum number of iterations, convergence tolerance) is adopted to achieve fine matching between the predicted point cloud and the sampling points of the CAD curve, and to generate a matching residual distribution for detecting high-frequency registration errors.

[0127] Furthermore, by using a residual field weighted smoothing method (parameters: weighted window radius, smoothing coefficient), local mean smoothing of the residual data is achieved, generating a spatiotemporally aligned dual-path comparison benchmark.

[0128] By using the above-mentioned unified spatial coordinate alignment algorithm and residual optimization processing method, the predicted contour and CAD path in the previous step are transformed into a dual-path geometric datum with consistent registration, thereby achieving high-precision input conditions for subsequent deviation calculation.

[0129] For example, in a scenario of cutting a flexible leather bag 3D pattern, the L-KGNN model outputs a 50ms boundary prediction point cloud containing 500 3D coordinate points, with coordinate ranges of X: [0, 1200] mm, Y: [0, 800] mm, and Z: [0, 50] mm in the machine coordinate system. The original CAD cutting path is sampled every 2 mm, forming 600 2D plane points (Z=0 mm), which are then expanded into a 3D curve representation. The spatial coordinate unification alignment algorithm sets the initial values ​​of the rigid body transformation parameters as translation vector (5 mm, -3 mm, 0 mm) and rotation vector (0.5°, -0.2°, 0°), with the reference origin being the center of the positioning hole at the upper left corner of the pattern piece. During the optimization of the extrinsic parameter matrix, the least squares method is used to iteratively calculate the correction amounts for rotation and translation to match the mean residual value approaching zero. The iterative nearest point algorithm sets the maximum number of iterations to 50 and the convergence tolerance to... mm, the final registration error converges to mm. When performing weighted smoothing of the residual field, the weighted window radius is set to 5 sampling points, and the smoothing coefficient is... The generated spatiotemporally aligned dual-path comparison benchmark has significantly improved geometric consistency across the entire path range, meeting the accuracy requirements for deviation calculation in S5.2.

[0130] S5.2: Based on the aligned predicted contour and the original CAD path, perform segmented Euclidean distance calculation to obtain the spatial deviation sequence between each path sampling point; use the sliding window method to perform local mean filtering on the deviation sequence to suppress misjudgment caused by isolated noise points and generate a smoothed path deviation distribution curve as the basic input for abnormal segment identification.

[0131] S5.3: Based on the preset dynamic deviation threshold (adaptively set based on material type), the smoothed deviation distribution curve is binarized and discriminated, and continuous path segments whose deviations continuously exceed the threshold are marked; combined with the minimum effective length constraint (to prevent fragmented segments from triggering replanning), local abnormal segments with actual impact are identified, and their start and end index parameter sets in the original path are generated.

[0132] Based on the spatial sampling point sequence of the smoothed path deviation distribution curve and the original CAD path, a dynamic threshold discrimination algorithm (parameter: adaptive setting of material type) is used to realize the function of distinguishing the significance of the deviation sequence.

[0133] Furthermore, through the binarization mapping function (parameter: deviation threshold T) material This allows the smoothed deviation curve to be converted into a logical discrimination sequence, and the over-threshold state label data or result for each sampling position.

[0134] Furthermore, through a continuous segment detection algorithm (parameter: minimum continuous length L) min This enables segment continuity analysis of the over-threshold state label sequence and generates an index set of continuous over-threshold path segments.

[0135] Furthermore, length constraint filtering is applied (parameter: L). min Based on the mechanical path resolution setting, short segments with a length less than the minimum effective length in the index set are removed, and a set of uninterrupted abnormal segments with actual impact in physical execution is generated.

[0136] Furthermore, by using the segment start-end index extraction function, the start and end sampling index values ​​of each abnormal segment in the original CAD path are calculated to form the abnormal segment start-end index parameter set.

[0137] By combining dynamic threshold binarization discrimination with length constraint filtering, the smoothed deviation curve result from the previous step is transformed into spatial anomaly segment index data that can be used for path replanning, thereby achieving the technical effect of accurate identification and location of local anomaly segments.

[0138] For example, in the scenario of cutting bag panels made of flexible leather, the smoothing deviation curve is taken from the Euclidean distance sequence of 64 path sampling points. The material type is leather, and the dynamic deviation threshold T is set based on its stretch rate. material for Millimeters, using a binary discriminant function to map the deviation value of each sampling point to logic 0 or 1, where This indicates a threshold exceeding state. For the output sequence, a continuous segment detection algorithm is applied, with a minimum continuous length L set. min for The length of each sampling point was measured, and three lengths were obtained respectively. , and The segments exceeding the threshold are filtered out by length constraint, removing segments with a length of [length value missing]. For short segments, only the first two segments are retained as valid outlier segments. During the start and end index extraction process, the starting index of the first outlier segment is located at... The endpoint index is The starting index of the second abnormal segment is located as follows: The endpoint index is The final output set of start and end index parameters for the abnormal segment is {(15,22),(35,46)}, which is used as input for the local path reconstruction task in the subsequent incremental Bezier fitting module. After execution, the shape accuracy of the clipping trajectory in the abnormal region is significantly improved, and redundant path adjustments in the non-abnormal region are effectively avoided.

[0139] S5.4: Based on the identified start and end index parameter set of local abnormal segments, extract the original CAD path geometric feature fragments of the corresponding area, and overlay the corresponding predicted contour evolution trend information to construct a replanning requirement description unit containing the context environment; encapsulate each description unit into a path correction task package with a spatial identifier.

[0140] The input condition is the set of start and end index parameters of the local abnormal segment generated in step S5.3. This set of parameters contains the spatial location and geometric range information of each abnormal path segment in the original CAD clipping path, and can also access the corresponding predicted contour evolution trend data as a dynamic supplementary reference.

[0141] A path segment extraction algorithm (parameters: start and end index parameter set and CAD vector path data) is used to extract the corresponding geometric feature segments from the original CAD path and retain their control point sequence and curvature distribution information.

[0142] Furthermore, by using a contour fusion algorithm (parameters: predicted contour point cloud and CAD path segment control points), the dynamic offset trend of the predicted contour is spatially superimposed with the original path segment to generate an enhanced geometric feature set containing the deformation trend in the next 50ms.

[0143] Furthermore, an environmental context modeling method is adopted (parameters: enhanced geometric feature set, material properties of adjacent path segments and current production cycle speed) to construct a replanning requirement description unit that integrates path geometry, material properties and dynamic trends, and attach process constraint labels (such as allowable acceleration range and allowable offset tolerance) to it.

[0144] Furthermore, by utilizing a task encapsulation algorithm (parameters: description unit ID and spatial coordinate reference), each replanning requirement description unit is encapsulated into a spatially identifiable path correction task package, and the generation timestamp and priority level of the task package are recorded.

[0145] By using the above task encapsulation processing method, the abnormal path segments identified in the previous step are transformed into spatially identifiable path correction task packages, enabling accurate regional positioning and orderly execution of the subsequent local path reconstruction module.

[0146] For example, in a CAD clipping path containing 500 sampling points, the start and end indices of the abnormal segment are 120 and 185, respectively. The corresponding CAD path segment length is 65mm, and the curvature ranges from 0.02 to 0.05. The control point set extracted for this segment is {P0(10,25),P1(12,27),P2(18,32),P3(20,35)}. The predicted contour offset trend in the next 50ms is a displacement of 0.8mm in the positive Y-axis direction, accompanied by a bending increment of 0.5mm in the negative X-axis direction. The contour fusion process sequentially translates and superimposes the coordinates of the CAD segment control points, such as updating the position of point P2 to (17.5,32.8). In the environmental context modeling, the material property is flexible PU, the allowable acceleration is 0.4m / s², and the offset tolerance is 1mm. The generated replanning requirement description unit is numbered RP-20240215-001, encapsulated as a task package and attached with the spatial identifier P2_Index_Updated=17.5_32.8, with a high priority, and used as the input signal for the incremental Bezier fitting algorithm module to support the generation of local alternative trajectories that meet physical constraints.

[0147] S5.5: Summarize all path correction task packages and generate a spatial index set for the areas where path replanning is required. This set stores the mechanical coordinates, length range, and priority labels of each abnormal segment in a list structure, and outputs them to the incremental Bezier fitting algorithm module in the next stage through the communication interface, serving as a trigger signal for local path reconstruction and a basis for area positioning.

[0148] Step S6: For the spatial index set of the local abnormal segments, an incremental Bezier fitting algorithm combined with a physical constraint rule library is invoked to generate a local alternative trajectory that satisfies the mechanical dynamics characteristics, forming an adaptively adjusted trimming path. Specifically, this includes: S6.1: Based on the spatial index set of local abnormal segments output by S5, obtain the original CAD path fragments and their neighborhood coordinate sequences corresponding to each abnormal segment, which serve as the geometric input basis for local path replanning to ensure the spatial positioning consistency of the path segments to be corrected.

[0149] S6.2: The incremental Bézier fitting algorithm is used to perform segmented smoothing on the predicted contour point set in the neighborhood of the abnormal segment. Based on the control point iterative optimization mechanism, a set of Bézier curve segments with C² continuity is calculated to generate a preliminary smooth alternative path, so as to eliminate high-frequency noise and meet the trajectory smoothness requirements of the cutting motion.

[0150] Based on the predicted contour point set of the abnormal segment neighborhood obtained by S6.1, an incremental Bezier fitting algorithm (parameters: segment length = 10mm, initial number of control points = 3) is used to achieve segmented smoothing of the local path to isolate the influence of high-frequency noise.

[0151] Furthermore, an adaptive adjustment of the position and number of control points is achieved through a control point iterative optimization mechanism (parameters: iteration step size = 0.5 mm, convergence condition is curvature change rate < 0.01), and an optimized control point set that satisfies local geometric continuity is obtained.

[0152] Furthermore, by utilizing the C² continuity constraint, the first and second derivatives between curve segments are matched through a cubic Bézier curve segment splicing algorithm (input: optimized control point set), and a set of Bézier curve segments that maintains trajectory smoothness is generated.

[0153] Furthermore, based on the high-frequency noise detection function, spectral analysis is performed on the curve control point normal vector change sequence to filter out jitter components with frequency domain energy concentrated above 10Hz, thereby achieving steady-state smooth output of the curve segment.

[0154] The incremental Bessel fitting algorithm described above transforms the predicted contour point set of local abnormal sections into a preliminary smooth alternative path with physical feasibility, thereby achieving high-stability trajectory generation under the dynamic deformation conditions of flexible materials.

[0155] For example, in the scenario of cutting three-dimensional pieces for bags, the predicted contour of the abnormal segment neighborhood acquired by the camera contains the coordinates of 120 spatiotemporally registered points, the segment length is configured to be 10mm, the initial number of control points for each segment is set to 3, and the iteration step size is 0.5mm.

[0156] For each set of control points on the curve segment, an iterative optimization mechanism is used to adjust the positions of the control points until the convergence condition of the rate of curvature change being less than 0.01 is met. The control point optimization formula is as follows:

[0157] in, The coordinates of the original control points. For the updated control point coordinates, This is the iterative displacement vector.

[0158] The C² continuity formula for a cubic Bézier curve segment is:

[0159] During the curve segment splicing process, ensure that adjacent segments at t=0 and t=1 are aligned. and Consistency ensures that the changes in velocity and acceleration of the overall curve are smooth and without abrupt changes.

[0160] After spectrum analysis, the high-frequency noise amplitude was reduced from the original 2.5mm to 0.3mm. The alternative path did not exhibit any vibration or sudden stop when running on the CNC cutting machine, and the smoothness of the trajectory was significantly improved, meeting the execution stability requirements for high-speed cutting of flexible materials.

[0161] S6.3: Input the initially generated set of Bézier curve segments into the physical constraint rule base, and perform feasibility verification on the key nodes of the curve segments based on the dynamic parameters of the cutting mechanism (including maximum acceleration, maximum angular velocity, and minimum turning radius), identify infeasible path segments that violate mechanical motion constraints, and form constraint conflict areas that need further optimization.

[0162] Input the set of Bézier curve segments with C² continuity output from S6.2 into the physical constraint rule base, and call the path feasibility analysis method based on the dynamic parameters of the cutting mechanism (parameters include maximum acceleration a). max Maximum angular velocity ω max Minimum turning radius R min This enables dynamic executability determination of key nodes in the curve segment.

[0163] Furthermore, using a velocity-curvature joint verification algorithm (parameters: curvature κ, node spacing Δs), the instantaneous centripetal acceleration of each curve segment at the maximum allowable speed of the machine tool is calculated and compared with a. max The threshold is used to generate speed constraint judgment results, resulting in a set of overspeed risk nodes.

[0164] Furthermore, the instantaneous radius of curvature of each curve segment is calculated using the radius of curvature analysis method (parameter: R=1 / κ), and compared node by node with R_min to generate the turning radius constraint judgment result, thus obtaining the set of radius violation nodes.

[0165] Furthermore, the angular velocity estimation formula is used.

[0166] in, Let be the linear velocity at the node. Let ω be the radius of curvature, and let ω be the calculated value. max The comparison is performed to generate angular velocity constraint determination results, resulting in a set of overspeed rotating nodes.

[0167] By using a multi-constraint joint matrix synthesis method, the velocity constraint judgment results, radius constraint judgment results, and angular velocity constraint judgment results are uniformly mapped according to the node coordinate displacement index to generate a set of constraint conflict regions, thereby achieving complete identification of infeasible path segments that violate any mechanical dynamics constraint.

[0168] Through the above verification process, the input set of Bézier curve segments is transformed into a dataset of path segments with constraint violation labels, thereby achieving accurate labeling of the parts that need further optimization.

[0169] For example, in a three-axis CNC cutting machine, the maximum acceleration a max Set as m / s², maximum angular velocity ω max Set as rad / s, minimum turning radius R min Set as mm; for a given set of initially generated Bézier curve segments, the node spacing Δs is... mm, the linear velocity v at the node is estimated by the control system as follows: m / s, the curvature κ of a certain node is obtained through curvature estimation. mm - ¹, then the radius R = ≈ mm, significantly lower than R min Marked as a radius violation; the angular velocity of this node is given by the formula ≈ rad / s, much higher than ω max Also marked as an angular velocity violation; the corresponding instantaneous centripetal acceleration 'a' is given by the formula ≈ m / s², exceeding a max The node is marked as having a speed violation. Because all three constraints are exceeded, this node is classified into the constraint conflict region set and output to S6.4 for local parameterization adjustment based on gradient projection to ensure that subsequent paths meet mechanical executability and reduce vibration risk.

[0170] S6.4: For areas with constraint conflicts, a local parameterization adjustment strategy based on gradient projection is adopted to apply directional constraints and amplitude limits to the Bézier control points, recalculate the modified Bézier segments that meet physical feasibility, and output a local alternative trajectory that conforms to the dynamic characteristics of the cutting device, ensuring that the path can be executed without causing mechanical chatter or overshoot.

[0171] For constraint conflict regions identified by the physical constraint rule base, a local parameterized adjustment strategy based on gradient projection is adopted (input parameters include the coordinate set of conflict control points and the parameter vector of the curve segment violating the constraint) to apply directional constraints to the control points of the Bézier curve. Furthermore, through a directional constraint setting method (parameters: constraint direction vector, normal boundary angle threshold), the geometric correction of the control points is achieved, ensuring that the tangential direction of the curve segment meets the maximum allowable deflection angle limit of the trimming mechanism, and obtaining the coordinate set of the control points after directional adjustment.

[0172] Furthermore, the amplitude limit setting method (parameters: maximum acceleration limit, maximum angular velocity limit, minimum turning radius) is adopted to realize the constraint adjustment of the displacement vector magnitude of the control point and generate a set of corrected control point coordinates that meet the amplitude limit conditions.

[0173] Furthermore, an iterative optimization algorithm based on gradient projection is used (parameter: iteration termination tolerance). The projection step size η is used to treat the direction and magnitude constraints as a joint constraint surface. A projection update operation is performed in the local parameter space of the control points to obtain the optimized control point vector that simultaneously satisfies both types of constraints. This operation uses the following iterative update formula:

[0174] in, Let be the coordinate vector of the control point in the i-th iteration. This is the step size coefficient. The gradient is the physical feasibility cost function constraining the conflict region. Used for projection direction calculation.

[0175] Furthermore, by using the control point recalculation method (parameter: sampling step size Δt for the Bezier parameter t), the optimized control point set is substituted into the Bezier curve parametric equation to regenerate the modified Bezier segment curve data that meets the physical feasibility constraints, and outputs a set of local alternative trajectories that conform to the dynamic characteristics of the cutting equipment.

[0176] By employing the aforementioned gradient projection-based local parameterization adjustment strategy, the infeasible curve segment from the previous step is transformed into a physically feasible and geometrically smooth alternative trajectory, achieving the expected technical effect that the trajectory does not induce mechanical vibration or overshoot during execution.

[0177] For example, in the process of cutting three-dimensional patterns for bags, the control points of a certain abnormal section curve segment generated by the original incremental Bézier fitting are (120.5, 85.3), (130.7, 90.2), and (140.8, 95.4). When the maximum acceleration limit is 5.0 m / s², the maximum angular velocity is 3.0 rad / s, and the minimum turning radius is 25 mm, the direction of the second control point of this curve segment deviates from the tangent by 35°, and the displacement amplitude is 18 mm, both violating the constraint conditions. Through the direction constraint setting method, the deflection angle is corrected to within the range of ≤20°, and the direction correction vector Δd = ( 2.1 mm, 1.5 mm). By using the amplitude limit setting method, the module length is restricted to a range of ≤15 mm, resulting in the amplitude correction vector Δm = ( 1.8 mm, 1.2 mm). Input these two corrected vectors into the gradient projection iterative optimization algorithm, setting the iteration step size η = 0.05, and the tolerance... =0.001, the cost function is defined as the sum of squares of control point constraints violated, and the coordinates of the control points satisfying all constraints are calculated iteratively as (128.6, 88.0). Substituting the optimized control point set into the cubic Bessel formula, the curve coordinate sequence is recalculated under the condition of sampling step size Δt=0.01. The generated alternative trajectory is verified by dynamic simulation. The equipment runs smoothly without flutter or trajectory overshoot, the trimming accuracy is significantly improved, and the trajectory feasibility fully meets the mechanical execution requirements.

[0178] S6.5: Topologically stitch all physically compliant optimized local alternative trajectories with the unaffected original CAD path segments to generate a spatially continuous and geometrically consistent adaptively adjusted clipping path, and mark the update timestamp and version identifier for subsequent execution module calls and feedback verification.

[0179] The input conditions are a set of local alternative trajectories generated through physical compliance optimization and unaffected path segments of the original CAD path, both of which are in a unified mechanical coordinate system and have spatial continuity detection labels.

[0180] A topology splicing algorithm (parameters: spatial connection tolerance ≤ 0.01 mm, curvature continuity order C²) is used to achieve endpoint matching and seam smoothing of the local alternative trajectory and the original CAD path segment, ensuring that there are no geometric gaps or abrupt changes in normal at the connection.

[0181] Furthermore, by using a node merging method based on coordinate difference (parameter: node distance threshold = 0.005mm), spatial overlap and fusion of path end nodes are achieved, resulting in a topologically consistent global path node set.

[0182] Furthermore, a path sequence reordering algorithm is adopted (parameter: path segment index priority follows the original CAD design order) to optimize the traversal order of the spliced ​​path, ensuring that the motion instructions in the cutting process are consistent with the order of the geometric path.

[0183] Furthermore, a local weighted smoothing method based on radius of curvature analysis (parameters: smoothing window length = 3 nodes, weight function is Gaussian distribution σ = 1) is used to achieve uniform transition curvature of the path at the splicing point and generate continuous geometric curve data for clipping control.

[0184] The path version identifier generation module adds an update cycle timestamp and version number index to the assembled adaptive adjustment and trimming path, thereby realizing version management and traceability of path data.

[0185] For example, in the process of cutting 3D patterns for bags, the set of local alternative trajectories includes 3 curve segments generated by incremental Bézier fitting and satisfying dynamic constraints, while the original CAD path retains 7 unaffected curve segments. When using the topology stitching algorithm, the spatial connectivity tolerance is set to... The order of curvature continuity is set as C 2. Achieve endpoint matching between each alternative curve segment and the adjacent original path segment. During the matching process, if the coordinate difference between the two endpoints is less than 1, the endpoints will be matched accordingly. Then, node merging is performed. After node merging, a path sequence reordering algorithm is used to insert the local replacement segment into the correct position index in the original CAD path, avoiding misalignment of the trimming order. A Gaussian weighted smoothing function is used at the splicing point. , σ take The smoothing window has a length of 3 nodes to eliminate possible curvature abrupt changes at the connection points. The final generated adaptive adjustment trimming path has no breakpoints in the global spatial continuity detection, and the curvature radius changes smoothly, meeting the requirements of the trimming machine's maximum angular velocity and minimum turning radius. The path version identifier is recorded as "v202405060930", enabling the function of real-time invocation and closed-loop feedback verification in subsequent execution modules.

[0186] Step S7: During the execution of the adjusted path, the observation data of the multi-camera system in the next cycle is received in real time, and the consistency between the predicted contour and the actual boundary is checked to generate a prediction error evaluation value. Specifically, this includes: S7.1: Obtain the latest multi-view high frame rate image sequence acquired by the multi-camera collaborative vision system in the next cycle, as the perceptual input for the actual state of the current cropping piece, perform cross-view 3D reconstruction based on the standardized image sequence with consistent spatial registration, and generate the actual boundary point cloud set of the cropping piece at the current moment to provide a real geometric benchmark for comparison with the predicted contour.

[0187] S7.2: The predicted outline of the future 50ms cut piece boundary output by S4 is mapped to the current physical space coordinate system through coordinate transformation, and spatiotemporally aligned with the actual boundary point cloud set generated by S7.1. The Euclidean distance between corresponding points is calculated based on the nearest neighbor matching algorithm to obtain the point-by-point spatial deviation vector field as a spatial distribution representation of the prediction error.

[0188] S7.3: Perform statistical analysis on the spatial deviation vector field obtained in S7.2, calculate the root mean square error (RMSE) and maximum deviation (Max Deviation) as core indicators, and generate a quantitative prediction error evaluation value. This evaluation value reflects the overall consistency level between the predicted profile and the actual boundary, and is used for threshold judgment in subsequent adaptive control logic.

[0189] S7.4: Based on the preset tolerance range, the prediction error evaluation value generated in S7.3 is conditionally judged. If the RMSE is lower than the first threshold and the Max Deviation is not exceeded, the prediction model is determined to be in the stable working area; otherwise, it is marked as an abnormal fluctuation state, forming a binary state flag used to drive the switching of sampling rate and compensation mode in S8.

[0190] Based on the set of prediction error evaluation values ​​generated by S7.3, a dual-threshold conditional discrimination algorithm (parameters: RMSE first threshold θ1, Max Deviation second threshold θ2) is used to classify and determine the accuracy status of the cutting piece boundary prediction model within the current cutting cycle.

[0191] Furthermore, through the root mean square error comparison calculation module (parameter: θ1 is set based on process accuracy tolerance), the following is achieved: The conditions are matched and filtered, and a judgment flag is obtained that satisfies the first precision condition.

[0192] Furthermore, the maximum offset limit detection module (parameter: θ2 is set based on the mechanical trajectory deviation tolerance) is used to match and verify the conditions, and generate a judgment flag bit that satisfies the second stability condition.

[0193] Furthermore, through a multi-condition logic convergence arithmetic unit (logic mode: AND), the above precision conditions and stability conditions are jointly judged, and a Boolean result True is generated for the stable working area state of the prediction model; if any condition is not met, a Boolean result False is generated for the abnormal fluctuation state.

[0194] The binary state coding module maps the Boolean result True to the state code "0" and the Boolean result False to the state code "1", forming a binary state flag data packet to drive the switching logic of inference sampling rate and compensation mode in S8, thereby realizing the rapid switching of the pruning path prediction model's operating mode.

[0195] For example, on a smart bag cutting production line, the system sets the first RMSE threshold θ1 as... millimeters, the second threshold θ2 of MaxDeviation is Millimeters. In the current periodic forecast error assessment, RMSE is... The root mean square (RMS) calculation module (where n is the number of point cloud samples, and the error value is in millimeters) yields the following results: millimeters, to meet Max Deviation is calculated by taking the maximum value of the modulus of the pointwise matching vector field. The millimeter value is met. The logic concatenation arithmetic unit outputs True, corresponding to status code "0". The system enters low-load mode in S8, and the inference sampling interval can be extended from 50ms to 100ms. In another test batch, the RMSE was... millimeters, exceeding θ1, although Max Deviation is When the millimeter matches θ2, the logic converges to output False, corresponding to status code "1". The system enters high-fidelity compensation mode in S8, adapting to scenarios with rapid deformation of cut pieces and significantly improving boundary tracking accuracy.

[0196] S7.5: Write the status flag output from S7.4 and the historical error trend data together into the system status register to generate a timestamped feedback log record, providing traceable data support for subsequent online model fine-tuning and system robustness analysis, and triggering the dynamic adjustment mechanism in S8 to enter the next closed-loop optimization cycle.

[0197] Step S8: Based on the prediction error evaluation values ​​obtained from two consecutive feedback verifications, dynamically adjust the inference frequency and input weighting strategy of the L-KGNN model: if the error is lower than the tolerance, reduce the sampling rate to save computing power; if the error increases, increase the sampling rate and activate the high-fidelity compensation mode to achieve closed-loop adaptive optimization. Specifically, this includes: S8.1: Based on the prediction error assessment value obtained by comparing the predicted profile generated in the previous period with the current multi-camera observation data, calculate the current error change trend and generate an error dynamic identifier as the basis for decision-making on inference frequency adjustment.

[0198] S8.2: Perform differential operation on the prediction error evaluation values ​​of two consecutive cycles to determine whether the error sequence meets the convergence condition; if the error is lower than the preset tolerance threshold in two consecutive cycles, generate a low load mode trigger signal as an operation instruction to reduce the model inference frequency.

[0199] S8.3: Based on the low-load mode trigger signal, perform the inference sampling interval extension operation to adjust the sampling frequency of the input image sequence of the L-KGNN model from once every 50ms to once every 100ms, and generate a sparse temporal input tensor to reduce redundant computational overhead.

[0200] S8.4: If the prediction error evaluation values ​​exceed the tolerance range for two consecutive predictions, a high-fidelity compensation mode activation instruction is generated, and the weighting coefficients of the input features of each view camera are dynamically adjusted based on the attention weight spatial distribution map to enhance the visual data contribution of deformation-sensitive areas and generate an optimized multi-view fusion feature tensor.

[0201] S8.5: Based on the high-fidelity compensation mode activation instruction, the inference cycle of the L-KGNN model is shortened, the frequency of input image sequence acquisition is increased to once every 25ms, and the historical frame buffer is updated in combination with the incremental sliding window mechanism to generate high time-frequency resolution input feature sequences to support fine-grained boundary prediction.

[0202] For those skilled in the art, various other corresponding changes and modifications can be made based on the technical solutions and concepts described above, and all such changes and modifications should fall within the protection scope of the claims of this application.

[0203] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains. The terms “first,” “second,” “third,” and similar terms used in this patent application specification and claims do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an” or “a” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “comprising” or “including” and similar terms mean that the elements or objects preceding “comprising” or “including” encompass the elements or objects listed following “comprising” or “including” and their equivalents, and do not exclude other elements or objects. The “multiple” mentioned in the embodiments of this application refers to two or more. A and / or B indicate three possibilities: A; B; and A and B.

[0204] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-camera collaborative method for 3D pattern alignment and cutting optimization of bags, specifically including: S1: Acquire multi-view image sequences of three-dimensional cut pieces of bags during the transmission and positioning process based on a multi-camera collaborative vision system; S2: Perform denoising and geometric correction processing on the multi-view image sequence to generate a standardized image sequence with consistent spatial registration; S3: Utilize depthwise separable convolution to extract local texture motion features from image sequences at various viewpoints, and combine sparse graph structures to model the topological evolution relationship between key points at the edge of the cropped piece, thus constructing the input feature tensor of the lightweight convolutional-graph neural network hybrid model L-KGNN; S4: Input the input feature tensor into a lightweight convolutional-graph neural network hybrid model L-KGNN trained by knowledge distillation, and output the predicted contour; S4.1: Based on the spatially registered and standardized image sequence generated in the previous steps, a depthwise separable convolutional layer is used to extract spatiotemporal features of local texture motion features from each viewpoint to obtain a high-dimensional sparse feature response map. S4.2: Based on the high-dimensional sparse feature response map obtained in S4.1, identify the set of key points on the edge of the cut piece and construct a sparse graph structure to generate an initial graph data representation; S4.3: Input the initial graph data representation generated in S4.2 into the message passing layer of the graph neural network, perform multiple rounds of node feature aggregation and update operations, use the neighborhood weighted average mechanism to fuse context deformation information, and output an enhanced edge state embedding vector sequence containing global consistency constraints; S4.4: Based on the enhanced edge state embedding vector sequence output by S4.3, combined with the time sliding window mechanism, it is input into the lightweight temporal inference module. The state evolution path between consecutive frames is modeled using a gated recurrent unit architecture to predict the spatial distribution trend of the cut piece boundary points at the next moment and generate a short temporal dynamic contour sequence. S4.5: Performs Bezier curve fitting and geometric smoothing on the short time-domain dynamic contour sequence output by S4.4 to eliminate high-frequency jitter noise and generate a continuous and physically realizable predicted contour of the cut piece boundary within the next 50ms. S5: Based on the difference comparison results between the predicted contour and the original CAD cutting path, identify local abnormal segments with deviations exceeding a set threshold to generate a spatial index set. S5.1: Obtain the predicted contour point cloud data of the cut piece boundary within the next 50ms output by the lightweight convolutional-graph neural network hybrid model L-KGNN, as well as the vector curve data of the original CAD cutting path that is pre-imported, as input objects for differential comparison; use the spatial coordinate unified alignment algorithm to map the two types of path data to the same mechanical coordinate system, eliminate the registration error caused by the viewpoint transformation, and generate a spatiotemporally aligned dual-path comparison benchmark; S5.2: Based on the aligned predicted contour and the original CAD path, perform segmented Euclidean distance calculation to obtain the spatial deviation sequence between each path sampling point; use the sliding window method to perform local mean filtering on the deviation sequence to suppress misjudgment caused by isolated noise points and generate a smoothed path deviation distribution curve as the basic input for abnormal segment identification. S5.3: Based on the preset dynamic deviation threshold, the smoothed deviation distribution curve is binarized and discriminated, and continuous path segments whose deviations continuously exceed the threshold are marked; combined with the minimum effective length constraint, local abnormal segments with actual impact are identified, and their start and end index parameter sets in the original path are generated. S5.4: Based on the identified start and end index parameter set of local abnormal segments, extract the original CAD path geometric feature fragments of the corresponding area, and overlay the corresponding predicted contour evolution trend information to construct a replanning requirement description unit containing the context environment; encapsulate each description unit into a path correction task package with a spatial identifier. S5.5: Summarize all path correction task packages and generate a spatial index set for the areas that need to be replanned. This set stores the mechanical coordinates, length range and priority labels of each abnormal segment in a list structure and outputs them to the incremental Bézier fitting algorithm module in the next stage through the communication interface. S6: For the set of spatial indexes, the incremental Bezier fitting algorithm is called in combination with the physical constraint rule library to generate a local alternative trajectory that satisfies the mechanical dynamics characteristics, forming an adaptively adjusted clipping path for optimization.

2. The method for multi-camera collaborative alignment and cutting optimization of three-dimensional pattern pieces for bags according to claim 1, characterized in that, Following S6, it also includes: S7: During the execution of the adjusted path, the observation data of the multi-camera system in the next cycle is received in real time, the consistency between the predicted contour and the actual boundary is checked, and the prediction error evaluation value is generated. S8: Based on the prediction error evaluation value obtained from two consecutive feedback verifications, dynamically adjust the inference frequency and input weighting strategy of the L-KGNN model: if the error is lower than the tolerance, reduce the sampling rate; if the error increases, increase the sampling rate and activate the high-fidelity compensation mode.

3. The method for multi-camera collaborative alignment and cutting optimization of three-dimensional cutting pieces for bags according to claim 1, characterized in that, The S1 specifically includes: S1.1: Deploy a circumferential vision array consisting of multiple wide-angle industrial cameras, which are distributed at different heights and orientations around the transport path of the bag 3D cut piece according to a preset spatial angle. The field of view of each camera covers all key areas of the 3D curved surface of the cut piece, generating a physical layout structure for multi-view synchronous imaging. S1.2: Perform hardware-level synchronization triggering configuration on each industrial camera in the circumferential vision array, control the image acquisition time based on a globally unified clock signal, and generate a multi-view high frame rate image sequence with strict spatiotemporal alignment; S1.3: Capture the texture details of the cut surface under complex lighting conditions using high dynamic range imaging mode, perform pixel-level gain compensation and non-uniformity correction on the original image, and generate a multi-view video stream with enhanced lighting stability. S1.4: Based on the pre-calibrated camera intrinsic and extrinsic rotation and translation matrices, epipolar geometric correction and depth map fusion processing are performed on multi-view image sequences to map heterogeneous images to a unified three-dimensional point cloud coordinate system and generate a standardized multi-view video stream with consistent spatial registration. S1.5: Pack the spatially registered multi-view high frame rate image sequence into a structured video data stream by timestamp and attach metadata tags to obtain raw perceptual data with spatiotemporal dual-domain alignment characteristics.

4. The method for multi-camera collaborative alignment and cutting optimization of three-dimensional pattern pieces for bags according to claim 1, characterized in that, The S2 specifically includes: S2.1: Based on the original high frame rate image sequence output by the multi-camera collaborative vision system, acquire the RAW format pixel data of each frame image, and read the intrinsic parameter matrix and extrinsic parameter calibration parameters of the corresponding camera; use the camera calibration parameters to perform demosaicing and radiometric correction processing on the original image. S2.2: Apply the inverse lens distortion transformation algorithm based on the radial-tangential model to the radiometrically corrected image data. Perform nonlinear mapping correction on the pixel coordinates according to the distortion coefficient set corresponding to each camera to compensate for barrel distortion caused by wide-angle imaging. After correction, use bilinear interpolation to reconstruct the pixel grid and generate a corrected image frame sequence with geometric distortion suppression. S2.3: Based on the global synchronization timestamp of the multi-camera system and the known extrinsic rotation and translation matrix, construct the three-dimensional reprojection relationship under the unified world coordinate system; backproject the feature point set in the corrected image frame of each viewpoint to the common three-dimensional space through the perspective projection equation, and reproject it to the virtual standard view plane, perform image registration operation based on Homography transformation, and generate a spatially normalized image sequence with viewpoint alignment. S2.4: For the spatially registered multi-view image sequence, detect the illumination intensity drift between adjacent frames and calculate the gradient of the mean brightness change of each image block; perform nonlinear stretching adjustment of local contrast based on the adaptive histogram equalization algorithm, and combine grayscale reference field modeling technology to perform piecewise linear fitting compensation of the overall illumination field to generate illumination-stability-enhanced image data without illumination fluctuations. S2.5: The multi-view image frames that have completed geometric correction, spatial registration and illumination normalization are indexed and organized by timestamp and camera ID, and encapsulated into a standardized image tensor with uniform spatial resolution and color space standard; the standardized image sequence is output as the input condition for the subsequent depthwise separable convolutional feature extraction module.

5. The method for multi-camera collaborative alignment and cutting optimization of three-dimensional pattern pieces for bags according to claim 1, characterized in that, The S3 specifically includes: S3.1: Based on the geometrically corrected multi-view high frame rate image sequence, a channel-wise spatial convolution operation is performed on each frame image using a depthwise separable convolution kernel to extract local texture gradient and motion direction features, and generate a low-dimensional local feature map to reduce computational redundancy and retain texture dynamic information. S3.2: Perform key point detection on the local feature map, use the Harris-Laplace operator to identify significant feature points with scale invariance in the edge region of the cut piece, and generate a set of spatial key points containing coordinate positions and local response intensities; S3.3: Based on the set of spatial key points, construct a sparse Euclidean distance adjacency matrix, set a dynamic threshold to connect spatially adjacent key points from adjacent viewpoints, and form a sparse graph structure that is connected across viewpoints; S3.4: Assign the feature vectors in the local feature map to each node of the sparse graph structure according to their corresponding positions to construct a graph signal input with attribute features; S3.5: Perform time-dimensional stacking processing on the graph signal input, integrate the multi-frame graph structure data within the continuous time window, and generate a four-dimensional feature tensor.

6. The method for multi-camera collaborative alignment and cutting optimization of three-dimensional cutting pieces for bags according to claim 3, characterized in that, The epipolar geometry correction, depth map fusion, and spatial registration include stereo matching of image features and depth disparity maps, outputting a dense 3D point cloud, and transforming it to a unified world coordinate system using an extrinsic parameter matrix.

7. The method for multi-camera collaborative alignment and cutting optimization of three-dimensional pattern pieces for bags according to claim 1, characterized in that, The denoising and geometric correction processing of the multi-view image sequence also includes: demosaicing, radiometric correction, gain equalization, radial-tangential distortion compensation of RAW pixel data, and spatial registration based on Homography to form a tensor input with a unified color space standard.

8. The method for multi-camera collaborative alignment and cutting optimization of three-dimensional pattern pieces for bags according to claim 1, characterized in that, The edge key point detection employs Harris-Laplace, Canny edge thinning, K-nearest neighbor pairing, and multi-view spatial correction, combined with a dynamic threshold of Euclidean distance and the total number of key points.

Images