Robot autonomous disassembly method and system based on multi-source visual perception

Abstract

The present invention discloses a method and system for autonomous robotic disassembly based on multi-source visual perception, relating to the field of biopharmaceutical technology. The method comprises the following steps: obtaining initial image data of a target bagged product, including a color image acquired by an RGB camera and a depth image acquired by a depth camera, and marking each with an acquisition timestamp; synchronizing the image data based on the timestamps, and matching the spatial position information of images from different sources at the same moment by constructing a unified time coordinate system. By integrating multi-source visual image synchronization, three-dimensional modeling and trajectory control, and residue detection technologies, the present invention enables high-precision autonomous robotic disassembly of biopharmaceutical bagged products, resolves the problem of misoperation caused by image timing asynchrony, improves system stability and cleanliness control capabilities, and possesses excellent application value.

Description

Technical Field

[0001] The present invention relates to the field of biopharmaceutical technology, and in particular to a robot autonomous disassembly method and system based on multi-source visual perception. Background Art

[0002] The "robotic autonomous disassembly method based on multi-source visual perception" refers to the use of a combination of visual sensing technologies (such as RGB cameras, depth cameras, and infrared thermal imaging) in the biopharmaceutical field to collect information such as the shape, position, and material of bagged products. This information is then identified and judged using artificial intelligence algorithms, enabling robots to perceive the environment, make dynamic decisions, and perform precise operations. This enables the automated positioning, grasping, unsealing, and sorting of sealed bagged items (such as sterile and pharmaceutical bags). This method can be widely applied to the loading process before sterilization or sorting in pharmaceutical production lines, addressing issues such as low manual bag unpacking efficiency, high contamination risks, and inconsistent operations, significantly improving production cleanliness, automation, and efficiency.

[0003] Existing technologies have the following shortcomings: In existing robotic autonomous disassembly processes based on multi-source visual perception, the timing asynchrony of multi-source visual information is an often overlooked but significant issue. For example, in the disassembly of multi-layer sealed bags in the biopharmaceutical industry, if the data collected by the RGB camera and the depth camera are not aligned in real time, the system may mistakenly determine that the bag is stable when, in fact, it is still slightly wobbling. In this situation, if the robot performs a tearing or cutting operation, it is very likely to cause uneven damage to the bag, or even accidentally damage the sterile layer of the drug inside, leading to serious consequences such as contamination, product scrapping, and entire production line shutdown.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0005] The purpose of the present invention is to provide a robot autonomous disassembly method and system based on multi-source visual perception. Through the integration of multi-source visual image synchronization, three-dimensional modeling and trajectory control, residue detection and other technologies, the present invention realizes high-precision autonomous disassembly of biopharmaceutical bagged products by robots, solves the problem of misoperation caused by image timing asynchrony, improves system stability and cleanliness control capabilities, and has good application value to solve the problems in the above-mentioned background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a robot autonomous disassembly method and system based on multi-source visual perception, comprising the following steps:

[0007] Obtain initial image data of the target bagged product, including color images captured by an RGB camera and depth images captured by a depth camera, and mark the acquisition timestamps for each.

[0008] Synchronize image data based on timestamps and build a unified time coordinate system to match the spatial location information of images from different sources at the same time.

[0009] The synchronized image data is fused and processed to extract the three-dimensional contour information and dynamic state characteristics of the bag, and determine whether the target is in a stable and static state;

[0010] Under the premise that the target bag is stable and still, a 3D model positioning operation based on the fused image is performed to determine the initial starting point and end point of the tearing path;

[0011] Control the robot to perform disassembly actions along a fixed trajectory, cutting with constant pressure and preset speed, and maintaining path stability during operation through feedback control;

[0012] After disassembly is completed, the visual system is called again to perform integrity verification on the disassembly area. After confirming that the packaging bag is completely removed and there is no risk of internal contamination, the product is transferred to the next process.

[0013] Preferably, the acquisition frequencies of the RGB image and the depth image are set to 60 frames per second and 30 frames per second respectively, and the time points of the depth image are interpolated to the time points of the RGB image by a linear interpolation algorithm to ensure that there are matching data points at every moment in the fusion processing stage;

[0014] To ensure synchronization accuracy, a timestamp difference threshold is applied before the image data enters the processing module. If the time difference between any pair of images exceeds the set threshold of 20 milliseconds, the image data of that pair is discarded and re-collected to ensure the consistency of image input and processing accuracy.

[0015] In addition, a sliding window mechanism is set up in the image frame data cache area to perform alignment and matching within the cache queue, thereby improving the success rate of image synchronization and ultimately ensuring the consistency of subsequent 3D modeling data.

[0016] Preferably, a synchronous alignment step of the image data is performed based on the timestamp, a unified time coordinate system is constructed, and a master clock module is used to uniformly schedule the sampling behavior of all visual sensors;

[0017] The master clock sends a global sampling trigger signal with millisecond-level accuracy. All image sensors immediately execute image acquisition after responding to the trigger signal and embed the current timestamp in the acquired data to ensure that each data source maintains logical time consistency.

[0018] The collected color images and depth images are sorted in time sequence respectively, and a sliding window matching algorithm is used to find the depth image frame with the closest timestamp for each color image frame. If the time difference is within the set threshold, the image is included in the fusion processing queue, otherwise it is marked as invalid data and discarded;

[0019] To further improve the accuracy of spatial alignment, spatial correction of image resolution and field of view is performed after each pair of images are matched. The images are remapped using the pre-established camera internal and external parameter matrices. Finally, multi-source visual fusion input data is constructed under a unified time axis and a unified spatial reference to ensure consistency and accuracy in subsequent image analysis and processing.

[0020] Preferably, the extraction of three-dimensional contour information adopts a deep learning algorithm based on point cloud fusion, and the steps are as follows:

[0021] The RGB image and depth map data are respectively extracted through the feature extraction network to extract key area features, construct a joint feature vector, and then use the spatial coordinate transformation model to restore each pixel point in the depth map to a three-dimensional spatial coordinate point to generate a dense point cloud model;

[0022] A convolutional neural network is used to judge the posture of the point cloud model, and the stability confidence threshold is used as the basis to determine whether the stable state has been reached;

[0023] When the confidence level exceeds 0.92, the bag is judged to be in an operable state, otherwise it waits for the stationary state to be tested again.

[0024] Preferably, the determination of the stable static state of the bag body is based on a dynamic image frame difference analysis algorithm, which quantifies the movement trend of the bagged product in a short time window by performing frame-by-frame difference calculation on the continuously collected RGB image sequence. Suppose that the continuous N frames of RGB images are {I1, I2, ..., I N Each frame contains M pixels. The pixel difference between each two frames is normalized and calculated to obtain the frame difference value, which is used to measure the stability of the current bag surface texture and edge position. The frame difference value calculation expression is as follows:

[0025]

[0026] , where D i is the average frame difference between the i-th frame and the i+1-th frame, I i (f) is the grayscale value of the f-th pixel in the i-th frame image, i i+1 (f) is the grayscale value of the f-th pixel in the i+1-th frame image;

[0027] A residual sequence is constructed for all frame differences and the standard deviation is calculated to identify the consistency of image changes between frames. The standard deviation calculation formula is as follows:

[0028]

[0029] , where is the average value of the frame difference sequence, σ D is the standard deviation of the frame difference sequence, reflecting the stability of inter-frame changes;

[0030] If σ D <θ1, and It is determined that the bag is in a stable state and meets the prerequisite for performing the cutting operation, where θ1 is the judgment threshold of the frame difference standard deviation, and θ2 is the judgment threshold of the average frame difference.

[0031] Preferably, the cutting path is determined using a composite algorithm based on edge detection and structural texture analysis, and the specific steps are as follows:

[0032] Use the Canny algorithm to perform preliminary edge extraction on the fused image;

[0033] The noise and false edges are removed by morphological closing operation to obtain a stable bag contour line;

[0034] The sealing position of the bag is determined based on the image texture gradient direction, and the sealing line is used as the starting point to input into the path planning module;

[0035] The path planning module uses the curvature minimization principle to calculate the optimal cutting trajectory, ensuring that the trajectory extends along the sealing direction and avoids cutting areas outside the packaging bag.

[0036] Preferably, the robot control system performs trajectory planning based on a vision-guided model and uses an adaptive trajectory calibration mechanism;

[0037] During the trajectory calibration process, a feedback controller is introduced to compare the deviation between the actual tool head path and the predetermined path. If the deviation exceeds 1.5 mm, the cutting direction is adjusted in real time and the constant pressure is adjusted synchronously.

[0038] Use PID control algorithm to continuously adjust the angle of the three-axis robot arm to ensure that the trajectory cutting is stable within the set error threshold range, with a maximum allowable error of 2 mm;

[0039] When the robot performs the disassembly operation, it uses a force control feedback system to dynamically adjust the pressure parameters. The specific steps are as follows:

[0040] During the cutting process, the blade pressure sensor data is collected every 5 milliseconds to determine the deviation between the current contact force and the preset value in real time;

[0041] If the current value is too high, the power supply voltage of the actuator motor is reduced to reduce the contact force of the cutter head; if it is too low, the voltage is increased to maintain a constant cutting depth;

[0042] At the same time, an anti-interference filtering mechanism is introduced to filter out noise data caused by micro-vibration to ensure that the feedback results are stable and effective.

[0043] Preferably, the three-dimensional model positioning and path control adopts a high-order path smoothing function and combines it with an error feedback control mechanism to ensure that the robot executes along the predetermined trajectory with high precision when disassembling the bagged products. The specific deficiencies are as follows:

[0044] Extract the three-dimensional path point set P, P = {p j}={p1, p2, ..., p n}, where p j is the jth discrete path point in the three-dimensional path, n is the total number of discrete path points, and each discrete path point is defined as p j =(x j ,y j , z j ), x j 、y j and z j Represent the coordinates of the jth discrete path point in the x-axis, y-axis and z-axis directions respectively;

[0045] To achieve smooth motion of the robot in three-dimensional space, a spline interpolation path function is constructed based on a set of path points to provide a continuous and differentiable path curve for the robot end effector to track. The formula of the spline interpolation path function is as follows:

[0046]

[0047] , where B j (t) is the B-spline basis function corresponding to the j-th discrete path point, a j is the interpolation weight coefficient corresponding to the jth discrete path point, S(t) is the spline interpolation path function;

[0048] During the robot motion execution process, the actual spatial position coordinates (x r 、y r 、z r ), and solve the ideal trajectory point (x(t), y(t), z(t)) corresponding to time t from the path function, and calculate the current execution error using the Euclidean distance formula. The calculation expression is as follows:

[0049]

[0050] , where x r 、y r 、z rIt is the actual three-dimensional coordinate position of the robot end effector fed back in real time by the encoder and position sensor equipment. x(t), y(t), and z(t) represent the three-dimensional theoretical coordinate position corresponding to the spline interpolation path function S(t) at time point t, which is used to compare with the current position of the robot. Δ(t) is the Euclidean distance error between the actual position of the robot in the current motion state and the theoretical path position.

[0051] When Δ(t) exceeds the set threshold δ, it is determined that there is a path deviation and feedback correction needs to be performed;

[0052] In the error offset state, the direction angle correction logic is started, and the attitude adjustment angle in the three-dimensional direction is calculated according to the local geometric change trend between continuous path points, and the direction angle vector θ=(θ x ,θ y ,θ z ), the calculation formula for each direction angle is as follows:

[0053]

[0054] , where is a discrete path point p j The coordinate value in the kth dimension, is a discrete path point p j+1 The coordinate value in the kth dimension, d is the average Euclidean distance between consecutive path points, θ k Represents the angle change in the k-axis direction, which is used to correct the direction of movement, k∈{x, y, z};

[0055] The angle vector is transmitted to the motion solver module in the robot controller, which drives it to adjust the current joint angle so that the end effector posture is realigned with the predetermined path direction, achieving accurate tracking of the cutting path.

[0056] Preferably, the final integrity check is completed by visual inspection combined with residual quality control, and the specific steps are as follows:

[0057] The visual inspection part uses a template matching algorithm to compare the pixel differences of the bag boundary before and after disassembly. If the boundary residual image exceeds the set threshold, the "disassembly incomplete" status is prompted;

[0058] The quality control part uses infrared imaging to detect whether there are residual particles or liquid reflective features on the workbench surface, and converts the infrared reflection intensity into a binary image for residue identification;

[0059] After combining the two parts of the results, a weighted confidence algorithm is used to comprehensively judge whether the disassembly is successful. The product transfer process will only be started when both the visual and quality control results are qualified.

[0060] Preferably, the robot autonomous disassembly system based on multi-source visual perception includes an image acquisition module, an image synchronization module, an image fusion and state determination module, a three-dimensional positioning and path planning module, a trajectory execution and force control feedback module, and a result detection and qualification determination module:

[0061] The image acquisition module obtains the initial image data of the target bagged product, including the color image acquired by the RGB camera and the depth image acquired by the depth camera, and marks the acquisition timestamp for each;

[0062] The image synchronization module synchronizes image data based on timestamps and builds a unified time coordinate system to match the spatial position information of images from different sources at the same time.

[0063] The image fusion and state determination module performs fusion processing on the synchronized image data, extracts the three-dimensional contour information and dynamic state characteristics of the bag, and determines whether the target is in a stable and stationary state;

[0064] The 3D positioning and path planning module, under the premise of detecting that the target bag is stable and stationary, performs a 3D model positioning operation based on the fused image to determine the initial starting point and end point of the tearing path;

[0065] The trajectory execution and force control feedback module controls the robot to execute the disassembly action of a fixed trajectory, uses constant pressure and preset speed for cutting, and maintains the path stability during the operation through feedback control;

[0066] After the disassembly is completed, the result detection and qualification judgment module calls the visual system again to perform integrity verification on the disassembly area. After confirming that the packaging bag is completely removed and there is no risk of internal contamination, the product is transferred to the next process.

[0067] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0068] The present invention constructs an image synchronization mechanism based on multi-source visual perception, integrates three-dimensional modeling and dynamic trajectory planning control, and combines a full-process intelligent disassembly method with residue detection and quality confirmation. This enables robots to perform high-precision, safe, and automated disassembly operations on bagged products in the biopharmaceutical field, effectively solving the problem of misjudgment caused by the lack of synchronization in the time series of multi-source images, fundamentally reducing the risk of product contamination and the probability of operational errors, and significantly improving the stability and controllability of the disassembly process. It has good industrial application prospects and promotion value. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction to the drawings required for use in the embodiments will be given below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0070] Figure 1 This is a flow chart of the method for autonomous robot disassembly based on multi-source visual perception of the present invention.

[0071] Figure 2 This is a module schematic diagram of the robot autonomous disassembly system based on multi-source visual perception of the present invention. DETAILED DESCRIPTION

[0072] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0073] The present invention provides Figure 1 The robot autonomous disassembly method based on multi-source visual perception shown includes the following steps:

[0074] Obtain initial image data of the target bagged product, including color images captured by an RGB camera and depth images captured by a depth camera, and mark the acquisition timestamps for each.

[0075] The acquisition frequencies of RGB images and depth images are set to 60 frames per second and 30 frames per second, respectively. The time points of the depth images are interpolated to the time points of the RGB images through a linear interpolation algorithm to ensure that there are matching data points at every moment in the fusion processing stage.

[0076] To ensure synchronization accuracy, a timestamp difference threshold is applied before the image data enters the processing module. If the time difference between any pair of images exceeds the set threshold of 20 milliseconds, the image data of that pair is discarded and re-collected to ensure the consistency of image input and processing accuracy.

[0077] In addition, a sliding window mechanism is set up in the image frame data cache area to perform alignment and matching within the cache queue, thereby improving the success rate of image synchronization and ultimately ensuring the consistency of subsequent 3D modeling data.

[0078] Synchronize image data based on timestamps and build a unified time coordinate system to match the spatial location information of images from different sources at the same time.

[0079] Perform synchronization alignment of image data based on timestamps, build a unified time coordinate system, and use a master clock module to uniformly schedule the sampling behavior of all visual sensors;

[0080] The master clock sends a global sampling trigger signal with millisecond-level accuracy. All image sensors immediately execute image acquisition after responding to the trigger signal and embed the current timestamp in the acquired data to ensure that each data source maintains logical time consistency.

[0081] The collected color images and depth images are sorted in time sequence respectively, and a sliding window matching algorithm is used to search for the depth image frame with the closest timestamp for each color image frame. If the time difference is within the set threshold (for example, less than 10 milliseconds), the image is included in the fusion processing queue; otherwise, it is marked as invalid data and discarded.

[0082] To further improve the accuracy of spatial alignment, spatial correction of image resolution and field of view is performed after each pair of images are matched. The images are remapped using the pre-established camera internal and external parameter matrices. Finally, multi-source visual fusion input data is constructed under a unified time axis and a unified spatial reference to ensure consistency and accuracy in subsequent image analysis and processing.

[0083] The synchronized image data is fused and processed to extract the three-dimensional contour information and dynamic state characteristics of the bag, and determine whether the target is in a stable and static state;

[0084] The extraction of 3D contour information adopts a deep learning algorithm based on point cloud fusion, and the steps are as follows;

[0085] The RGB image and depth map data are respectively extracted through the feature extraction network to extract key area features, construct a joint feature vector, and then use the spatial coordinate transformation model to restore each pixel point in the depth map to a three-dimensional spatial coordinate point to generate a dense point cloud model;

[0086] A convolutional neural network (CNN) is used to judge the posture of the point cloud model, and the stability confidence threshold is used as the basis to determine whether the stable state has been reached;

[0087] When the confidence level exceeds 0.92, the bag is judged to be in an operable state, otherwise it waits for the stationary state to be tested again.

[0088] The above-mentioned deep point cloud fusion mechanism is used to improve the accuracy and safety of the disassembly action.

[0089] The judgment of the stable and static state of the bag body is based on the dynamic image frame difference analysis algorithm, which specifically includes the construction of a combined judgment mechanism of the dynamic segment difference matrix and the average frame difference index. By performing frame-by-frame difference calculation on the continuously collected RGB image sequence, the movement trend of the bagged product in a short time window is quantified. Suppose that the continuous N frames of RGB images are {I1, I2, ..., I N Each frame contains M pixels. The pixel difference between each two frames is normalized and calculated to obtain the frame difference value, which is used to measure the stability of the current bag surface texture and edge position. The frame difference value calculation expression is as follows:

[0090]

[0091] , where D i is the average frame difference between the i-th frame and the i+1-th frame, I i (f) is the grayscale value of the f-th pixel in the i-th frame image, which represents the light intensity of the image at a certain frame and a certain pixel point, and is used for basic judgment of image changes between frames. i+1 (f) is the grayscale value of the f-th pixel in the i+1-th frame image;

[0092] A residual sequence is constructed for all frame differences and the standard deviation is calculated to identify the consistency of image changes between frames. The standard deviation calculation formula is as follows:

[0093]

[0094] , where is the average value of the frame difference sequence, that is, all D i The arithmetic mean, σ D It is the standard deviation of the frame difference sequence, reflecting the stability of the inter-frame changes. For each frame difference D i Average frame difference If the standard deviation is very small, it means that the target changes tend to be stable and the movement amplitude is low.

[0095] If σ D <θ1, and It is determined that the bag is in a stable state and meets the prerequisite for performing the cutting operation, where θ1 is the judgment threshold of the frame difference standard deviation, and θ2 is the judgment threshold of the average frame difference.

[0096] For example, in a typical application, setting θ1 = 0.08 and θ2 = 2.5 ensures a false positive probability of less than 0.5%. This stability judgment mechanism effectively filters out false triggering caused by bag shaking, vibration, or environmental interference, improving the decision-making reliability and production safety of the disassembly system.

[0097] Under the premise that the target bag is stable and still, a 3D model positioning operation based on the fused image is performed to determine the initial starting point and end point of the tearing path;

[0098] The cutting path is determined using a composite algorithm based on edge detection and structural texture analysis. The specific steps are as follows:

[0099] Use the Canny algorithm to perform preliminary edge extraction on the fused image;

[0100] The noise and false edges are removed by morphological closing operation to obtain a stable bag contour line;

[0101] The sealing position of the bag is determined based on the image texture gradient direction, and the sealing line is used as the starting point to input into the path planning module;

[0102] The path planning module uses the curvature minimization principle to calculate the optimal cutting trajectory, ensuring that the trajectory extends along the sealing direction and avoids cutting areas outside the packaging bag.

[0103] Through the above image structure analysis and path optimal control algorithm, automatic, accurate and lossless bag unloading path identification is achieved.

[0104] Control the robot to perform disassembly actions along a fixed trajectory, cutting with constant pressure and preset speed, and maintaining path stability during operation through feedback control;

[0105] The robot control system performs trajectory planning based on a vision-guided model and uses an adaptive trajectory calibration mechanism;

[0106] During the trajectory calibration process, a feedback controller is introduced to compare the deviation between the actual tool head path and the predetermined path. If the deviation exceeds 1.5 mm, the cutting direction is adjusted in real time and the constant pressure is adjusted synchronously.

[0107] The PID control algorithm is used to continuously adjust the angle of the three-axis robot arm to ensure that the trajectory cutting is stable within the set error threshold range, with a maximum allowable error of 2 mm.

[0108] When executing the path, the robot will also dynamically update its end point coordinates based on the current visual feedback to prevent cutting errors caused by slight displacement of the bag.

[0109] When the robot performs the disassembly operation, it uses a force control feedback system to dynamically adjust the pressure parameters. The specific steps are as follows:

[0110] During the cutting process, the blade pressure sensor data is collected every 5 milliseconds to determine the deviation between the current contact force and the preset value in real time;

[0111] If the current value is too high, the power supply voltage of the actuator motor is reduced to reduce the contact force of the cutter head; if it is too low, the voltage is increased to maintain a constant cutting depth;

[0112] At the same time, an anti-interference filtering mechanism is introduced to filter out noise data caused by micro-vibration to ensure that the feedback results are stable and effective.

[0113] This force control mechanism can effectively reduce the risk of interference to internal products during the cutting process and improve disassembly accuracy.

[0114] The 3D model positioning and path control uses a high-order path smoothing function combined with an error feedback control mechanism to ensure that the robot executes along the predetermined trajectory with high precision when disassembling bagged products. The specific deficiencies are as follows:

[0115] Extract the three-dimensional path point set P, P = {p j}={p1, p2, ..., p n}, where p j is the jth discrete path point in the three-dimensional path, n is the total number of discrete path points, which comes from the structural edge coordinate extraction after fusion visual image processing. The set is automatically screened from the point cloud data generated by fusion RGB image and depth map. Each discrete path point is defined as p j =(x j ,y j , z j ), x j 、y j and z j Represent the coordinates of the jth discrete path point in the x-axis, y-axis and z-axis directions respectively;

[0116] The three-dimensional path point set covers the edge seal of the bag as the target area, and the cutting start and end positions are identified through the structural boundary enhancement model to construct a complete cutting path;

[0117] To achieve smooth motion of the robot in three-dimensional space, a spline interpolation path function is constructed based on a set of path points to provide a continuous and differentiable path curve for the robot end effector to track. The formula of the spline interpolation path function is as follows:

[0118]

[0119] , where B j (t) is the B-spline basis function corresponding to the j-th discrete path point. The B-spline basis function is a piecewise defined polynomial function used to construct a smooth curve within its definition interval. jis the interpolation weight coefficient corresponding to the jth discrete path point, which controls the path shape during the spline interpolation process. S(t) is the spline interpolation path function. As t changes from 0 to 1, it represents the spatial trajectory that the robot end effector should move. t∈[0,1] is the normalized time parameter of the path, which is used for continuous interpolation control from the path starting point (t=0) to the end point (t=1).

[0120] Spline interpolation path functions typically use B-spline functions or cubic splines (CubicSpline) as their mathematical expression. In practical applications, especially in robot path planning, cubic B-spline (CubicB-spline) is the most common form of interpolation path function. Its characteristics are: each curve segment has continuous first-order and second-order derivatives at the starting point and end point, ensuring smooth transitions and suitability for dynamic motion control. In addition, variants such as natural spline (Naturalspline), Bezier spline, or Catmull-Rom spline can also be used, but B-spline has more obvious advantages in flexibility and local controllability, and is therefore widely used in precision control fields such as industrial robots, advanced modeling, and medical image reconstruction.

[0121] B-spline basis function (B-splineBasisFunction) is the mathematical basis for constructing B-spline curves. It is a set of piecewise polynomial functions with local support. Each basis function B j (t) corresponds to a control point, and its non-zero intervals are limited to fixed "knots." This allows for localization: a single basis function affects only a small segment of the curve. This property allows adjustments to a single control point to remain constant across the entire path, enabling local optimization and real-time corrections. This approach is well-suited for robot path generation in dynamic environments.

[0122] During the robot motion execution process, the actual spatial position coordinates (x r 、y r 、z r ), and solve the ideal trajectory point (x(t), y(t), z(t)) corresponding to time t from the path function, and calculate the current execution error using the Euclidean distance formula. The calculation expression is as follows:

[0123]

[0124] , where x r 、y r 、z rIt is the actual three-dimensional coordinate position of the robot end effector (such as the tool head) fed back in real time by the encoder and position sensor equipment. x(t), u(t), and z(t) represent the three-dimensional theoretical coordinate position corresponding to the spline interpolation path function S(t) at time point t, which is used for comparison with the current position of the robot. Δ(t) is the Euclidean distance error between the actual position of the robot in the current motion state and the theoretical path position.

[0125] When Δ(t) exceeds the set threshold δ (e.g., 2 mm), it is determined that there is a path deviation and feedback correction needs to be performed;

[0126] This error control strategy ensures that the trajectory executed by the robot strictly adheres to the path planning curve, and avoids cutting offset and damage to the bag edge or contents due to the accumulation of small displacements.

[0127] In the error offset state, the direction angle correction logic is started, and the attitude adjustment angle in the three-dimensional direction is calculated according to the local geometric change trend between continuous path points, and the direction angle vector θ=(θ x ,θ y ,θ z ), the calculation formula for each direction angle is as follows:

[0128]

[0129] is a discrete path point p j The coordinate value in the kth dimension (i.e., the x, y, and z directions), is a discrete path point p j+1 The coordinate value in the kth dimension (i.e., x, y, and z directions), j∈{1, 2, ..., n-1}, d is the average Euclidean distance between consecutive path points, which is used to normalize the angle calculation to avoid the problem of infinite angle magnification when the path spacing is small, θ k Represents the angle change in the k-axis direction (x, y, z), which is used to correct the direction of movement, k∈{x, y, z};

[0130] The angle vector is transmitted to the motion solver module in the robot controller, which drives it to adjust the current joint angle so that the end effector posture is realigned with the predetermined path direction, achieving accurate tracking of the cutting path.

[0131] Through the aforementioned path interpolation function modeling, real-time error feedback calculation, and angle fine-tuning mechanism, this invention significantly improves the spatial accuracy and dynamic robustness of the unpacking operation, effectively avoiding unpacking failures caused by accumulated path errors. This approach is particularly suitable for automated unpacking scenarios in the biopharmaceutical field, where cleanliness and product integrity are crucial. While ensuring unpacking efficiency, this mechanism also enhances the system's adaptability to complex bag shapes and improves the success rate of task execution.

[0132] After disassembly is completed, the visual system is used again to verify the integrity of the disassembly area. After confirming that the packaging bag is completely removed and there is no risk of internal contamination, the product is transferred to the next process;

[0133] The final integrity check is completed by combining visual inspection and residual quality control. The specific steps are as follows:

[0134] The visual inspection part uses a template matching algorithm to compare the pixel differences of the bag boundary before and after disassembly. If the boundary residual image exceeds the set threshold, the "disassembly incomplete" status is prompted;

[0135] The quality control part uses infrared imaging to detect whether there are residual particles or liquid reflective features on the workbench surface, and converts the infrared reflection intensity into a binary image for residue identification;

[0136] After combining the two parts of the results, a weighted confidence algorithm is used to comprehensively judge whether the disassembly is successful. The product transfer process will only be started when both the visual and quality control results are qualified.

[0137] Implementation Method 1: In this implementation, a stability assessment system based on a multi-source visual data synchronization mechanism is designed to meet the practical application needs of the biopharmaceutical industry for automated disassembly of sterile bagged materials. This system primarily consists of a high-frame-rate RGB industrial camera, a structured light depth camera, a unified timestamp module, a data caching and alignment algorithm module, a 3D reconstruction module, and a stability assessment module. Its core objective is to accurately determine whether the bagged object is stable and stationary before the robot performs the disassembly operation. This prevents serious problems such as misjudgment of the cutting path and damage to the pharmaceutical packaging caused by bag movement, shaking, or tilting.

[0138] First, during the image acquisition phase, an RGB camera and a depth camera are deployed. The RGB camera is used to capture high-resolution color images, while the depth camera is used to obtain spatial depth data of the target object. Because the two types of cameras have different sampling frequencies and operating mechanisms, direct data acquisition can cause time offsets, potentially leading to image synchronization issues and affecting the accuracy of 3D reconstruction and subsequent decision-making. To address this issue, the system integrates a unified clock signal source into the image acquisition module. All image acquisition devices are triggered strictly according to this unified clock, and each frame is stamped with a high-precision timestamp upon acquisition.

[0139] After image acquisition is complete, the image data first enters a cache sliding window mechanism. This mechanism uses timestamps as the primary key and automatically pairs image frames within a short period of time, ensuring that each RGB image frame can find a depth image frame with a time difference of less than 20 milliseconds. If the time difference exceeds a threshold, the paired image is discarded and the system waits for the next frame acquisition. After the sliding window mechanism filters, the image fusion stage begins.

[0140] The image fusion processing module uses a feature matching algorithm to pair texture features in the RGB image with spatial points in the depth image, mapping the 2D image information into a 3D point cloud. This generates a dense 3D model with spatial structure and surface texture information. This 3D model fully restores the actual spatial posture, edge structure, and surface morphology of the current bag.

[0141] After generating the 3D model, the stability determination module continuously receives multiple frames of image data and determines whether the object is stable by comparing the degree of change in the point cloud position in consecutive frames. If the system detects that the spatial changes in consecutive frames are minimal and the fluctuation trend is converging (for example, the bag no longer shakes, rotates, or tilts), it is determined to be in an operable state; otherwise, the cutting process will be carried out only after the bag stabilizes. This mechanism ensures that the robot only performs operations at the safest time, greatly reducing the probability of misoperation and potential contamination risks during the disassembly process.

[0142] Through the above mechanism, this implementation method effectively improves the temporal consistency of visual perception, the accuracy of three-dimensional model construction, and the ability to judge the timing of operation triggering, providing high-reliability premise judgment support for the entire autonomous disassembly system.

[0143] Implementation Method 2: This implementation method addresses critical issues during bagged material unpacking, such as inaccurate path recognition, large operational errors, and strict sterility requirements. It proposes a high-precision cutting path planning and execution mechanism based on multi-source image fusion. Through multi-stage recognition and path optimization, this mechanism achieves precise alignment of the cutting path with the target sealing structure, stable and smooth unpacking, and dynamic adaptive control of cutting pressure and speed during operation. This ensures that the bag is opened intact, safely, and efficiently, preventing drug contamination and material waste.

[0144] First, the image processing component extracts edge features from the RGB image and performs image preprocessing. The system uses the Canny edge detection algorithm for preliminary image processing, extracting a clear outline of the bag. Morphological operations such as dilation and erosion then eliminate noise and virtual edges caused by background interference, uneven packaging texture, or varying lighting. Subsequently, image gradient direction analysis and structural pattern matching methods are used to identify the precise location of the bag seal. This seal line serves as the starting structure for subsequent path planning and is input into the path planning module.

[0145] The path planning module generates multiple path candidates based on the sealing lines identified by structural matching and selects the optimal path using a curvature minimization algorithm to ensure a smooth, continuous, and highly accurate cutting path. This path is then discretized into multiple points and fed into the trajectory control module for quadratic interpolation and transition smoothing, ultimately generating a three-dimensional motion trajectory suitable for industrial robots.

[0146] During the trajectory control phase, the system converts this three-dimensional trajectory into an actual execution action through the robot control unit. During execution, the system activates real-time position detection and pressure feedback mechanisms to ensure that the robot moves stably along the set trajectory. Once the system detects that the displacement of the cutter head deviates from the set path, the controller immediately issues an adjustment command for fine-tuning. At the same time, a force control sensor integrated at the front end of the cutter head monitors the cutting pressure in real time. If the pressure exceeds the set threshold, the drive voltage is adjusted to control the cutting depth and speed to prevent the cutter head from penetrating too deep and damaging the outer packaging of the drug. The entire operation process is carried out in a closed, clean operating area to ensure that no external contamination is introduced during the system operation.

[0147] The technical advantage of this implementation method is that it can realize the linkage of high-precision sealing recognition and dynamic path planning, and has adaptive adjustment capabilities, which greatly improves the execution reliability and operational safety of the robot in uncertain scenarios.

[0148] Implementation Method 3: This implementation method focuses on residue detection and product integrity verification after bagged materials are disassembled. Specifically, in the high-purity biopharmaceutical production environment, this method requires ensuring that the work area after disassembly is free of residual debris, liquid, or particulate contamination. Furthermore, it requires accurate, non-human-involved assessment of disassembly success and drug exposure or contamination. To this end, the system has designed a residue detection mechanism and quality assurance process based on the fusion of visual inspection and infrared sensing to comprehensively enhance the safety and controllability of the disassembly process.

[0149] First, after the robot completes its cutting and disassembly operations, the visual inspection module immediately activates. This module uses a reference image of the target bag taken before the task begins. Using an image template matching algorithm, it compares the edge structure, lighting characteristics, and color distribution of the current image with the pre-disassembly image to determine if any anomalies remain, such as incompletely peeled corners, cracks at the cut, or residual material on the platform surface. To improve detection accuracy, the system employs a multi-angle, multi-light source fill-in structure to eliminate recognition errors caused by shadows and uneven lighting.

[0150] After visual inspection, the system activates the infrared thermal imaging module. This module scans the disassembly area for thermal radiation signatures to identify any suspicious residual liquids, oil stains, or highly reflective particles on the surface. While these substances may be difficult to detect in visible light images, they often appear as areas of temperature differences or abnormal reflectivity in thermal images. The system binarizes detected high-temperature spots or non-background hot spots and compares them to pre-set area templates. If irregular reflective patterns exceeding a threshold are present within the detection area, the area is deemed to be at risk of contamination.

[0151] Ultimately, the system integrates the visual and infrared inspection results into a weighted confidence decision module. This module assigns weights to each inspection metric and comprehensively calculates whether the disassembly has met the standards of "complete," "safe," and "residue-free." If both subsystems return a positive result, the control system signals the product's release. If either test fails, the system triggers an automatic cleanup mechanism or an alarm prompting operator intervention.

[0152] By establishing this dual detection mechanism, this implementation efficiently completes the post-disassembly quality verification process without human intervention, effectively preventing drug contamination and production anomalies, and enhancing the safety closed-loop level of automated operations. This has significant practical application value for high-end pharmaceutical packaging, high-purity environments, and unmanned pharmaceutical production lines.

[0153] The present invention constructs an image synchronization mechanism based on multi-source visual perception, integrates three-dimensional modeling and dynamic trajectory planning control, and combines a full-process intelligent disassembly method with residue detection and quality confirmation. This enables robots to perform high-precision, safe, and automated disassembly operations on bagged products in the biopharmaceutical field, effectively solving the problem of misjudgment caused by the lack of synchronization in the time series of multi-source images, fundamentally reducing the risk of product contamination and the probability of operational errors, and significantly improving the stability and controllability of the disassembly process. It has good industrial application prospects and promotion value.

[0154] The present invention provides Figure 2 The robot autonomous disassembly system based on multi-source visual perception shown in the figure includes an image acquisition module, an image synchronization module, an image fusion and state determination module, a three-dimensional positioning and path planning module, a trajectory execution and force control feedback module, and a result detection and qualification determination module:

[0155] The image acquisition module obtains the initial image data of the target bagged product, including the color image acquired by the RGB camera and the depth image acquired by the depth camera, and marks the acquisition timestamp for each;

[0156] The image synchronization module synchronizes image data based on timestamps and matches the spatial position information of images from different sources at the same time by building a unified time coordinate system;

[0157] The image fusion and state determination module performs fusion processing on the synchronized image data, extracts the three-dimensional contour information and dynamic state characteristics of the bag, and determines whether the target is in a stable and stationary state;

[0158] The 3D positioning and path planning module, under the premise of detecting that the target bag is stable and stationary, performs 3D model positioning operations based on the fused image to determine the initial starting point and end point of the tearing path;

[0159] The trajectory execution and force control feedback module controls the robot to execute the disassembly action of a fixed trajectory, uses constant pressure and preset speed for cutting, and maintains the path stability during the operation through feedback control;

[0160] After the disassembly is completed, the result detection and qualification judgment module calls the visual system again to perform integrity verification on the disassembly area. After confirming that the packaging bag is completely removed and there is no risk of internal contamination, the product is transferred to the next process.

[0161] The robot autonomous disassembly method based on multi-source visual perception provided in an embodiment of the present invention is realized by the above-mentioned robot autonomous disassembly system based on multi-source visual perception. The specific methods and processes of the robot autonomous disassembly system based on multi-source visual perception are detailed in the embodiment of the robot autonomous disassembly method based on multi-source visual perception, which will not be repeated here.

[0162] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0163] The above description is merely illustrative of certain exemplary embodiments of the present invention. It goes without saying that those skilled in the art will be able to modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims.

[0164] It should be noted that, in this document, if there are relational terms such as first and second, etc., they are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0165] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0166] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0167] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0168] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0169] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0170] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0171] The above description is merely illustrative of certain exemplary embodiments of the present invention. It goes without saying that those skilled in the art will be able to modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims.

Claims

1. A robot autonomous disassembly method based on multi-source visual perception, characterized in that: The following steps are involved: Obtain initial image data of the target bagged product, including color images captured by an RGB camera and depth images captured by a depth camera, and mark the acquisition timestamps for each. Synchronize image data based on timestamps and build a unified time coordinate system to match the spatial location information of images from different sources at the same time. The synchronized image data is fused and processed to extract the three-dimensional contour information and dynamic state characteristics of the bag, and determine whether the target is in a stable and static state; Under the premise that the target bag is stable and still, a 3D model positioning operation based on the fused image is performed to determine the initial starting point and end point of the tearing path; Control the robot to perform disassembly actions along a fixed trajectory, cutting with constant pressure and preset speed, and maintaining path stability during operation through feedback control; After disassembly is completed, the visual system is called again to perform integrity verification on the disassembly area. After confirming that the packaging bag is completely removed and there is no risk of internal contamination, the product is transferred to the next process.

2. The robot autonomous disassembly method based on multi-source visual perception according to claim 1 is characterized in that: The acquisition frequencies of RGB images and depth images are set to 60 frames per second and 30 frames per second, respectively. The time points of the depth images are interpolated to the time points of the RGB images through a linear interpolation algorithm to ensure that there are matching data points at every moment in the fusion processing stage. To ensure synchronization accuracy, a timestamp difference threshold is applied before the image data enters the processing module. If the time difference between any pair of images exceeds the set threshold of 20 milliseconds, the image data of that pair is discarded and re-collected to ensure the consistency of image input and processing accuracy. In addition, a sliding window mechanism is set up in the image frame data cache area to perform alignment and matching within the cache queue, thereby improving the success rate of image synchronization and ultimately ensuring the consistency of subsequent 3D modeling data.

3. The robot autonomous disassembly method based on multi-source visual perception according to claim 1 is characterized in that: Perform synchronization alignment of image data based on timestamps, build a unified time coordinate system, and use a master clock module to uniformly schedule the sampling behavior of all visual sensors; The master clock sends a global sampling trigger signal with millisecond-level accuracy. All image sensors immediately execute image acquisition after responding to the trigger signal and embed the current timestamp in the acquired data to ensure that each data source maintains logical time consistency. The collected color images and depth images are sorted in time sequence respectively, and a sliding window matching algorithm is used to find the depth image frame with the closest timestamp for each color image frame. If the time difference is within the set threshold, the image is included in the fusion processing queue, otherwise it is marked as invalid data and discarded; To further improve the accuracy of spatial alignment, spatial correction of image resolution and field of view is performed after each pair of images are matched. The images are remapped using the pre-established camera internal and external parameter matrices. Finally, multi-source visual fusion input data is constructed under a unified time axis and a unified spatial reference to ensure consistency and accuracy in subsequent image analysis and processing.

4. The robot autonomous disassembly method based on multi-source visual perception according to claim 1 is characterized in that: The extraction of 3D contour information adopts a deep learning algorithm based on point cloud fusion, and the steps are as follows; The RGB image and depth map data are respectively extracted through the feature extraction network to extract key area features, construct a joint feature vector, and then use the spatial coordinate transformation model to restore each pixel point in the depth map to a three-dimensional spatial coordinate point to generate a dense point cloud model; A convolutional neural network is used to judge the posture of the point cloud model, and the stability confidence threshold is used as the basis to determine whether the stable state has been reached; When the confidence level exceeds 0.92, the bag is judged to be in an operable state, otherwise it waits for the stationary state to be tested again.

5. The robot autonomous disassembly method based on multi-source visual perception according to claim 1 is characterized in that: The judgment of the stable static state of the bag body is based on the dynamic image frame difference analysis algorithm. By performing frame-by-frame difference calculation on the continuously collected RGB image sequence, the movement trend of the bagged product in a short time window is quantified. Suppose that the continuous N frames of RGB images are {I1, I2, ..., I N Each frame contains M pixels. The pixel difference between each two frames is normalized and calculated to obtain the frame difference value, which is used to measure the stability of the current bag surface texture and edge position. The frame difference value calculation expression is as follows: ， Where D i is the average frame difference between the i-th frame and the i+1-th frame, I i (f) is the grayscale value of the f-th pixel in the i-th frame image, I i+1 (f) is the grayscale value of the f-th pixel in the i+1-th frame image; A residual sequence is constructed for all frame differences and the standard deviation is calculated to identify the consistency of image changes between frames. The standard deviation calculation formula is as follows: ， Where, is the average value of the frame difference sequence, σ D is the standard deviation of the frame difference sequence, reflecting the stability of inter-frame changes; If σ D <θ1, and It is determined that the bag is in a stable state and meets the prerequisite for performing the cutting operation, where θ1 is the judgment threshold of the frame difference standard deviation, and θ2 is the judgment threshold of the average frame difference.

6. The robot autonomous disassembly method based on multi-source visual perception according to claim 1 is characterized in that: The cutting path is determined using a composite algorithm based on edge detection and structural texture analysis. The specific steps are as follows: Use the Canny algorithm to perform preliminary edge extraction on the fused image; The noise and false edges are removed by morphological closing operation to obtain a stable bag contour line; The sealing position of the bag is determined based on the image texture gradient direction, and the sealing line is used as the starting point to input into the path planning module; The path planning module uses the curvature minimization principle to calculate the optimal cutting trajectory, ensuring that the trajectory extends along the sealing direction and avoids cutting areas outside the packaging bag.

7. The robot autonomous disassembly method based on multi-source visual perception according to claim 1 is characterized in that: The robot control system performs trajectory planning based on a vision-guided model and uses an adaptive trajectory calibration mechanism; During the trajectory calibration process, a feedback controller is introduced to compare the deviation between the actual tool head path and the predetermined path. If the deviation exceeds 1.5 mm, the cutting direction is adjusted in real time and the constant pressure is adjusted synchronously. Use PID control algorithm to continuously adjust the angle of the three-axis robot arm to ensure that the trajectory cutting is stable within the set error threshold range, with a maximum allowable error of 2 mm; When the robot performs the disassembly operation, it uses a force control feedback system to dynamically adjust the pressure parameters. The specific steps are as follows: During the cutting process, the blade pressure sensor data is collected every 5 milliseconds to determine the deviation between the current contact force and the preset value in real time; If the current value is too high, the power supply voltage of the actuator motor is reduced to reduce the contact force of the cutter head; if it is too low, the voltage is increased to maintain a constant cutting depth; At the same time, an anti-interference filtering mechanism is introduced to filter out noise data caused by micro-vibration to ensure that the feedback results are stable and effective.

8. The robot autonomous disassembly method based on multi-source visual perception according to claim 1 is characterized in that: The 3D model positioning and path control uses a high-order path smoothing function combined with an error feedback control mechanism to ensure that the robot executes along the predetermined trajectory with high precision when disassembling bagged products. The specific deficiencies are as follows: Extract the three-dimensional path point set P, P = {p j }={p1, p2, ..., p n }, where p j is the jth discrete path point in the three-dimensional path, n is the total number of discrete path points, and each discrete path point is defined as p j =(x j ,y j , z j ), x j 、y j and z j Represent the coordinates of the jth discrete path point in the x-axis, y-axis and z-axis directions respectively; To achieve smooth motion of the robot in three-dimensional space, a spline interpolation path function is constructed based on a set of path points to provide a continuous and differentiable path curve for the robot end effector to track. The formula of the spline interpolation path function is as follows: ， Where B j (t) is the B-spline basis function corresponding to the j-th discrete path point, a j is the interpolation weight coefficient corresponding to the jth discrete path point, S(t) is the spline interpolation path function; During the robot motion execution process, the actual spatial position coordinates (x r 、y r 、z r ), and solve the ideal trajectory point (x(t), y(t), z(t)) corresponding to time t from the path function, and calculate the current execution error using the Euclidean distance formula. The calculation expression is as follows: ， Where x r 、y r 、z r It is the actual three-dimensional coordinate position of the robot end effector fed back in real time by the encoder and position sensor equipment. x(t), y(t), and z(t) represent the three-dimensional theoretical coordinate position corresponding to the spline interpolation path function S(t) at time point t, which is used to compare with the current position of the robot. Δ(t) is the Euclidean distance error between the actual position of the robot in the current motion state and the theoretical path position. When Δ(t) exceeds the set threshold δ, it is determined that there is a path deviation and feedback correction needs to be performed; In the error offset state, the direction angle correction logic is started, and the attitude adjustment angle in the three-dimensional direction is calculated according to the local geometric change trend between continuous path points, and the direction angle vector θ=(θ x ,θ y ,θ z ), the calculation formula for each direction angle is as follows: ， Where, is a discrete path point p j The coordinate value in the kth dimension, is a discrete path point p j+1 The coordinate value in the kth dimension, d is the average Euclidean distance between consecutive path points, θ k Represents the angle change in the k-axis direction, which is used to correct the direction of movement, k∈{x, y, z}; The angle vector is transmitted to the motion solver module in the robot controller, which drives it to adjust the current joint angle so that the end effector posture is realigned with the predetermined path direction, achieving accurate tracking of the cutting path.

9. The robot autonomous disassembly method based on multi-source visual perception according to claim 1 is characterized in that: The final integrity check is completed by combining visual inspection and residual quality control. The specific steps are as follows: The visual inspection part uses a template matching algorithm to compare the pixel differences of the bag boundary before and after disassembly. If the boundary residual image exceeds the set threshold, the "disassembly incomplete" status is prompted; The quality control part uses infrared imaging to detect whether there are residual particles or liquid reflective features on the workbench surface, and converts the infrared reflection intensity into a binary image for residue identification; After combining the two parts of the results, a weighted confidence algorithm is used to comprehensively judge whether the disassembly is successful. The product transfer process will only be started when both the visual and quality control results are qualified.

10. A robot autonomous disassembly system based on multi-source visual perception, used to implement the robot autonomous disassembly method based on multi-source visual perception as described in any one of claims 1 to 9, characterized in that: It includes image acquisition module, image synchronization module, image fusion and state determination module, 3D positioning and path planning module, trajectory execution and force control feedback module, and result detection and qualification determination module: The image acquisition module obtains the initial image data of the target bagged product, including the color image acquired by the RGB camera and the depth image acquired by the depth camera, and marks the acquisition timestamp for each; The image synchronization module synchronizes image data based on timestamps and builds a unified time coordinate system to match the spatial position information of images from different sources at the same time. The image fusion and state determination module performs fusion processing on the synchronized image data, extracts the three-dimensional contour information and dynamic state characteristics of the bag, and determines whether the target is in a stable and stationary state; The 3D positioning and path planning module, under the premise of detecting that the target bag is stable and stationary, performs a 3D model positioning operation based on the fused image to determine the initial starting point and end point of the tearing path; The trajectory execution and force control feedback module controls the robot to execute the disassembly action of a fixed trajectory, uses constant pressure and preset speed for cutting, and maintains the path stability during the operation through feedback control; After the disassembly is completed, the result detection and qualification judgment module calls the visual system again to perform integrity verification on the disassembly area. After confirming that the packaging bag is completely removed and there is no risk of internal contamination, the product is transferred to the next process.

Images