Battery pack flexible assembly line and deviation cooperative control method based on machine vision

Abstract

The invention relates to the technical field of industrial robot control, and discloses a battery pack flexible assembly line and deviation cooperative control method and system based on machine vision. The method comprises the following steps: constructing a digital twin model synchronous with a physical production line, and defining a key feature point network representing assembly precision; for the current assembly station, generating a feedforward pre-adjustment instruction through a preset cooperative control strategy; acquiring real-time assembly data by utilizing a plurality of sensors deployed at a current station, and generating a real-time feedback fine tuning instruction; the collaborative decision module fuses the feedforward and feedback instructions to generate a final execution instruction, and drives the assembly robot to complete high-precision assembly; and finally, the assembly result data is transmitted back to the digital twin model for dynamically optimizing the cooperative control strategy. The problems of deviation accumulation, passive compensation and low production line remodeling efficiency in multi-station assembly are solved, and production line-level deviation active collaborative management and flexible production are achieved.

Description

Technical Field

[0001] This invention relates to the field of industrial robot technology, specifically to a machine vision-based flexible assembly line for battery packs and a method for coordinated control of deviations. Background Technology

[0002] In the production of power battery packs for new energy vehicles, the assembly process is crucial for ensuring product performance, safety, and consistency. Battery packs have complex structures, require high assembly precision, and experience rapid product model iterations, placing extremely high demands on the precision, flexibility, and intelligence of automated assembly systems. While existing technologies have made some progress in assembly control using robots and vision, they still have systemic shortcomings when dealing with complex assembly scenarios like battery packs, which involve multiple workstations, high precision, and rapid replacement.

[0003] Chinese invention patent application CN120588231A discloses a compliant assembly method based on force feedback. This method adjusts the robot's pose by sensing contact force in real time, effectively solving the collision and jamming problems during the insertion of precision components and achieving "adaptive" precision in individual assembly actions. However, the control logic of this method starts after contact occurs, belonging to a typical "feedback correction" mode. In multi-station series assembly of battery packs, pose deviations generated by upstream stations (such as housing positioning and module stacking) accumulate and are transmitted to downstream stations (such as busbar installation). This solution only performs local force control adjustments within the downstream station, lacking the "predictive" and "proactive" compensation capabilities for accumulated upstream deviations. When upstream deviations exceed the correction range of its feedback controller, assembly failure is highly likely. Therefore, this method essentially fails to solve the problem of production line-level deviation transmission and collaborative control.

[0004] Chinese invention patent application CN120645216A proposes an intelligent fixture system integrating multimodal perception and digital twins. This system adapts to different components through simulation training and online learning, demonstrating good single-station "flexibility." While emphasizing perception fusion and offline strategy optimization, its core focus is on the "grab-place" process of a single complex component, with control decisions revolving around a single fixture or robotic arm. For battery pack assembly lines with dozens of workstations, a model and mechanism describing how deviations at each workstation affect each other and how cross-workstation collaborative compensation works is not constructed. Its digital twin is mainly used for single-point strategy training, rather than for simulating and optimizing the "deviation flow" of the entire production line. Therefore, it cannot solve the core challenge of modeling the transmission of deviations and achieving system-level collaboration in multi-workstation assembly.

[0005] Chinese invention patent application CN120621296A employs a method of recording and replaying robotic arm motion paths to adapt to unstructured environments (such as rugged terrain), demonstrating a "memory-like" adaptation to fixed-pattern deviations. This method is effective in scenarios with relatively fixed environments and simple processes. However, battery pack production is a typical example of discrete manufacturing, with significant differences in size and interface between different models, and random deviations in incoming materials and positioning. The "teach-playback" mode of this application cannot cope with such highly dynamic and multivariate deviations. It lacks dynamic decision-making capabilities based on real-time perception and model prediction, and cannot proactively predict unknown or random deviations and generate compensation strategies in real time during the assembly process. Therefore, it is insufficiently adaptable to mixed-flow assembly lines that require high flexibility and intelligent decision-making.

[0006] In summary, existing technologies offer partial solutions from the perspectives of force control feedback, multimodal perception and learning, and path memory reproduction, but all have limitations: they lack a feedforward and systemic perspective, a production line-level deviation coordination model, and dynamic intelligent decision-making capabilities. Therefore, there is an urgent need for a collaborative control method that deeply integrates production line-level deviation modeling, feedforward prediction and compensation, multimodal real-time feedback, and rapid changeover adaptation to systematically address the common problems of deviation accumulation, passive control, and slow changeover in flexible battery pack assembly. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a machine vision-based method and system for collaborative control of deviations in flexible battery pack assembly lines. This method constructs a digital twin model that integrates multi-source information and innovatively employs a collaborative decision-making mechanism combining "feedforward prediction compensation" and "real-time feedback fine-tuning" to achieve proactive management and collaborative control of assembly deviations at the production line level. Simultaneously, it supports rapid switching between product models, significantly improving the accuracy, flexibility, and intelligence level of the assembly line.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a machine vision-based method for coordinated control of deviations in a flexible battery pack assembly line, applicable to a battery pack assembly line containing multiple assembly stations, comprising the following steps: S1. Digital Twin and Feature Network Construction: Construct a digital twin model corresponding to the physical assembly line, and define a network of key feature points in the model to characterize the assembly position and orientation accuracy of each component of the battery pack. This includes: establishing a digital twin model that runs synchronously with the physical assembly line. In this model, key feature points on the battery pack, from the casing to each internal component, are precisely defined to measure assembly quality, forming a network of key feature points covering the entire process.

[0009] S2. Acquisition of upstream deviation information: For the current target workstation on the assembly line, the actual assembly deviation data generated by the upstream completed workstations are obtained from the digital twin model; including: when the battery pack is transferred to a target workstation, the actual deviation data recorded after the battery pack has been assembled in all upstream workstations is retrieved in real time from the digital twin model.

[0010] S3. Feedforward pre-adjustment instruction generation: Based on the actual assembly deviation data and the preset collaborative control strategy, a feedforward pre-adjustment instruction is generated to actively compensate for the impact of upstream deviation on the current assembly operation; including: based on the acquired upstream actual deviation data and the collaborative control strategy established in advance through data analysis and simulation, calculating the pre-adjustment action of the robot or fixture required to compensate for the upstream accumulated deviation and improve the assembly success rate of the current workstation, i.e., generating the feedforward pre-adjustment instruction.

[0011] S4. Real-time Feedback Fine-tuning Instruction Generation: Utilizing the multimodal sensing unit deployed at the current target workstation, machine vision images, contact force information, and component posture information are collected in real time during the assembly process. Real-time feedback fine-tuning instructions are generated according to preset feedback control rules. This includes: during the assembly execution at the current workstation, the assembly status is monitored in real time using multimodal sensing units such as 3D vision and force sensors. When abnormal contact force or visual alignment deviation is detected, real-time feedback fine-tuning instructions for immediate correction are generated based on preset feedback control logic.

[0012] S5. Collaborative Decision-Making and Command Fusion: The collaborative decision-making module fuses the feedforward pre-adjustment command and the real-time feedback fine-tuning command to generate the final execution command driving the actuator of the current target workstation. This includes: intelligently fusing the prediction-based feedforward pre-adjustment command and the real-time perception-based feedback fine-tuning command through the collaborative decision-making module. The fusion weight is dynamically adjusted according to the assembly stage (such as coarse positioning and fine insertion) and the quality of sensor data, ultimately generating a safe and efficient final execution command.

[0013] S6. Execution and Strategy Optimization Closed Loop: Based on the final execution command, the actuator is controlled to complete the assembly operation. The actual assembly accuracy data and process sensor data after the operation are used as new measured data and fed back to the digital twin model to update and optimize the collaborative control strategy. This includes: the actuator completing the assembly according to the final command. After completion, the actual measurement results of this assembly (such as coaxiality and press-fitting force curves) and the entire process sensor data are fed back to the digital twin model. This new data is used to continuously optimize the collaborative control strategy, enabling the system to have self-learning and self-evolution capabilities.

[0014] Furthermore, to support rapid product changeover, the method also includes a rapid adaptation process: after importing the new product model, simulation and strategy optimization are performed in a digital twin virtual environment to generate an initial control scheme; this scheme is compressed into a lightweight module suitable for edge deployment using techniques such as knowledge distillation; and during physical trial assembly, the module is fine-tuned online using a small amount of data to achieve rapid product launch.

[0015] Secondly, the present invention provides a system for implementing the above method, including a digital twin and strategy management platform, sensing and execution units deployed at each workstation, edge collaborative control nodes, and a high-speed industrial communication network.

[0016] Thirdly, the present invention provides a storage medium storing a related computer program.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Achieved proactive collaborative control of production line-level deviations: This invention breaks through the limitations of the prior art, which only focuses on single-point force control and single-station flexibility. By connecting the deviation information of each workstation through a digital twin model, and using a collaborative control strategy to proactively predict and compensate for upstream influences, the invention suppresses the accumulation of deviations at the system level, thereby improving the overall assembly accuracy and first-pass success rate.

[0018] 2. Enhanced the extreme flexibility of the production line: Through the technical path of "virtual simulation optimization -> knowledge distillation and transfer -> online fine-tuning and adaptation", the shortcomings of the teaching and reproduction mode in the background technology cannot adapt to new products and random deviations. When facing new products, the production line can generate and optimize control strategies almost automatically, reducing changeover time from several hours of traditional manual intervention to minutes, and greatly enhancing the production line's ability to cope with multi-variety, small-batch production.

[0019] 3. A smart closed loop of perception-decision-execution has been constructed: It innovatively integrates feedforward predictive compensation with feedback-based real-time adjustment through an intelligent decision-making module. Feedforward commands provide "pre-aiming" capabilities, significantly reducing adjustment time; feedback commands ensure "precise" implementation, guaranteeing assembly smoothness. The two work together to make the assembly process both efficient and robust.

[0020] 4. A continuously optimizing, data-driven ecosystem has been established: each assembly generates data that flows back to the digital twin to optimize control strategies. This makes the system no longer static, but an intelligent agent that can continuously learn and evolve as production data accumulates, laying a solid foundation for realizing intelligent manufacturing. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0023] Figure 1 The overall flowchart of the method provided in the embodiments of the present invention is shown.

[0024] Figure 2 This is a schematic diagram illustrating the principle of the feedforward-feedback collaborative control process in an embodiment of the present invention.

[0025] Figure 3 This is a schematic diagram of the product model rapid changeover and adaptation process in an embodiment of the present invention.

[0026] Figure 4 This is a schematic diagram of the overall system architecture provided for an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0028] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0029] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" and "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0030] Example 1 This embodiment details an intelligent collaborative control method applied to battery packaging assembly lines. This method aims to solve problems such as deviation accumulation, passive control, and difficulties in production line changeovers in multi-station series assembly. Its core lies in constructing a "global digital twin model" and, based on this, innovatively implementing a dynamic collaborative control mechanism of "feedforward prediction-feedback fine-tuning." For example... Figure 1 As shown, this invention provides an overall flow chart of a machine vision-based flexible battery pack assembly line and a deviation collaborative control method, as follows: Figure 2 As shown, the core of this method lies in the collaborative control mechanism of "feedforward prediction compensation" and "real-time feedback fine-tuning," specifically, This invention provides a machine vision-based method for coordinated control of deviations in flexible battery pack assembly lines, applicable to battery pack assembly lines with multiple assembly stations, comprising the following steps: S1. Digital Twin and Feature Network Construction: Construct a digital twin model corresponding to the physical assembly line, and define a network of key feature points in the model to characterize the assembly position and orientation accuracy of each component of the battery pack. This includes: establishing a digital twin model that runs synchronously with the physical assembly line. In this model, key feature points on the battery pack, from the casing to each internal component, are precisely defined to measure assembly quality, forming a network of key feature points covering the entire process.

[0031] The key feature point network includes geometric feature points defined on the battery pack housing, battery module, electrical connectors and structural components. Each feature point contains at least its three-dimensional position and normal vector information in the reference coordinate system.

[0032] Feature selection: Not all points on the battery pack are important. Engineers will select geometric features that play a decisive role in assembly quality, based on the assembly process. For example: Housing: The four corner points of the main positioning surface (used to determine the reference position of the housing on the tray) and the center of the mounting holes for connection with the chassis.

[0033] Battery module: The center of the reference surface on each module for stacking and positioning, and the position of the studs or clips for fixing.

[0034] Electrical connectors: such as the center of each stud hole on a busbar, and the center of the mating surface of a high-voltage connector.

[0035] Structural components: mounting holes for end plates and side plates, sealing surface edges, etc.

[0036] Information Description: For each feature point, not only are its three-dimensional coordinates (X, Y, Z) recorded in a unified world coordinate system or workpiece coordinate system, but also the normal vector of the surface on which the point lies (describing the orientation of the surface). For example, the normal vector of the center point of a stud hole indicates the direction of the hole's axis, which is crucial for subsequent accurate alignment.

[0037] Networking: All feature points do not exist in isolation. They have topological relationships (such as several holes forming a mounting surface) and belong to specific assembly stages. Organizing these points and their relationships into a network or graph structure forms a precision control network covering the entire process from the first station to the last station.

[0038] Step S1 specifically includes, for example, using the 3D layout of the physical production line, the precise dimensions of equipment (robots, conveyor belts, workbenches), and CAD models of various battery pack models, to establish a digital model in a computer virtual environment that corresponds one-to-one with the physical world and is synchronized with its data. This model is not only a static 3D display, but also a dynamic system that can receive and map the real-time operating status of the physical production line (such as real-time robot pose, sensor readings, and workstation cycle time). "Twin" means that the virtual model and the real production line appear in pairs and evolve synchronously.

[0039] Step S1 lays the data foundation for the entire system's intelligent decision-making. The digital twin model is the system's "sandbox," used for simulation, prediction, and optimization; the key feature point network is the "coordinate grid" on the sandbox, enabling the system to accurately quantify, track, and correlate errors in each assembly step. This is a prerequisite for achieving subsequent collaborative control and a key difference from traditional single-point control (such as focusing only on the target point of the current workstation).

[0040] S2. Acquisition of upstream deviation information: For the current target workstation on the assembly line, the actual assembly deviation data generated by the upstream completed workstations are obtained from the digital twin model; including: when the battery pack is transferred to a target workstation, the actual deviation data recorded after the battery pack has been assembled in all upstream workstations is retrieved in real time from the digital twin model.

[0041] The specific process of step S2 includes, for example, the following: when a pallet (or AGV) carrying a specific battery pack enters the current target workstation (e.g., busbar installation workstation), the control system will perform the following operations: Identification: Obtain the unique identification (ID) of the battery pack through a barcode reader (such as RFID or QR code scanning).

[0042] Data Query: The system uses this ID to initiate a query request to the database of the digital twin platform. The database returns the historical data of the battery pack corresponding to this ID in all completed assembly stations upstream of the current workstation.

[0043] Data Content: These records are not simple "pass / fail" labels, but rather "actual assembly deviation data." For example: Shell positioning station: Record the translational deviation (ΔX, ΔY) and rotational deviation (Δθz) of the shell relative to the theoretical position.

[0044] Module stacking station: Record the flatness error of the bottom surface of the first module and the relative height difference between each module.

[0045] End plate installation station: Record the offset of the actual position and theoretical position of the end plate installation hole in the XY plane.

[0046] Information transmission: These upstream measured, multi-dimensional deviation data are packaged into a data packet and transmitted in real time to the edge controller of the current target workstation.

[0047] Step S2 is the starting point of "cooperative control." It breaks down the traditional "information silos" between workstations, enabling the controller at the current workstation to clearly identify the target position and adjust the target based on the accumulated deviation upstream. This provides precise input for subsequent active compensation (feedforward), and is a key step in realizing the transformation from "isolated control" to "associated control."

[0048] S3. Feedforward pre-adjustment instruction generation: Based on the actual assembly deviation data and the preset collaborative control strategy, a feedforward pre-adjustment instruction is generated to actively compensate for the impact of upstream deviation on the current assembly operation; including: based on the acquired upstream actual deviation data and the collaborative control strategy established in advance through data analysis and simulation, calculating the pre-adjustment action of the robot or fixture required to compensate for the upstream accumulated deviation and improve the assembly success rate of the current workstation, i.e., generating the feedforward pre-adjustment instruction.

[0049] The "preset collaborative control strategy" in step S3 is a mapping relationship rule library established based on historical assembly data and simulation analysis, which describes how upstream station deviations affect the assembly success rate of downstream stations. The generation of feedforward pre-adjustment instructions refers to querying or calculating the mapping relationship rule library based on the actual assembly deviation data to predict the pre-adjustment amount required for the current assembly robot end pose or fixture action.

[0050] In step S3, as Figure 2 As shown, the generation of the feedforward pre-adjustment command depends on "upstream deviation data" and "cooperative control strategy library". Specifically, the implementation process of step S3 includes, for example: Step 1: Establish a pre-defined collaborative control strategy (collaborative control strategy library) by combining historical assembly data analysis with physical simulation analysis. The collaborative control strategy library may be represented as a lookup table, a set of rules (if-then), or a trained lightweight prediction model. Historical data analysis, including the analysis of a large number of historical successful and unsuccessful assembly cases, uses machine learning methods (such as regression analysis and decision trees) to uncover the statistical regularity between "the success rate of busbar installation and the required compensation direction / size when the shell has a certain directional deviation".

[0051] Physical simulation analysis involves deliberately setting a series of deviations at upstream feature points in a virtual environment of a digital twin model (simulating various possible incoming material or assembly errors), and then running physical simulations to accurately calculate the quantitative impact of these deviations on assembly operations (such as insertion force and alignment) when they are transmitted to downstream workstations. Through a large number of such simulations, a high-fidelity "deviation-impact" mapping relationship is established.

[0052] Step 2: Generate feedforward pre-adjustment instructions, including: Matching and Calculation: After receiving the upstream deviation data packet from step S2, the edge controller of the current target station immediately uses this data as input and sends it to the collaborative control strategy library for querying or calculation.

[0053] Predicted compensation amount: The strategy library outputs a prediction result based on its embedded knowledge. For example: "Based on the upstream shell's +0.3mm Y-axis deviation and the module's 0.1mm flatness error, it is predicted that during installation, the busbar's 3rd and 5th stud holes will have expected alignment deviations of approximately -0.25mm Y-axis and +0.05° around the X-axis, respectively."

[0054] Generate instructions: The controller directly converts this predicted "expected alignment deviation" into a reverse adjustment instruction that the robot's end effector needs to execute in advance, i.e., a feedforward pre-adjustment instruction. This instruction is superimposed on the basic motion trajectory when the robot begins to perform the main "grasp-move-place" task.

[0055] Step S3 is the core of achieving the "active compensation" of this invention. It utilizes the upstream information provided in step S2 and the model knowledge constructed in step S1 to "foresee" potential difficulties and make adjustments in advance before the physical assembly begins. This is equivalent to equipping the robot with "predictive capabilities," allowing it to be in a more advantageous starting position before contacting the workpiece, thereby significantly shortening the travel and time of subsequent feedback adjustments and significantly improving the success rate of the first attempt. This is a fundamental improvement over traditional pure feedback control (solving problems after they are discovered).

[0056] S4. Real-time Feedback Fine-tuning Instruction Generation: Utilizing the multimodal sensing unit deployed at the current target workstation, machine vision images, contact force information, and component posture information are collected in real time during the assembly process. Real-time feedback fine-tuning instructions are generated according to preset feedback control rules. This includes: during the assembly execution at the current workstation, the assembly status is monitored in real time using multimodal sensing units such as 3D vision and force sensors. When abnormal contact force or visual alignment deviation is detected, real-time feedback fine-tuning instructions for immediate correction are generated based on preset feedback control logic.

[0057] The multimodal sensing unit includes: 3D vision sensors (such as structured light cameras and binocular cameras) are used to acquire depth and color images of the assembly area to reconstruct the 3D information of the workpiece. Specifically, as the robot carrying the workpiece (such as a busbar) approaches the target area, a rapid scan of the assembly scene is performed to acquire "depth images" (each pixel contains distance information) and color images. Using point cloud processing algorithms, the precise 3D position and orientation of the workpiece (such as a hole on a busbar) and the assembly target (such as a stud on a battery pack) are reconstructed in real time. By comparing the real-time images / point clouds with the desired ideal model, the pose deviation is calculated.

[0058] A six-dimensional force / torque sensor, mounted on the robot's end effector, is used to measure multidimensional contact forces and torques during assembly. It can simultaneously measure forces in three directions (X, Y, Z) and torques in three directions (around the X, Y, and Z axes). When the workpiece (busbar) begins contact with the target (stud), any misalignment will generate lateral forces or torsional torques that can be detected by the sensor. This is the direct input signal for force control or admittance / impedance control.

[0059] An inertial measurement unit (IMU) is used to monitor minute displacements and vibrations of parts to be assembled or fixtures. The IMU is attached to the fixture or critical components to sense minute high-frequency vibrations or low-frequency slippage. For example, when clamping a smooth-surfaced part, the IMU can detect whether the fixture has experienced minute slippage that is not easily detected directly by a force sensor.

[0060] In step S4, the generation of feedback fine-tuning instructions is based on real-time data provided by the "multimodal sensing unit." The feedback control rule is a closed-loop control logic based on the principles of force sensing and visual servoing. It is used to dynamically calculate the fine-tuning action of the robot's end effector or gripper based on the real-time sensed contact force deviation or visual positioning deviation, and generate real-time feedback fine-tuning instructions. The force sensing principle works as follows: once contact occurs and the force sensor signal becomes reliable, a compensation motion command is calculated in real time based on a preset force control strategy (such as "maintaining a constant light insertion force" or "translating in the opposite direction of the force when encountering a lateral force exceeding a threshold"). This force sensing principle ensures the compliance of the assembly process and prevents rigid collisions from damaging the workpiece.

[0061] The principle of visual servoing is as follows: based on the real-time positioning results of 3D vision, it continuously calculates small pose deviations and generates instructions to drive the robot's end effector to make the feature points in the vision aligned.

[0062] The real-time sensing and feedback mechanism in step S4 forms a fast and adaptive closed loop, capable of correcting these unforeseen minor deviations in real time and ensuring a smooth and safe assembly process. It perfectly connects the "macroscopic pre-aiming" in step S3 with the "microscopic hands-on operation" on site.

[0063] S5. Collaborative Decision-Making and Command Fusion: The collaborative decision-making module fuses the feedforward pre-adjustment command and the real-time feedback fine-tuning command to generate the final execution command driving the actuator of the current target workstation. This includes: intelligently fusing the prediction-based feedforward pre-adjustment command and the real-time perception-based feedback fine-tuning command through the collaborative decision-making module. The fusion weight is dynamically adjusted according to the assembly stage (such as coarse positioning and fine insertion) and the quality of sensor data, ultimately generating a safe and efficient final execution command.

[0064] In step S5, the collaborative decision-making module dynamically allocates the fusion weights of the feedforward pre-adjustment instruction and the real-time feedback fine-tuning instruction in the final execution instruction based on the current assembly stage, the confidence level of the perceived data, and the historical assembly success rate. The decision-making module does not simply add the feedforward instruction (U_ff) and the feedback instruction (U_fb). Instead, it assigns a dynamically changing fusion weight (α and β, where α + β = 1) to both based on the real-time context of the current assembly.

[0065] Weighting strategy example: Phase 1 (Coarse Localization): The robot is far from the target, and visual and force signals may be unreliable or ineffective. At this stage, feedforward instructions are given high weight (e.g., α=0.9), while feedback instructions are given low weight (β=0.1), allowing the robot to rely primarily on prediction to quickly approach the target.

[0066] Phase Two (Visual Precision Alignment): The visual sensor obtains a clear and stable image. At this point, the feedforward weight is reduced, and the visual feedback weight is significantly increased (e.g., α=0.3, β_visual=0.6, β_force=0.1) to perform pixel-level precise alignment.

[0067] Phase 3 (Force-Controlled Insertion): Contact occurs, and the force sensor data is reliable. At this point, the feedforward weight is reduced to its minimum, and the force feedback weight is increased to its maximum (e.g., α=0.1, β_force=0.8, β_visual=0.1) to ensure smooth insertion and avoid jamming.

[0068] Confidence assessment: The decision module also evaluates the quality of data from each sensor (such as visual image sharpness and force signal noise level). If the data quality of a certain sensor is poor, the weight of its corresponding feedback command will be temporarily reduced.

[0069] Generate the final execution instruction: The decision module performs weighted fusion of U_ff and each U_fb according to dynamic weights to generate a smooth, continuous, and conflict-free final execution instruction (U_cmd), which is directly sent to the robot's underlying servo driver.

[0070] Through the dynamic fusion in step S5, the system not only takes advantage of the speed of feedforward, but also retains the accuracy and safety of feedback, thus avoiding the limitations of a single control mode.

[0071] S6. Execution and Strategy Optimization Closed Loop: Based on the final execution command, the actuator is controlled to complete the assembly operation. The actual assembly accuracy data and process sensor data after the operation are used as new measured data and fed back to the digital twin model to update and optimize the collaborative control strategy. This includes: the actuator completing the assembly according to the final command. After completion, the actual measurement results of this assembly (such as coaxiality and press-fitting force curves) and the entire process sensor data are fed back to the digital twin model. This new data is used to continuously optimize the collaborative control strategy, enabling the system to have self-learning and self-evolution capabilities.

[0072] The specific execution process of step S6 includes, for example: 1. Execution and Result Measurement: The robot executes the final instructions generated in step S5 to complete the assembly action. Subsequently, an independent online measurement system (such as a high-precision laser scanner or photogrammetric measuring instrument) will perform authoritative testing on the assembly results and generate "actual assembly accuracy data" (such as the actual value of coaxiality and the final tightening torque value of each stud).

[0073] 2. Data Packaging and Transmission: The system packages all the data from this assembly, including: upstream deviation data (input), feedforward commands, feedback commands at each stage, final commands after fusion (process), final measured accuracy data (result), and all raw / processed data from all sensors (environment). This rich "data package" is transmitted back to the database of the digital twin platform via the network.

[0074] 3. Strategy optimization: Model calibration: The data analysis engine of the digital twin platform compares the "actual test results" with the "feedforward prediction results". If a systematic bias is found (for example, the compensation amount of the feedforward prediction is always 5% smaller than the actual requirement), the parameters of the collaborative control strategy (rule base) in step S3 will be automatically tuned to make the next prediction more accurate.

[0075] Knowledge accumulation: Every successful or unsuccessful assembly case becomes "nourishment" for enriching the strategy library. Through continuous online learning or regular batch learning, the coverage and prediction accuracy of the strategy library will be continuously improved.

[0076] Step S6 transforms a one-time control action into a continuously improving intelligent process. The returned data is not only used for recording and traceability, but more importantly, for optimizing front-end strategies. This enables the system to adapt to long-term changes such as production line aging, tool wear, and slow drift in incoming material characteristics, achieving a qualitative leap from a "fixed program" to an "adaptive system." It closes the loop of the entire process from S1 to S5, making the system a truly vibrant and continuously evolving intelligent manufacturing unit.

[0077] Furthermore, to support rapid model changeover, such as Figure 3 As shown, the method also includes a rapid adaptation process: after importing the new product model, simulation and strategy optimization are performed in a digital twin virtual environment to generate an initial control scheme; this scheme is compressed into a lightweight module suitable for edge deployment using techniques such as knowledge distillation; during physical trial assembly, the module is fine-tuned online using a small amount of data to achieve rapid product launch. Specific steps include: S01. Import the three-dimensional design model (such as a 3D CAD model), BOM (Bill of Materials), and assembly process requirements (such as tightening sequence, recommended pressing force, etc.) of the new product model's battery pack into the digital twin model; S02. In the virtual environment provided by the digital twin model, artificial intelligence algorithms such as reinforcement learning (RL) are used to simulate and optimize the assembly process of the new product model, and generate an initial collaborative control strategy and assembly parameter set for the model. S03. The optimized strategy and parameter set are transformed into a lightweight control module that can run on the edge controller of the production line through model compression technology. S04. Perform trial assembly of the new product model on the physical production line, and based on the actual data generated from the trial assembly, perform online adaptive fine-tuning of the lightweight control module until the assembly quality requirements are met.

[0078] The model compression technique is a knowledge distillation method, which obtains approximate performance by having the lightweight control module learn the behavior of the expert control model in the simulation environment.

[0079] This invention reduces the time required for new product introduction from several days or even weeks of "manual programming-debugging-verification" to several hours of "automatic simulation-distillation-fine-tuning." Knowledge distillation serves as a bridge between powerful offline simulation capabilities and limited online computing resources, enabling cutting-edge AI algorithms to be truly implemented in industrial settings. This completely solves the fundamental pain points of long downtime and poor flexibility in traditional production lines, truly achieving "ultimate flexibility."

[0080] Example 2 like Figure 1 As shown, this embodiment demonstrates the application of a machine vision-based flexible assembly line and deviation collaborative control method for battery packs in a key workstation of "busbar installation" for a new energy vehicle battery pack. The method includes: Step 1: System Initialization and Model Building During the system deployment phase, a digital twin model that perfectly corresponds to the physical assembly line is first constructed using 3D design software and production line scanning data. In this model, the four corner points of the main positioning surface of the battery pack casing, the reference surface of the module stack, and the center points of all stud mounting holes on the busbar are precisely defined as a network of key feature points. Simultaneously, based on historical production line operation data, empirical rules such as "when the casing positioning has a positive deviation in the X direction, pre-compensation in the Y direction is required during busbar installation" are analyzed and summarized, and these are formalized into an initial collaborative control strategy and stored in the strategy library.

[0081] Step 2: Feedforward pre-adjustment (active compensation) When a specific battery pack enters the "busbar installation" station, the edge collaborative control node of that station immediately initiates a query to the digital twin platform. The platform returns the measured data of the upstream station recorded under the battery pack ID, such as: "The housing assembly station recorded an offset of +0.3mm in the Y direction of the housing", and "The module pressing station recorded a flatness error of 0.1mm".

[0082] Based on this upstream deviation data, the control node queries the collaborative control strategy. The strategy logic, after calculation, outputs: to offset the cumulative effect of the aforementioned deviations, when the robot grasps the busbar approaching the target, its end effector needs to be pre-compensated by -0.25mm in the Y direction and finely adjusted by 0.05 degrees around the X axis during the placement phase. This calculation result serves as part of the feedforward pre-adjustment instruction.

[0083] Step 3: Real-time feedback and fine-tuning (precise alignment) The robot, carrying the busbar, moves to the vicinity of the pre-compensation position under the guidance of feedforward pre-adjustment commands. Subsequently, the 3D vision sensor installed at the robot's end effector is activated to precisely scan and locate the battery pack stud group below, and compare it with the theoretical model to generate sub-millimeter-level precise positioning commands (a type of visual feedback fine-tuning command).

[0084] Simultaneously, as the robot lowers the busbar to begin fitting the mounting holes into the studs, the six-dimensional force / torque sensor activates. If it senses an abnormal lateral force from a stud, the system immediately generates a force feedback fine-tuning command based on preset force control feedback rules (e.g., "if resistance is encountered, slightly back and perform lateral micro-motion to find the hole").

[0085] Step 4: Collaborative Decision Making and Command Integration The collaborative decision-making module at the workstation operates continuously. During the coarse positioning stage of assembly, it assigns high weights to feedforward pre-adjustment commands and visual feedback commands to ensure rapid and generally accurate positioning. When entering the fine insertion stage, after contact begins, the module automatically reduces the weight of feedforward commands and significantly increases the weight of force feedback commands to ensure the smoothness of the assembly process and prevent scratches or jamming.

[0086] The decision-making module merges the weighted instructions from each path to generate a smooth and safe final execution instruction, which is then sent to the robot controller.

[0087] Step 5: Execution and Data Loop Optimization The robot executes the final fusion command to complete the installation and fastening of the busbar. After installation, a high-precision online measuring camera photographs the installation result, measuring the actual coaxiality data as 0.08mm (better than the target value of 0.1mm). Simultaneously, all sensor data, command sequences, and final measurement results throughout the assembly process are packaged into an "assembly data package."

[0088] The data packet was transmitted back to the digital twin and strategy management platform via the industrial network. The platform's data analysis engine discovered a slight difference between the actual Y-axis compensation amount used in this successful assembly (-0.23mm) and the initial strategy prediction (-0.25mm). Based on this, the platform automatically made a minor optimization adjustment to the collaborative control strategy rule parameters related to the "impact of housing Y-axis deviation on busbar installation," making it more closely aligned with the actual production line conditions. This completes a full closed loop from perception to execution to strategy optimization.

[0089] Step Six: Application of Rapid Model Changeover (e.g.) Figure 3 (As shown) When the production line needs to switch to producing a new battery pack model (Model B), the operator imports the 3D CAD model and assembly process requirements of Model B into the digital twin platform and performs the following operations: 1. Virtual Simulation and Optimization: The platform automatically creates an assembly scenario for Model B in a virtual environment. A virtual control agent attempts thousands of assembly steps in the simulation, learning through trial and error to autonomously explore the optimal robot movement path, fixture clamping force, visual recognition parameters, etc., forming an expert control strategy for Model B.

[0090] 2. Knowledge Distillation and Compression: Due to the complexity of expert policies, they cannot be directly deployed to edge controllers with high real-time requirements. Therefore, knowledge distillation is employed: a simple student model is trained to mimic the output behavior of a complex teacher model (i.e., expert policy) under the same simulation input. This results in a lightweight, efficient, yet performance-approaching deployable control module.

[0091] 3. Online Fine-tuning and Adaptation: The lightweight control module is downloaded to the edge control node of the "busbar installation" station. When the first physical model B battery pack arrives, the system uses the new module for assembly and collects actual data from the first few pieces. The edge node uses this limited amount of real data to quickly fine-tune the control parameters of the new module online. Typically, after adapting to fewer than five pieces, the assembly quality reaches a stable standard, completing the rapid switch from model A to model B.

[0092] Through the above steps, this invention achieves intelligent collaborative control of the entire battery pack assembly process, from global planning to local execution, from predictive compensation to real-time adjustment, and from offline learning to online optimization.

[0093] Example 3 This embodiment provides a system for implementing the methods described in Embodiments 1 and 2, such as Figure 4 As shown, the system includes: 1. Digital Twin and Strategy Management Platform: As the "intelligent brain" of the system, it runs on a high-performance server and is responsible for maintaining the digital twin model, storing all key feature point data, historical deviation database and collaborative control strategy library, and undertaking the simulation optimization and knowledge distillation tasks of new product strategies.

[0094] 2. Sensing and Execution Units: Serving as the "hands and feet" and "sensors" of the system, these units are distributed across each physical workstation. Each unit includes: a high-precision collaborative robot (such as UR or FANUC), a servo actuator with force control capabilities, an adaptive gripper with switchable grippers, and a multimodal sensing module integrating a 3D camera, a six-dimensional force sensor, and an IMU.

[0095] 3. Edge Collaborative Control Node: Acting as the system's "local cerebellum," it is deployed near each production line or key workstation. It possesses powerful built-in edge computing capabilities, runs a collaborative decision-making module and a lightweight control model, and is responsible for receiving strategies from the platform, processing real-time data from local sensors, and generating final execution instructions to drive the equipment at its workstation. Its characteristics include low latency and high reliability.

[0096] 4. High-speed industrial communication network: Serving as the system's "neural network," it employs a hybrid architecture of Time-Sensitive Networking (TSN) and Industrial Ethernet. TSN ensures the timely and error-free transmission of time-sensitive information such as robot control commands and force sensor data; while Industrial Ethernet handles the stable transmission of large volumes of data, including 3D point clouds and model parameters. The entire network uses a precision clock synchronization protocol to ensure strict alignment of all data on the timeline.

[0097] The working principle of this system is as follows: Starting with upstream deviation and collaboration strategies provided by the digital twin platform, and then to the generation of feedforward instructions by edge nodes; combined with feedback instructions generated by local multimodal perception, these instructions are fused by the collaborative decision-making module to drive the actions of the actuators; the results of these actions are then perceived and fed back to the platform, forming a continuously optimized intelligent closed loop.

[0098] like Figure 4 As shown, all modules of the system are fully connected through a high-speed industrial communication network (such as TSN+ industrial Ethernet) to ensure real-time data synchronization and command transmission between the digital twin platform, edge nodes and sensing and execution units, supporting microsecond-level timing synchronization and meeting the requirements of high-precision collaborative control.

[0099] The specific implementation methods of the functions of the above modules are the same as those of the methods in Embodiment 1 and Embodiment 2, and will not be repeated here.

[0100] Example 4 The present invention also provides an electronic device, including: a processor, a transmitting device, an input device, an output device, and a memory. The processor may be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit, or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory may be implemented using a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), and is used to store computer program code. The computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes a method as described in any of the above possible implementation methods.

[0101] Example 5 The present invention also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor of an electronic device, cause the processor to perform a method as described in any of the above possible implementations.

[0102] In the description of this specification, the references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0103] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A machine vision-based flexible assembly line for battery packs and a deviation collaborative control method, applied to a battery pack assembly line containing multiple assembly stations, characterized in that... The method includes the following steps: S1. Construct a digital twin model corresponding to the physical assembly line, and define a network of key feature points in the model to characterize the assembly position and attitude accuracy of each component of the battery pack. S2. For the current target workstation on the assembly line, obtain the actual assembly deviation data generated by the upstream completed workstations from the digital twin model. S3. Based on the actual assembly deviation data and the preset collaborative control strategy, generate a feedforward pre-adjustment instruction for actively compensating for the impact of upstream deviation on the current assembly operation. S4. Using the multimodal sensing unit deployed at the current target workstation, machine vision images, contact force information and component posture information are collected in real time during the assembly process, and real-time feedback fine-tuning instructions are generated according to the preset feedback control rules. S5. The collaborative decision-making module integrates the feedforward pre-adjustment instruction and the real-time feedback fine-tuning instruction to generate the final execution instruction that drives the current target workstation actuator. S6. Control the actuator to complete the assembly operation according to the final execution instruction, and use the actual assembly accuracy data and process sensing data after the operation as new measured data to send back to the digital twin model for updating and optimizing the collaborative control strategy.

2. The method according to claim 1, characterized in that, The key feature point network includes geometric feature points defined on the battery pack housing, battery module, electrical connectors and structural components. Each feature point contains at least its three-dimensional position and normal vector information in the reference coordinate system.

3. The method according to claim 1, characterized in that, The "preset collaborative control strategy" in step S3 is a mapping relationship rule base established based on historical assembly data and simulation analysis, which describes how upstream workstation deviations affect the assembly success rate of downstream workstations. The feedforward pre-adjustment instruction refers to the pre-adjustment amount required for the current assembly robot end pose or fixture action by querying or calculating the mapping relationship rule base according to the actual assembly deviation data.

4. The method according to claim 1, characterized in that, In step S4, the feedback control rule is a closed-loop control logic based on the principle of force sensing and visual servoing, which is used to dynamically calculate the fine-tuning action of the robot end effector or gripper according to the real-time perceived contact force deviation or visual positioning deviation.

5. The method according to claim 1, characterized in that, In step S5, the collaborative decision-making module dynamically allocates the fusion weights of the feedforward pre-adjustment instruction and the real-time feedback fine-tuning instruction in the final execution instruction based on the current assembly stage, the confidence level of the sensing data, and the historical assembly success rate.

6. The method according to claim 1, characterized in that, The method also includes a rapid adaptation step when switching product models, including the following steps: S01. Import the three-dimensional design model and assembly process requirements of the new product model's battery pack into the digital twin model; S02. In the virtual environment provided by the digital twin model, the assembly process of the new product model is simulated and the strategy is optimized to generate an initial collaborative control strategy and assembly parameter set for the model. S03. The optimized strategy and parameter set are transformed into a lightweight control module that can run on the edge controller of the production line through model compression technology. S04. Perform trial assembly of the new product model on the physical production line, and based on the actual data generated from the trial assembly, perform online adaptive fine-tuning of the lightweight control module until the assembly quality requirements are met.

7. The method according to claim 6, characterized in that, The model compression technique described is a knowledge distillation method, which achieves equivalent performance by having a lightweight control module learn the behavior of an expert control model in a simulation environment.

8. The method according to claim 1, characterized in that, The multimodal sensing unit includes: A 3D vision sensor is used to acquire depth and color images of the assembly area to reconstruct the 3D information of the workpiece; A six-dimensional force / torque sensor is installed on the robot's end effector to measure multi-dimensional contact forces and torques during the assembly process; An inertial measurement unit is used to monitor minute displacements and vibrations of parts or fixtures to be assembled.

9. A machine vision-based flexible battery pack assembly line and deviation collaborative control system, used to implement the method described in any one of claims 1-8, characterized in that, The system includes: A digital twin and strategy management platform is used to run the digital twin model, store and manage the key feature point network, historical deviation data and collaborative control strategies; Sensing and execution units deployed at each assembly station include multimodal sensing units, as well as execution mechanisms including robots and fixtures; Edge collaborative control nodes, deployed on the production line or workstation side, have the collaborative decision-making module built in, which is used to receive strategies and data from the platform and generate the final execution instructions in real time; A high-speed industrial communication network connects all the above components, ensuring real-time and reliable transmission of data and instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-8.

Images