Multimodal sensing battery disassembly method, system and data acquisition device
By constructing a knowledge base for power battery dismantling through multimodal sensing technology and digital twin technology, the problem of difficulty in structurally expressing experience during the power battery dismantling process has been solved, and the dismantling sequence and risk handling have been digitally recorded, thereby improving dismantling efficiency and safety.
Patent Information
- Application Number
- CN202611327816.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-31
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2046-08-31
AI Technical Summary
Existing technologies lack a systematic collection of multi-source data during the dismantling of power batteries, making it difficult to express operational experience in a structured way, which affects dismantling efficiency and safety.
Multimodal perception technology is used to acquire visual information, motion trajectory information, muscle activity information, and tool mechanics information to build a knowledge base for power battery disassembly. This knowledge base is then combined with digital twin technology for virtual training and practical disassembly guidance.
It enables the digital recording of disassembly sequence, structural understanding, and risk management experience, and provides auxiliary guidance for virtual disassembly training and actual disassembly, thereby improving disassembly efficiency and safety.
Smart Images

Figure CN122843577B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of battery disassembly technology, and particularly relates to a multimodal sensing battery disassembly operation method, system and data acquisition device. Background Technology
[0002] With the rapid development of the new energy vehicle industry, a large number of power batteries are gradually entering the retirement stage. The internal structure of power batteries is complex, and their operation relies heavily on the practical experience of the operators. The disassembly experience and knowledge gained by operators over a long period of disassembly is an important basis for ensuring disassembly efficiency and operational safety.
[0003] However, existing technologies lack a systematic collection of multi-source data such as action behavior information and tool operation information during the disassembly process, making it difficult to express relevant experience in a structured way. Summary of the Invention
[0004] To overcome the shortcomings of the prior art, this invention provides a multimodal sensing battery disassembly operation method, system, and data acquisition device. It acquires multi-source feature data such as visual information, movement trajectory information, muscle activity information, and mechanical information of disassembly tools during the battery disassembly process. This enables the digital recording of key knowledge such as disassembly sequence, structural understanding, and risk management experience, thereby constructing a disassembly knowledge base containing disassembly strategies and operational experience. Combined with digital twin technology, it enables intelligent virtual disassembly training and actual disassembly scheme planning for power batteries.
[0005] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a multimodal sensing method for battery disassembly.
[0006] A multimodal sensing battery disassembly method includes the following steps: Collect multi-source heterogeneous data during the actual disassembly of battery samples, including power battery status data, operator behavior status data, and tool execution status data, and build a power battery disassembly knowledge base; Obtain the initial structural information of the power battery to be disassembled and construct a three-dimensional structural model of the power battery; By combining the three-dimensional structural model of the power battery and the power battery disassembly knowledge base, a digital twin disassembly training environment is constructed, and virtual training samples are generated for operators to train in disassembly operations. After completing the virtual training, the operator carried out the actual disassembly of the power battery to be disassembled, using the power battery disassembly knowledge base to provide guidance for the actual disassembly operation, and completed the disassembly of the power battery to be disassembled.
[0007] A second aspect of the present invention provides a multimodal sensing battery disassembly system.
[0008] A multimodal sensing battery disassembly system includes: The knowledge base construction module is configured to: collect multi-source heterogeneous data during the actual disassembly of battery samples, including power battery status data, operator behavior status data and tool execution status data, and construct a power battery disassembly knowledge base; The battery model building module is configured to: obtain the initial structural information of the power battery to be disassembled and build a three-dimensional structural model of the power battery; The virtual training module is configured to: combine the three-dimensional structural model of the power battery and the power battery disassembly knowledge base to build a digital twin disassembly training environment and generate virtual training samples for operators to train in disassembly operations; The real disassembly module is configured to: after completing virtual training, the operator carries out a real disassembly operation of the power battery to be disassembled, using the power battery disassembly knowledge base to provide guidance for the real disassembly operation, and complete the disassembly of the power battery to be disassembled.
[0009] A third aspect of the present invention provides a multimodal sensing battery disassembly data acquisition device.
[0010] A data acquisition device for a battery disassembly operation method applied to the first aspect of multimodal sensing includes: A power battery status data acquisition device includes an annular mounting frame positioned above the disassembly station and covering the entire disassembly station area, with multiple cameras mounted on the annular mounting frame; a radial connecting rod is provided inside the annular mounting frame, and a depth camera, an infrared thermal imager, and a smoke sensor are mounted on the radial connecting rod. Operator behavior status data acquisition device, including: Motion capture devices worn on the operator's body are used to acquire the operator's movement trajectory and posture changes during the disassembly process; An eye-tracking device worn on the operator's head is used to acquire the operator's visual attention area and gaze trajectory; An electromyography (EMG) acquisition device is installed on the operator's upper limb to collect data on muscle activity intensity and fatigue status. A physiological monitoring device worn on the operator's wrist to collect information on heart rate and other physiological states.
[0011] The above one or more technical solutions have the following beneficial effects: This invention provides a multimodal sensing battery disassembly operation method, system, and data acquisition device. By acquiring multi-source feature data such as visual information, motion trajectory information, muscle activity information, and mechanical information of disassembly tools during the expert disassembly process, it achieves digital recording of key knowledge such as disassembly sequence, structural understanding, and risk management experience, thereby constructing a disassembly knowledge base containing disassembly strategies and operational experience. By setting up a digital twin VR interactive area, and combining the digital twin model and the disassembly knowledge base to generate multi-condition virtual training samples, it provides operators with a virtual disassembly training platform, enabling operators to conduct power battery disassembly training under low-risk conditions. The disassembly knowledge base is used to guide and evaluate the training process. Finally, AR devices are used to overlay and display disassembly process, risk warnings, and other information on the real disassembly scene, realizing auxiliary guidance for operators in the real disassembly process.
[0012] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0013] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0014] Figure 1 This is a flowchart of Embodiment 1 of the present invention.
[0015] Figure 2 This is a schematic diagram of digital twin VR interaction according to Embodiment 3 of the present invention.
[0016] Figure 3 This is a three-dimensional structural schematic diagram of the device according to Embodiment 3 of the present invention.
[0017] Figure 4 This is a top view of the device according to Embodiment 3 of the present invention.
[0018] Figure 5 This is a schematic diagram of the operator behavior status data acquisition device according to Embodiment 3 of the present invention.
[0019] In the picture: 1. Insulated operating table; 2. Fixed platform; 3. Guide rail; 4. Adjustable clamping mechanism; 5. Electrostatic grounding structure; 6. Ring mounting bracket; 7. Radial connecting rod; 8. RGB camera; 9. Depth camera; 10. Infrared thermal imager; 11. Motion capture device; 12. Eye tracking device; 13. Electromyography (EMG) acquisition device; 14. Physiological monitoring device; 15. Disassembly toolbox; 16. Electric screwdriver; 17. Torque wrench; 18. Prying and dismantling tools; 19. Torque sensor; 20. Force sensor; 21. Displacement sensor; 22. Temperature sensor; 23. Smoke sensor; 24. Alarm; 25. Parts storage device; 26. Parts storage rack; 27. Controller; 28. VR headset; 29. Force feedback interactive handle; 30. 3D display terminal; 31. Augmented reality headset. Detailed Implementation
[0020] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0021] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.
[0022] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.
[0023] Example 1 To address the problem of the difficulty in effectively acquiring and utilizing experiential knowledge during the existing power battery dismantling process, this invention collects information on the actions and behaviors of experts, tool operation information, and power battery status information during the dismantling process through various sensing devices. Based on the acquired multi-source heterogeneous data, a power battery dismantling knowledge base is constructed to realize the structured expression of dismantling experience, providing data and knowledge support for intelligent power battery dismantling training and auxiliary operation guidance.
[0024] In addition, relying on the established knowledge base for power battery disassembly, a digital twin disassembly training environment is constructed, and virtual training samples are generated for operators to conduct disassembly operation training. This can cover a variety of working conditions and achieve enhanced training for operators.
[0025] Specifically, such as Figure 1 As shown, this embodiment discloses a multimodal sensing battery disassembly method, including the following steps: Collect multi-source heterogeneous data during the actual disassembly of battery samples, including power battery status data, operator behavior status data, and tool execution status data, and build a power battery disassembly knowledge base; Obtain the initial structural information of the power battery to be disassembled and construct a three-dimensional structural model of the power battery; By combining the three-dimensional structural model of the power battery and the power battery disassembly knowledge base, a digital twin disassembly training environment is constructed, and virtual training samples are generated for operators to train in disassembly operations. After completing the virtual training, the operator carried out the actual disassembly of the power battery to be disassembled, using the power battery disassembly knowledge base to provide guidance for the actual disassembly operation, and completed the disassembly of the power battery to be disassembled.
[0026] This embodiment provides an AR-assisted power battery disassembly method based on digital twins. Overall, it involves: performing real disassembly and collecting multi-source heterogeneous data to construct a power battery disassembly knowledge base; establishing a digital twin disassembly training environment; acquiring the initial structural information of the power battery to be disassembled and constructing a three-dimensional structural model of the power battery; performing virtual mutation based on sample data in the disassembly knowledge base to generate multi-condition virtual training samples; the operator using VR interactive devices to conduct virtual disassembly training and using the disassembly knowledge base to guide and evaluate the training process; and the operator using AR devices to perform real disassembly, using the power battery disassembly knowledge base to provide guidance for the real disassembly operation, thus completing the disassembly of the power battery to be disassembled.
[0027] This embodiment relies on the combined implementation of digital twin VR interaction technology and multimodal perception technology. Digital twin VR interaction technology is mainly used for training on the virtual disassembly process of power batteries. In this embodiment, a digital twin VR interaction device is provided, specifically including a digital twin modeling unit, a VR headset, a force feedback interactive handle, a 3D display terminal, and an augmented reality headset, wherein: In training on the virtual disassembly process of power batteries, the application of digital twin VR interactive technology includes: Digital twin modeling units are used to construct three-dimensional structural models of power batteries; VR headsets are worn on the operator's head to display a virtual disassembly scene. Force feedback interactive handles are used to simulate the force feedback effect of real disassembly; The 3D display terminal is used to display information about the structure and risk areas of the power battery.
[0028] The VR headset, force feedback interactive controller, and 3D display terminal mentioned above can all use existing mature equipment, and will not be described in detail in this embodiment.
[0029] Furthermore, in the process of conducting actual disassembly of power batteries, the application of digital twin VR interactive technology includes: Augmented reality head-mounted displays are worn on the operator's head to overlay disassembly instructions onto the actual disassembly scene, providing auxiliary guidance for the operator during the actual disassembly process.
[0030] In this embodiment, multimodal sensing technology is mainly applied to the acquisition of multi-source heterogeneous data during the actual disassembly of battery samples. Specifically, it is necessary to collect power battery status data, operator behavior status data, and tool execution status data. The power battery status data includes images and videos of the battery samples during the disassembly process, spatial pose information, surface temperature information, and smoke information. Operator behavior status data, including operator movement trajectory information, visual attention area and gaze trajectory, muscle activity status and physiological status information; The tool execution status data includes multidimensional mechanical information.
[0031] in: The method for obtaining power battery status data is as follows: Establish a disassembly station for operators to actually disassemble battery samples; Set up a ring-shaped mounting frame that can cover the disassembly station, and install radial connecting rods inside the ring-shaped mounting frame; Multiple cameras are installed on the ring mounting bracket to collect image and video information of the battery samples; a depth camera, an infrared thermal imager, and a smoke sensor are installed on the radial connecting rod to collect spatial pose information, surface temperature information, and smoke information of the battery samples, respectively.
[0032] The method for obtaining operator behavior status data is as follows: By using motion capture devices worn on the operator's body, the operator's movement trajectory and posture changes during the disassembly process can be obtained; By using an eye-tracking device worn on the operator's head, the operator's visual attention area and gaze trajectory can be obtained. The intensity of muscle activity and fatigue status are collected by an electromyography (EMG) acquisition device installed on the operator's upper limb. Heart rate and other physiological status information are collected through a physiological monitoring device worn on the operator's wrist.
[0033] The method for obtaining tool execution status data is as follows: The torque sensor integrated into the electric screwdriver and torque wrench acquires the applied torque information in real time. Force and displacement sensors integrated into the prying and dismantling tools are used to acquire information on mechanical changes and displacement during the prying and dismantling process.
[0034] After acquiring the aforementioned multi-source heterogeneous data, a power battery disassembly knowledge base was constructed based on the multi-source heterogeneous data corresponding to each battery sample. During the construction of the power battery disassembly knowledge base: Based on multi-source heterogeneous data, product structure data, operator multimodal human factors data, tool status data, and risk status data are determined in the power battery disassembly knowledge base. The product structure data includes the three-dimensional structural model of the power battery, component composition, spatial location, and initial assembly relationship; Operator multimodal human factors data includes motion trajectories, visual attention areas, and electromyographic signals; Tool status data includes tool type, contact area, direction of movement, operation time, and force, torque, or speed parameters; Risk status data includes information on temperature, voltage, component damage, and abnormal operating environment.
[0035] The further data processing flow is as follows: Retired power batteries were selected as samples, and the above-mentioned multi-source heterogeneous data were collected simultaneously during the actual dismantling process. The data was then denoised, time-synchronized, and correlated to ensure that the product structure data, operator multimodal human factors data, tool status data, and risk status data at the same dismantling moment corresponded to each other.
[0036] Further: Based on the start and end times of the motion trajectory, the visual focus object, the contact state between the tool and the component, and the state changes of the component before and after disassembly, the disassembly task performed by the operator is identified, the continuous disassembly process is divided into multiple operation steps, and the disassembly task sequence, preconditions and operation steps are determined according to the occurrence order and dependencies of each step, so as to obtain the disassembly process information. Based on the type of tool, target object, contact position, direction of movement, operation time and operation parameters of each disassembly task, and in combination with the result of whether the parts were successfully separated, determine the tools and operation methods required for the corresponding task, and obtain tool operation information; Match the component positions and initial assembly relationships in the 3D structural model with the movement, detachment, and separation states of the components during the actual disassembly process to determine the interconnected components, connection positions, connection methods, connection release conditions, and disassembly directions, thereby obtaining component connection relationship information. By correlating changes in temperature, voltage, component damage, and environmental anomalies with the operator's gaze shift, electromyography changes, pauses in movement, withdrawal movements, and tool operation adjustments, abnormal characteristics before and after a risk occur and the operator's risk avoidance behavior are identified. This allows for the determination of the risk type, the risk component, the triggering conditions, and the corresponding response strategies, thereby obtaining risk information.
[0037] Based on this, the operator's intentions for execution, risk avoidance and strategy adjustment under different disassembly tasks are analyzed, and the disassembly process, tool operation information, component connection relationship and risk information obtained from different disassembly samples are summarized, associated and structured for storage, forming a power battery disassembly knowledge base.
[0038] Existing technologies typically rely on product structure models and pre-set disassembly procedures, making it difficult to fully utilize the operational experience and risk awareness gained by operators during actual disassembly. The power battery disassembly knowledge base constructed in this invention can structurally store and reuse disassembly procedures, tool operations, component connection relationships, and risk management strategies. This provides a basis for configuring the interactive attributes of the three-dimensional structural model, thereby improving the realism, accuracy, and safety of the digital twin disassembly environment and enhancing its adaptability to different disassembly tasks and abnormal operating conditions.
[0039] Furthermore, in the process of obtaining the initial structural information of the power battery to be disassembled and constructing a three-dimensional structural model of the power battery, the initial structural information of the power battery to be disassembled includes the model information, structural composition information, connection relationship information and related parameter information of the power battery to be disassembled.
[0040] The specific process and method for constructing the three-dimensional structural model of the power battery in this embodiment include: Based on the model information, structural composition information, connection relationship information and related parameter information of the power battery to be disassembled, obtain the three-dimensional assembly model corresponding to the power battery to be disassembled; The three-dimensional assembly model includes multiple independent components such as the power battery casing, top cover, module, battery cell, busbar, wiring harness and connectors; The assembly, connection, and hierarchical relationships among the various components are established to obtain a three-dimensional structural model of the power battery to be disassembled, which serves as the basic model for subsequent digital twin disassembly training.
[0041] Next, combining the 3D structural model of the power battery and the power battery disassembly knowledge base, a digital twin disassembly training environment will be constructed. The specific method is as follows: The three-dimensional structural model of the power battery is imported into the digital twin platform and used as a virtual disassembly object; By combining the disassembly process, tool operation information, component connection relationship and risk information recorded in the power battery disassembly knowledge base, interactive attribute configuration is performed on the three-dimensional structural model so that each component can be disassembled step by step according to the actual disassembly process. Establish a data link between the virtual disassembly environment and the power battery disassembly knowledge base to provide a digital twin environment for generating virtual training samples and conducting disassembly operation training.
[0042] By combining the disassembly process, tool operation information, component connection relationships and risk information recorded in the power battery disassembly knowledge base, interactive attribute configuration is performed on the 3D structural model.
[0043] Next, the battery state data recorded in the power battery disassembly knowledge base is used to perform parameter mutations on the structural state, damage state, connection state, and risk state of the power battery to be disassembled, generating virtual training samples under different operating conditions, specifically including: By combining the power battery status data recorded in the power battery disassembly knowledge base, at least one of the status parameters of the power battery's structural status, damage status, connection status, and risk status is adjusted or combined to generate virtual training samples under different operating conditions. These different operating conditions include missing components, deformed components, loose connections, abnormal temperature, and abnormal power.
[0044] As a further optimization scheme, during the parameter variation process, the current structural state, damage state, connection state and risk state of the power battery to be disassembled are used as the basis, and the state parameters are adjusted within a limited range in combination with the historical state data in the power battery disassembly knowledge base, so as to generate virtual training samples of different working conditions that match the power battery to be disassembled, thereby improving the relevance and realism of the disassembly training.
[0045] It can be understood that the above process first involves assessing the condition of the power battery to be disassembled. Based on the incoming material information of the power battery and the results of multimodal perception and recognition, the structural condition, damage condition, connection condition, and risk condition of the power battery to be disassembled are determined. The incoming material information includes the power battery model, production information, usage history, maintenance records, and historical condition information; the multimodal perception and recognition results include information on the appearance and structure of the power battery obtained using an RGB camera and a depth camera, and information on the temperature distribution of the power battery obtained using an infrared thermal imager.
[0046] After obtaining the above information, a comprehensive analysis is performed using a power battery disassembly knowledge base to complete the state assessment of the power battery to be disassembled. The comprehensive analysis process can adopt mature practices from existing technologies, and this embodiment does not impose any limitations on it.
[0047] In this embodiment, the operator enters the digital twin disassembly training environment and performs disassembly operation training on a virtual training sample of a power battery in a virtual scene. During the disassembly operation training, the rationality of the operator's disassembly sequence and risk response capabilities are evaluated, and the power battery structure information, disassembly progress information, and risk area information are displayed in real time.
[0048] In practice: During the disassembly operation training, the operator's current disassembly steps are compared in real time with the standard disassembly process in the power battery disassembly knowledge base to evaluate the rationality of the operator's disassembly sequence. The operator's risk response capability is assessed based on their response to risk warnings, tool selection, and whether the disassembly operation complies with safety regulations. The system updates the power battery structure information, disassembled component information, disassembly progress information, and risk area information in real time during the disassembly process, and displays them through a 3D display terminal and an augmented reality head-mounted display device.
[0049] Existing disassembly training methods struggle to effectively cover disassembly scenarios under varying structural states and risky operating conditions. The training process lacks support from real-world disassembly experience, limiting its effectiveness. This embodiment addresses this by combining a digital twin model and a disassembly knowledge base to generate multi-condition virtual training samples. This provides operators with a virtual disassembly training platform, enabling them to conduct power battery disassembly training under low-risk conditions. Furthermore, the disassembly knowledge base guides and evaluates the training process, enhancing the training effectiveness for operators.
[0050] After completing virtual training, operators conduct real disassembly operations of the power battery to be disassembled. A power battery disassembly knowledge base provides guidance for the real disassembly operation, enabling the complete disassembly of the power battery. Specifically, this includes: Based on the incoming material information of the power battery to be disassembled and the results of multimodal perception and recognition, the disassembly scheme that matches the current power battery is called from the power battery disassembly knowledge base. Provide operators with real-time disassembly guidance information, including disassembly sequence, tool selection, disassembly location, and precautions, in accordance with the disassembly plan; During the disassembly process, the disassembly guidance information is dynamically updated based on the real-time power battery status information obtained by the multimodal sensing device, and warnings are issued for high-risk areas. The operator completes the actual disassembly of the power battery to be disassembled according to the disassembly instructions.
[0051] In a specific application, such as Figure 1 As shown, a complete technical architecture of a power battery intelligent dismantling system based on multimodal perception and digital twin technology is presented, covering four functional levels: physical dismantling device, multi-source data perception and acquisition, knowledge base construction and digital twin modeling, and VR / AR assisted training and practical guidance.
[0052] The four main functional levels specifically include: The first level is the disassembly device, representing the physical hardware used in the disassembly of the power battery, which constitutes the underlying execution carrier of the entire system. Specifically, it includes an insulated operating table, a fixed platform, guide rails, an adjustable clamping mechanism, and a static grounding structure.
[0053] The second layer is multimodal sensing, which is the core layer for data acquisition. This layer branches out with four lines, each corresponding to one of the four acquisition groups, and further subdivides into specific sensors or devices: (1) Battery status information acquisition device, including an RGB camera, a depth camera and an infrared thermal imager; (2) Operator behavior state acquisition device, including motion capture device, eye tracking device, electromyography acquisition device and physiological monitoring device; (3) Tool execution status acquisition device, including electric screwdriver, torque wrench and prying tools; (4) Risk status acquisition device, including temperature sensor, smoke sensor and alarm.
[0054] The outputs of the four acquisition groups all converge downwards via thin arrows, pointing to the "Controller" box at the bottom, indicating that the sensing data from all sources are uniformly fed into the controller for centralized processing.
[0055] The third level involves building a deconstructed knowledge base and establishing a digital twin model.
[0056] The logical flow at this level unfolds from left to right: Based on the initial information of the power battery and multi-source heterogeneous fusion data, a power battery disassembly knowledge base including disassembly sequence, risk handling experience and structural understanding is determined, and then a digital twin model is constructed, completing the transformation from knowledge accumulation to model building.
[0057] The fourth level consists of VR virtual disassembly training and AR-assisted disassembly.
[0058] The logical flow at this level is as follows: By generating virtual multi-condition training samples, operators can complete VR virtual disassembly training; during the VR virtual disassembly training process, the power battery disassembly knowledge base supports the VR training. After VR virtual disassembly training is completed, AR-assisted real disassembly is carried out. During the AR-assisted real disassembly process, disassembly step prompts and risk warnings are provided.
[0059] Specifically, the steps include the following: S1. Obtain the model information, structural composition information, connection relationship information and related parameter information of the power battery to be disassembled; S2. Experts complete the disassembly of the power battery within the multimodal sensing disassembly area. RGB cameras, depth cameras, and infrared thermal imagers collect power battery status information; motion capture equipment, eye-tracking equipment, electromyography (EMG) acquisition equipment, and physiological monitoring equipment collect behavioral status information of the experts during the disassembly process; torque sensors, force sensors, and displacement sensors collect tool execution status information; temperature sensors and smoke sensors collect risk status information during the disassembly process. The controller receives and stores the above multi-source heterogeneous information and constructs a power battery disassembly knowledge base.
[0060] S3. Based on the initial structural information of the power battery obtained in step S1, construct a three-dimensional structural model of the power battery, and combine it with the multi-source heterogeneous data obtained in step S2 to construct a digital twin disassembly training environment. S4. Using the power battery status data recorded in the disassembly knowledge base, perform parameter variation on the power battery structural status, damage status, connection status and risk status to generate virtual training samples under different working conditions. S5. The operator wears a VR headset and enters the digital twin disassembly training environment via a force feedback interactive handle. The operator performs disassembly training on a power battery training sample in a virtual scene. During the training, the rationality of the operator's disassembly sequence and risk response capabilities are evaluated; a 3D display terminal is used to display power battery structural information, disassembly progress information, and risk area information.
[0061] S6. After completing virtual training, the operator enters the multimodal perception disassembly area to carry out real disassembly operations. The augmented reality head-mounted display overlays disassembly step information and risk warning information onto the disassembly workstation area in real time, providing dynamic assistance and guidance to the operator during the actual disassembly process.
[0062] This embodiment combines a power battery disassembly knowledge base, digital twin technology, virtual reality technology, and augmented reality technology to achieve a closed-loop application of expert disassembly experience collection, knowledge base construction, virtual disassembly training, and real disassembly assistance guidance, providing technical support for intelligent disassembly training and disassembly scheme planning for power batteries.
[0063] Example 2 This embodiment discloses a multimodal sensing battery disassembly system.
[0064] A multimodal sensing battery disassembly system includes: The knowledge base construction module is configured to: collect multi-source heterogeneous data during the actual disassembly of battery samples, including power battery status data, operator behavior status data and tool execution status data, and construct a power battery disassembly knowledge base; The battery model building module is configured to: obtain the initial structural information of the power battery to be disassembled and build a three-dimensional structural model of the power battery; The virtual training module is configured to: combine the three-dimensional structural model of the power battery and the power battery disassembly knowledge base to build a digital twin disassembly training environment and generate virtual training samples for operators to train in disassembly operations; The real disassembly module is configured to: after completing virtual training, the operator carries out a real disassembly operation of the power battery to be disassembled, using the power battery disassembly knowledge base to provide guidance for the real disassembly operation, and complete the disassembly of the power battery to be disassembled.
[0065] Example 3 The purpose of this embodiment is to provide a data acquisition device for a multimodal sensing battery disassembly operation method.
[0066] To establish a functional entity for the data acquisition device provided in this embodiment, the disassembly station is described first. The disassembly station is used for the actual disassembly of the power battery. Specifically, as... Figure 3 and Figure 4 As shown: The disassembly station includes an insulated operating table 1, a fixed platform 2, guide rails 3, an adjustable clamping mechanism 4, and an electrostatic grounding structure 5, wherein: The insulated operating table 1 serves as the basic load-bearing structure, and its surface is provided with an anti-static insulation layer to reduce the risk of static electricity accumulation during disassembly. The fixed platform 2 is located in the central area of the insulated operating table 1 and is used to support the power battery; Guide rails 3 are installed around the fixed platform 2; The adjustable clamping mechanism 4 is mounted on the guide rail 3 and can be adjusted in position and extended along the guide rail 3 to adapt to the fixing requirements of power batteries of different sizes and shapes. The static electricity grounding structure 5 is installed inside the insulated operating table 1 and connected to the external grounding line to discharge the static electricity generated during the disassembly process, thereby improving operational safety.
[0067] In this embodiment, the disassembly station area is provided with an insulated operating table 1. The surface of the insulated operating table 1 is covered with an anti-static insulating layer to reduce the risk of static electricity accumulation during disassembly. The fixed platform 2 is set in the central area of the insulated operating table 1, and guide rails 3 are set around its surface. An adjustable clamping mechanism 4 is set on the guide rails 3. The adjustable clamping mechanism 4 can extend and retract to clamp and fix power batteries of different sizes. The static electricity grounding structure 5 is set inside the insulated operating table 1 and connected to the grounding line to realize static electricity discharge.
[0068] Next, the data acquisition device provided in this embodiment will be described in detail.
[0069] Overall, such as Figure 3 and Figure 4 As shown, the data acquisition device in this embodiment includes a top ring mounting bracket 6, an RGB camera 8, a depth camera 9, an infrared thermal imager 10, a motion capture device 11, an eye-tracking device 12, an electromyography (EMG) acquisition device 13, and a physiological monitoring device 14. The RGB camera 8, depth camera 9, infrared thermal imager 10, motion capture device 11, eye-tracking device 12, EMG acquisition device 13, and physiological monitoring device 14 are respectively connected to the controller 27. The aforementioned devices are functionally categorized as a power battery status data acquisition device, an operator behavior status data acquisition device, and a tool execution status data acquisition device.
[0070] Specifically, the data acquisition device applied to the battery disassembly operation method of multimodal sensing in Embodiment 1 includes: (1) A power battery status data acquisition device, including an annular mounting frame 6 set above the disassembly station and covering the entire disassembly station area, and multiple RGB cameras 8 set on the annular mounting frame 6; a radial connecting rod 7 is set inside the annular mounting frame 6, and a depth camera 9, an infrared thermal imager 10 and a smoke sensor 23 are set on the radial connecting rod 7.
[0071] A ring-shaped mounting bracket 6 is arranged in a circular pattern above the disassembly station area to achieve global coverage of the disassembly area. Multiple RGB cameras 8 are evenly distributed on the ring-shaped mounting bracket 6 to collect images and video information from different spatial perspectives during the power battery disassembly process, thereby enabling visual recording of the operation. The annular mounting bracket 6 is also equipped with radial connecting rods 7, which are located inside the annular mounting bracket 6 and connect opposite sides of the annular mounting bracket 6. A depth camera 9 is located at the middle of the radial connecting rod 7 to acquire the spatial pose information of the power battery, enabling the perception of changes in the three-dimensional structure of the power battery during disassembly. An infrared thermal imager 10 is located at the middle of the radial connecting rod 7, facing the center area of the disassembly station, to collect temperature distribution information on the surface of the power battery.
[0072] Furthermore, the RGB camera 8, depth camera 9, and infrared thermal imager 10 are connected to the controller 27 to collect the status information of the power battery in real time.
[0073] (2) such as Figure 5 As shown, the operator behavior status data acquisition device includes: A motion capture device 11 worn on the operator's body is used to acquire the operator's movement trajectory and posture changes during the disassembly process; An eye-tracking device 12 worn on the operator's head is used to acquire the operator's visual attention area and gaze trajectory. The electromyography (EMG) acquisition device 13, installed on the operator's upper limb, is used to collect muscle activity intensity and fatigue status. A physiological monitoring device 14 worn on the operator's wrist is used to collect information on heart rate and other physiological states.
[0074] Among the various wearable or peripheral sensing devices for collecting operator status information, the motion capture device 11 is worn on the operator's body to acquire the operator's movement trajectory and posture changes during the disassembly process; the eye-tracking device 12 is worn on the operator's head to acquire the operator's visual attention area and gaze trajectory; the electromyography acquisition device 13 is set on the operator's upper limb (arm) to collect muscle activity intensity and fatigue status during the disassembly process; and the physiological monitoring device 14 is worn on the operator's wrist to collect heart rate and other physiological status information to reflect the operator's workload.
[0075] The motion capture device 11, eye tracking device 12, electromyography acquisition device 13, and physiological monitoring device 14 are all connected to the controller 27 to collect information on the expert's action status, visual attention status, muscle activity status, and physiological status during the disassembly of the power battery in real time.
[0076] (3) It also includes a tool execution status data acquisition device, including: The torque sensor 19, integrated inside the electric screwdriver 16 and torque wrench 17, is used to acquire applied torque information in real time. The force sensor 20 and displacement sensor 21 integrated on the prying tool 18 are used to acquire mechanical changes and displacement information during the prying process. The electric screwdriver 16, torque wrench 17 and prying tools 18 are stored inside the disassembly toolbox 15, which is located on one side of the disassembly station.
[0077] A disassembly toolbox 15 is located on one side of the disassembly workstation area for storing and managing disassembly tools. An electric screwdriver 16, a torque wrench 17, and prying tools 18 are stored inside the disassembly toolbox 15 for performing different types of disassembly operations. A torque sensor 19 is integrated into the electric screwdriver 16 and torque wrench 17 to acquire applied torque information in real time; a force sensor 20 and a displacement sensor 21 are integrated into the prying tools 18 to acquire mechanical changes and displacement information during the prying process.
[0078] Torque sensor 19, force sensor 20 and displacement sensor 21 are all connected to controller 27.
[0079] The controller 27 is connected to the torque sensor 19, the force sensor 20 and the displacement sensor 21 respectively, and is used to collect the mechanical state information of the disassembly tool in real time during the disassembly process.
[0080] (4) It also includes a risk monitoring device, which is set in the dismantling station area and includes a temperature sensor 22, a smoke sensor 23 and an alarm 24.
[0081] Temperature sensor 22 is installed on the surface of the insulated operating table 1 to monitor the temperature change of the power battery in real time; smoke sensor 23 is installed on the radial connecting rod 7, on one side of the depth camera 9, to monitor the smoke or abnormal gas generated during disassembly; alarm 24 is installed in the surrounding area of the insulated operating table 1 and connected to controller 27. When temperature sensor 22 detects that the power battery is in an abnormal temperature state or smoke sensor 23 detects an abnormal smoke state, controller 27 controls alarm 24 to issue an audible and visual alarm.
[0082] In this embodiment, a parts storage device 25 is also provided, which is located on one side of the disassembly station and is used for the storage and management of disassembled parts. It includes a parts storage rack 26, which is used to store different types of disassembled parts in separate areas.
[0083] This embodiment can not only be used for the disassembly of power batteries, but also to collect and record multimodal sensing information during the disassembly process of power batteries by experts, build a power battery disassembly knowledge base, realize the digital storage of expert disassembly experience, and provide knowledge support for subsequent intelligent disassembly training and disassembly scheme planning of power batteries.
[0084] In this embodiment, the power battery disassembly station is located in the central area of the platform, data acquisition devices are arranged around the perimeter of the power battery disassembly station, risk monitoring devices are arranged around and below the power battery disassembly station, and parts storage device 25 is arranged on one side of the disassembly station. The disassembly station is used to disassemble power batteries; the data acquisition devices are used to collect data in real time on the power battery status, operator behavior, and tool execution status during the actual disassembly process; the risk monitoring devices are used to monitor the risk status during the power battery disassembly process and issue early warnings for abnormal situations; the parts storage device 25 is used to classify, store, and manage the components after the power battery is disassembled.
[0085] By setting up a disassembly station and equipping it with an adjustable clamping mechanism 4, a liftable support frame, a rotatable mechanism, and a risk monitoring device, the system can achieve stable clamping and disassembly of the power battery. At the same time, it can monitor and warn of the risk status during the disassembly process in real time, thereby improving the safety of the power battery disassembly process.
[0086] like Figure 2As shown, the operator wears a VR headset 28 and enters the digital twin disassembly training environment via a force feedback interactive handle 29. The operator trains to disassemble a power battery training sample in a virtual scene. During the training, the rationality of the operator's disassembly sequence and risk response capabilities are evaluated; the 3D display terminal 30 displays power battery structural information, disassembly progress information, and risk area information. After completing the virtual training, the operator wears an augmented reality headset 31 to enter the multimodal perception disassembly area to carry out real disassembly operations. The augmented reality headset 31 overlays disassembly step information and risk warning information in real time on the disassembly workstation area, providing dynamic auxiliary guidance for the operator's real disassembly process.
[0087] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.
[0088] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.
Claims
1. A multimodal sensing battery disassembly method, characterized in that, Includes the following steps: Collect multi-source heterogeneous data during the actual disassembly of battery samples, including power battery status data, operator behavior status data, and tool execution status data, and build a power battery disassembly knowledge base; Obtain the initial structural information of the power battery to be disassembled and construct a three-dimensional structural model of the power battery; By combining the three-dimensional structural model of the power battery and the power battery disassembly knowledge base, a digital twin disassembly training environment is constructed, and virtual training samples are generated for operators to train in disassembly operations. After completing the virtual training, the operator carried out the actual disassembly of the power battery to be disassembled, using the power battery disassembly knowledge base to provide guidance for the actual disassembly operation and complete the disassembly of the power battery to be disassembled. The power battery status data includes image and video information, spatial pose information, surface temperature information, and smoke information of the battery sample during the disassembly process. The operator's behavioral state data includes the operator's movement trajectory information, visual attention area and gaze trajectory, muscle activity state and physiological state information; The tool's execution status data includes multidimensional mechanical information; Based on the multi-source heterogeneous data corresponding to each battery sample, a power battery disassembly knowledge base is constructed, specifically including: Based on multi-source heterogeneous data, product structure data, operator multimodal human factors data, tool status data, and risk status data are determined in the power battery disassembly knowledge base. For each power battery sample, the product structure data, operator multimodal human factors data, tool status data, and risk status data are synchronized and matched over time. Based on the operator's movement trajectory, visual focus, contact state between tools and parts, and changes in the state of parts before and after disassembly, determine the disassembly process information; Determine the tool operation information based on the tool type, target object, contact position, and direction of movement; Based on the initial assembly relationship in the three-dimensional structural model of the power battery and the separation state of the components before and after disassembly, the component connection relationship information is determined. Risk information is determined based on temperature, voltage, component damage, abnormal operating environment, and operator risk avoidance behavior; The disassembly process information, tool operation information, component connection relationships, and risk information are associated and stored in a structured manner to form a power battery disassembly knowledge base.
2. The multimodal sensing battery disassembly method as described in claim 1, characterized in that, The method for obtaining the power battery status data is as follows: Establish a disassembly station for operators to actually disassemble battery samples; Set up a ring-shaped mounting frame that can cover the disassembly station, and install radial connecting rods inside the ring-shaped mounting frame; Multiple cameras are installed on the ring mounting bracket to collect image and video information of the battery samples; a depth camera, an infrared thermal imager and a smoke sensor are installed on the radial connecting rod to collect spatial pose information, surface temperature information and smoke information of the battery samples, respectively. The operator behavior status data is obtained in the following way: By using motion capture devices worn on the operator's body, the operator's movement trajectory and posture changes during the disassembly process can be obtained; By using an eye-tracking device worn on the operator's head, the operator's visual attention area and gaze trajectory can be obtained. The intensity of muscle activity and fatigue status are collected by an electromyography (EMG) acquisition device installed on the operator's upper limb. Heart rate and other physiological status information are collected through a physiological monitoring device worn on the operator's wrist; The method for obtaining the tool's execution status data is as follows: The torque sensor integrated into the electric screwdriver and torque wrench acquires the applied torque information in real time. Force and displacement sensors integrated into the prying and dismantling tools are used to acquire information on mechanical changes and displacement during the prying and dismantling process.
3. The multimodal sensing battery disassembly method as described in claim 1, characterized in that, Product structure data includes the three-dimensional structural model of the power battery, component composition, spatial location, and initial assembly relationship; operator multimodal human factors data includes motion trajectory, visual attention area, and electromyographic signals; Tool status data includes tool type, contact point, direction of movement, operation time, and force, torque, or speed parameters; risk status data includes temperature, voltage, component damage, and abnormal operating environment information.
4. The multimodal sensing battery disassembly method as described in claim 1, characterized in that, The initial structural information of the power battery to be disassembled includes the model information, structural composition information, connection relationship information and related parameter information of the power battery to be disassembled. The related parameter information includes the external dimensions, weight, voltage, current, state of charge, health status and fastener specifications of the power battery.
5. The multimodal sensing battery disassembly method as described in claim 1, characterized in that, Using the power battery dismantling knowledge base, parameter variation is performed on the structural state, damage state, connection state, and risk state of the power battery to be dismantled, generating virtual training samples under different operating conditions, specifically including: By combining the power battery status data recorded in the power battery disassembly knowledge base, at least one of the status parameters of the power battery’s structural status, damage status, connection status and risk status is adjusted or combined to generate virtual training samples under different working conditions. The different operating conditions include missing parts, deformed parts, loose connections, abnormal temperature, and abnormal power.
6. The multimodal sensing battery disassembly method as described in claim 1, characterized in that, Also includes: The operator enters the digital twin disassembly training environment and performs disassembly operation training on a virtual training sample of a power battery in a virtual scene. During the disassembly operation training, the rationality of the operator's disassembly sequence and risk response capabilities are assessed, and the power battery structure information, disassembly progress information, and risk area information are displayed in real time.
7. A multimodal sensing battery disassembly system, characterized in that, include: The knowledge base construction module is configured to: collect multi-source heterogeneous data during the actual disassembly of battery samples, including power battery status data, operator behavior status data and tool execution status data, and construct a power battery disassembly knowledge base; The battery model building module is configured to: obtain the initial structural information of the power battery to be disassembled and build a three-dimensional structural model of the power battery; The virtual training module is configured to: combine the three-dimensional structural model of the power battery and the power battery disassembly knowledge base to build a digital twin disassembly training environment and generate virtual training samples for operators to train in disassembly operations; The real disassembly module is configured to: after completing virtual training, the operator carries out a real disassembly operation of the power battery to be disassembled, and uses the power battery disassembly knowledge base to provide guidance for the real disassembly operation, and completes the disassembly of the power battery to be disassembled; The power battery status data includes image and video information, spatial pose information, surface temperature information, and smoke information of the battery sample during the disassembly process. The operator's behavioral state data includes the operator's movement trajectory information, visual attention area and gaze trajectory, muscle activity state and physiological state information; The tool's execution status data includes multidimensional mechanical information; Based on the multi-source heterogeneous data corresponding to each battery sample, a power battery disassembly knowledge base is constructed, specifically including: Based on multi-source heterogeneous data, product structure data, operator multimodal human factors data, tool status data, and risk status data are determined in the power battery disassembly knowledge base. For each power battery sample, the product structure data, operator multimodal human factors data, tool status data, and risk status data are synchronized and matched over time. Based on the operator's movement trajectory, visual focus, contact state between tools and parts, and changes in the state of parts before and after disassembly, determine the disassembly process information; Determine the tool operation information based on the tool type, target object, contact position, and direction of movement; Based on the initial assembly relationship in the three-dimensional structural model of the power battery and the separation state of the components before and after disassembly, the component connection relationship information is determined. Risk information is determined based on temperature, voltage, component damage, abnormal operating environment, and operator risk avoidance behavior; The disassembly process information, tool operation information, component connection relationships, and risk information are associated and stored in a structured manner to form a power battery disassembly knowledge base.
8. A data acquisition device applied to the multimodal sensing battery disassembly operation method according to any one of claims 1-6, characterized in that, include: A power battery status data acquisition device includes an annular mounting frame positioned above the disassembly station and covering the entire disassembly station area. Multiple cameras are mounted on the annular mounting frame. A radial connecting rod is provided inside the annular mounting frame, and a depth camera, an infrared thermal imager, and a smoke sensor are mounted on the radial connecting rod. Operator behavior status data acquisition device, including: Motion capture devices worn on the operator's body are used to acquire the operator's movement trajectory and posture changes during the disassembly process; An eye-tracking device worn on the operator's head is used to acquire the operator's visual attention area and gaze trajectory; An electromyography (EMG) acquisition device is installed on the operator's upper limb to collect data on muscle activity intensity and fatigue status. A physiological monitoring device worn on the operator's wrist to collect information on heart rate and other physiological states.
9. The data acquisition device as described in claim 8, characterized in that, It also includes a tool execution status data acquisition device, including: A torque sensor integrated inside an electric screwdriver and torque wrench is used to acquire applied torque information in real time; Force and displacement sensors integrated into prying tools are used to acquire mechanical changes and displacement information during the prying process. The electric screwdriver, torque wrench, and prying tools are stored inside the disassembly toolbox, which is located on one side of the disassembly station.
Citation Information
Patent Citations
Battery disassembly simulation method and device based on digital twinning and storage medium
CN117725760A
Power battery auxiliary disassembly test device and disassembly information base construction method
CN120065019A
