Method for generating optimal sequence of household appliance disassembly based on cross-modal data fusion and reinforcement learning

Abstract

The invention discloses a household appliance disassembly optimal sequence generation method based on cross-modal data fusion and reinforcement learning, and the method comprises the following steps: collecting different modal data of waste household appliances through different sensors at the front end of a mechanical arm, and carrying out the preprocessing of the different modal data; extracting features from different modal data by using a lightweight convolutional network; carrying out adversarial training on the processed data features by adopting a cross-modal domain adversarial network (CM-DAN), and fusing the data features of different modals; and based on a reinforcement learning algorithm, according to the real-time perception data, dynamically adjusting the disassembling sequence and the execution strength, and generating an optimal disassembling action sequence. The method disclosed by the invention can effectively cope with dynamic changes and complex interferences in the disassembly of the waste household appliances, and improves the disassembly precision and efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of intersection of industrial automation and artificial intelligence, and in particular relates to a method for generating an optimal sequence for disassembly of household appliances based on cross-modal data fusion and reinforcement learning. Background Art

[0002] Dismantling used household appliances is a core link in achieving resource recycling and reducing environmental pollution. Its technical challenges stem from the complex characteristics of used household appliances: they come in a wide variety of types (such as refrigerators, washing machines, and televisions), their internal structures vary significantly (including multi-material components such as metal frames, plastic casings, and circuit boards), and after long-term use, they are prone to corrosion, adhesion, component deformation, and chemical corrosion. For example, long-term vibration in refrigerator compressors causes rust and solidification of bolts and brackets, and washing machine drums experience non-uniform deformation due to metal fatigue. These complex working conditions make traditional manual dismantling inefficient and pose significant safety risks (such as metal debris flying and toxic gas release during manual cutting). Existing automated dismantling systems mostly rely on pre-programmed robotic arms combined with a single sensor (such as an RGB camera or laser rangefinder) to perform fixed-path operations. Although this method can cope with standardized assembly line scenarios, the accuracy of the robotic arm's path planning and execution is significantly reduced when faced with dynamic changes in the dismantling of used household appliances (such as part displacement, sudden changes in material brittleness, and occlusion interference), seriously restricting large-scale recycling capabilities.

[0003] The limitations of traditional automated disassembly technology are primarily reflected in the following aspects: First, the single perception dimension hinders the effective fusion of multimodal data. Monocular vision-based positioning methods are susceptible to lighting variations, occlusion interference, and surface contamination. For example, when disassembling a corroded refrigerator door hinge, an RGB camera cannot accurately identify the screw position due to reflections or oil coverage. LiDAR is also prone to point cloud noise when detecting translucent plastic parts due to the material's light transmittance. Second, existing methods lack dynamic response capabilities. Traditional robotic arm path planning is often based on offline models (such as the torque curve of a non-corroded bolt) and cannot adapt in real time to changes in physical conditions during the disassembly process. For example, when disassembling a corroded screw, a fixed torque threshold can cause the robotic arm to overload or slip. When disassembling a brittle plastic casing, a robotic arm lacking real-time mechanical feedback can easily cause component breakage due to path deviation. More importantly, existing multimodal data fusion methods often employ simple weighting or splicing strategies, failing to effectively model the semantic associations between different modalities. This results in poor cross-modal data fusion and limits decision-making accuracy in complex disassembly tasks.

[0004] In recent years, deep learning technology, especially applications based on convolutional neural networks (CNNs) and reinforcement learning, has promoted the automation level of disassembly tasks. However, actual deployment in industrial scenarios still faces significant bottlenecks. First, the cross-domain generalization ability is poor, and the domain differences between the training data and the real industrial environment (such as illumination changes, mechanical noise, etc.) will cause the performance of the model to drop significantly in actual applications. In addition, existing multimodal data fusion methods fail to effectively consider the domain offset between modalities (for example, the visual domain offset mainly comes from illumination changes, while the offset of mechanical sensors comes from sensor calibration errors). Therefore, it is impossible to achieve good cross-modal data fusion in complex industrial environments. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a method for generating optimal sequences for household appliance disassembly based on cross-modal data fusion and reinforcement learning, so as to effectively cope with the dynamic changes and complex interferences in the disassembly of waste household appliances and improve the disassembly accuracy and efficiency.

[0006] To achieve the above object, the technical solution of the present invention is as follows:

[0007] A method for generating optimal sequences for household appliance disassembly based on cross-modal data fusion and reinforcement learning includes the following steps:

[0008] Step 1: collect different modal data of used household appliances through different sensors at the front end of the robotic arm and pre-process the different modal data;

[0009] Step 2: Use lightweight convolutional networks to extract features from different modal data;

[0010] Step 3: Use a cross-modal adversarial network to conduct adversarial training on the processed data features to fuse the data features of different modalities;

[0011] Step 4: Based on the reinforcement learning algorithm, the disassembly order and execution intensity are dynamically adjusted according to the real-time perception data to generate the optimal disassembly action sequence.

[0012] In the above scheme, in step 1, the RGB image captured by the depth camera, the torque signal measured by the torque meter, and the initial laser point cloud data of the laser point cloud collector are collected; the RGB image is cropped, scaled, and normalized to make it of the same size, and data enhancement is applied to increase data diversity; the torque signal is normalized, and the length of its time series is aligned with the RGB image data through interpolation; the initial laser point cloud data is downsampled and the spatial coordinates are aligned to ensure the consistency of the laser point cloud data in space.

[0013] In a further technical solution, the specific method of step 2 is as follows:

[0014] RGB image feature extraction:

[0015] F RGB =CNN RGB (I RGB )

[0016] Among them, F RGB is the feature of RGB image, I RGB For the input of RGB image, CNN RGB It is a convolutional neural network for RGB images;

[0017] Torque signal feature extraction:

[0018] A one-dimensional convolutional network is used to encode the torque signal of the robotic arm:

[0019] F T =CNN 1D (T)

[0020] Among them, F T is the encoding feature of the moment, CNN 1D is a single-layer convolutional network, T is the time series data of the torque signal;

[0021] Laser point cloud feature extraction:

[0022] Use the point cloud neural network PointNet to extract the structural features of the laser point cloud:

[0023] F P =PointNet(P)

[0024] Among them, F P is the structural feature of the laser point cloud, PointNet is the point cloud neural network, and P is the laser point cloud data.

[0025] In the above scheme, in step 3, in adversarial training, the adversarial loss function L adv for:

[0026]

[0027] Among them, θ F is the parameter of the feature encoder, θ D is the parameter of the domain discriminator, D s and D t Represent the data distribution of the source domain and the target domain respectively, F(x) is the feature extracted by the feature encoder, D is the domain discriminator, To minimize the parameters θ of the generator F F , is the parameter θ of the discriminator D D is optimized to maximize the loss function.

[0028] In the above scheme, in step 3, the final fused feature expression is obtained by weighted summation:

[0029]

[0030] Among them, α m represents the attention weights of different modalities, F m Represents the characteristics of each mode, F fused is the final fusion feature;

[0031] For RGB image modality, a spatial attention-based mechanism is adopted to calculate the weights of different areas in the image using convolution operations to focus on the key areas in the image. The attention weight expression is as follows:

[0032]

[0033] Among them, α RGB is the attention weight of the RGB image modality; represents the feature of the Cth channel of the RGB image modality; in the image, the RGB color information of each pixel is converted into features, C is the feature channel index, indicating the dimension of the feature map; is the decision to disassemble the sequence, called the target feature, which is the feature related to the input data; ‖·‖2 is the L2 norm, which represents the Euclidean distance of the feature vector and is used to measure the difference between two feature vectors;

[0034] For the torque signal modality, channel-based attention is used to enhance the signal related to physical depth; the channel attention weight formula of the torque signal is:

[0035]

[0036] Among them, α T is the attention weight of the torque signal; is the characteristic of the Cth channel representing the torque signal;

[0037] For the laser point cloud data modality, a combination of spatial attention and temporal attention is used to highlight the spatial distribution characteristics of depth information; the attention weight formula of the laser point cloud is:

[0038]

[0039] Among them, α P is the attention weight of the laser point cloud image; is the feature of the Cth channel of the laser point cloud.

[0040] In the above scheme, in step 4, the system will first initialize according to the information flow intensity of each sensor modality, and dynamically adjust the weight of each modality according to the requirements of the current task and environmental changes; the IMAF mechanism evaluates the contribution of each sensor modality through continuous feedback information; when the system obtains new task requirements or environmental data, the IMAF mechanism adjusts the weight of each modality based on this information, enhancing the focus on key modalities and suppressing dependence on unimportant modalities.

[0041] In the above scheme, in step 4, in the reinforcement learning algorithm, the system uses Q-value to evaluate the reward of each action; the Q-value calculation is based on the current state and optional actions, indicating the expected benefit of selecting a certain action in a specific state; at each decision moment, the system evaluates the Q-values ​​of different actions and selects the action with the largest Q-value to execute; the IMAF mechanism intervenes in this process, and it will adjust the calculation of the Q-value based on the current modal information. The IMAF mechanism dynamically adjusts the data flow weights of the modalities so that the contribution of each modality in the current task is fully considered when calculating the Q-value. The system will use IMAF feedback information to update the influence of each modality in the Q-value calculation to ensure the optimal action selection.

[0042] In a further technical solution, in step 4, the formula for the relative information flow strength between modalities is:

[0043]

[0044] in, Indicates that from mode m i To mode m j The information flow intensity, and They are mode m i and m j The feature representation of , C is the feature channel index, which represents the dimension of the feature map.

[0045] In a further technical solution, the state representation and Q-value calculation formula of reinforcement learning are:

[0046] Q(s t ,a t )=R(s t ,a t )+γmax a′ Q(s t+1 ,a t )

[0047] Among them, s t is the current state, a t is the current disassembly action, R(s t ,a t ) is the immediate reward, γ is the discount factor Q(st ,a t ) is based on the state s t and action a t The calculated Q value represents the expected long-term reward of the action in the current state, Q(s t+1 ,a t ) is in the new state s t+1 Next select action a t Q value.

[0048] In a further technical solution, the system outputs the action with the highest Q value as the disassembly strategy at the current moment:

[0049]

[0050] Among them, a* is the disassembly strategy output when the Q value is the highest.

[0051] Through the above technical solution, the present invention provides a method for generating an optimal sequence for household appliance disassembly based on cross-modal data fusion and reinforcement learning, which has the following beneficial effects:

[0052] 1. Multimodal adversarial fusion and dynamic attention enhancement: Traditional automated disassembly systems mostly rely on a single modality (such as vision or force) for environmental perception and target recognition. This makes the system perform poorly when dealing with complex interferences (such as rust, occlusion, reflections, stains, and material deformation) in the disassembly of scrap home appliances. The perception capabilities of a single modality are easily limited, resulting in reduced disassembly accuracy and an increased risk of misoperation. For example, the visual modality may not be able to accurately locate or identify the target under the influence of oil, rust, or occlusion. When relying on force signals, the diversity of home appliance materials and differences in mechanical properties often lead to misjudgment, thus affecting the reliability of the system.

[0053] This paper proposes a multimodal adversarial fusion method based on different attention mechanisms. It employs a Cross-Modal Domain Adversarial Network (CM-DAN) to fuse multimodal information, including vision, force perception, and acoustics, and achieves robust alignment of modal features through domain adversarial learning. Furthermore, to more effectively utilize the features of different modalities, the paper proposes multiple attention mechanisms for each modality. Specifically, for each modality (such as vision, force perception, and acoustics), an attention mechanism best suited to its characteristics is employed to enhance information extraction and focusing. For example, for the visual modality, a spatial attention-based mechanism is used to focus on key areas in the image; for the force modality, a channel-based attention mechanism is used to better capture signals related to physical contact. For the acoustic modality, a temporal attention mechanism is used to highlight sound features within specific time periods relevant to the disassembly process. Through these different attention mechanisms, the system can dynamically adjust the contribution of each modality in multimodal fusion based on its characteristics, effectively improving target localization accuracy and material recognition reliability. Especially under complex environmental interference, it can maximize the advantages of each modal information and significantly improve the disassembly success rate and system stability.

[0054] 2. Reinforcement Learning-Driven Optimal Disassembly Sequence Generation: During the disassembly of used appliances, the materials, degree of aging, and physical conditions of different components vary significantly. This requires the system to dynamically adjust the disassembly sequence and execution intensity based on real-time sensor data. This requires the system to rapidly respond to environmental changes, optimize disassembly strategies, and avoid misoperations. Traditional automated disassembly methods rely on offline rules or fixed path planning, and are unable to adapt to dynamically changing physical conditions, such as component breakage and sudden changes in material brittleness.

[0055] This paper proposes a method for generating disassembly sequences based on reinforcement learning. This method uses a reinforcement learning algorithm to train the system, automatically generating the optimal disassembly sequence and force based on different disassembly tasks (such as disassembling refrigerators and washing machines). The system dynamically adjusts path planning and execution force based on real-time sensory data (such as component deformation and material aging). For example, when disassembling a rusted screw, the reinforcement learning algorithm can adjust the torque based on real-time resistance changes to avoid overload or thread slippage, ensuring the accuracy and safety of the disassembly process.

[0056] 3. During the data preprocessing phase, the present invention first normalizes the RGB image, scaling pixel values ​​to the [0, 1] range. Then, random flipping, rotation, and cropping are used to enhance data diversity and improve model robustness. The laser point cloud data is normalized and downsampled to ensure consistency across different modalities in spatial and channel dimensions, facilitating subsequent feature extraction and fusion.

[0057] During the feature extraction phase, a lightweight convolutional network is used to process data from different modalities separately and extract the key features of each modality. Subsequently, a modality alignment algorithm is used to perform spatial and semantic alignment to ensure that the features of each modality are dimensionally consistent. The fusion process generates the final fused features through weighted fusion and an attention mechanism. These features simultaneously retain rich semantic information and spatial geometric characteristics. Through this fusion method, the system can accurately extract the most recognizable features from data from different modalities, significantly improving the accuracy of target detection. The system combines multimodal features from RGB images, depth images, torque signals, and laser point clouds, and enhances the system's adaptability to environmental changes by dynamically adjusting the attention weights of each modality. Reinforcement learning combined with the IMAF mechanism can automatically adjust strategies according to real-time environmental conditions in different disassembly tasks, thereby ensuring the flexibility and efficiency of the disassembly process.

[0058] In summary, the cross-modal data fusion and reinforcement learning-driven home appliance disassembly method proposed in the present invention, by effectively fusing multimodal information and optimizing disassembly decisions, not only improves the system's adaptability and real-time response capabilities in complex environments, but also solves key problems existing in traditional technologies such as low multimodal fusion efficiency, cross-modal data imbalance, and real-time reasoning difficulties. It can achieve efficient and accurate automatic disassembly of used home appliances in complex industrial environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for describing the embodiments or the prior art.

[0060] Figure 1 This is a flow chart of a method for generating an optimal sequence for disassembly of household appliances based on cross-modal data fusion and reinforcement learning, disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0062] The present invention provides a method for generating optimal sequence for household appliance disassembly based on cross-modal data fusion and reinforcement learning. Figure 1 As shown, the following steps are included:

[0063] Step 1: Collect different modal data of used household appliances through different sensors at the front end of the robotic arm and pre-process the different modal data.

[0064] Specifically, it collects RGB images taken by the depth camera, torque signals measured by the torque measuring instrument, and initial laser point cloud data from the laser point cloud collector.

[0065] RGB images provide visual information about the device's color, texture, and appearance; depth images provide information about the device's spatial geometry and 3D position; torque signals record the forces acting on the robotic arm during operation; and laser point clouds provide precise 3D structural information about the device. Data collection should cover a variety of scenarios, including varying lighting conditions, device placement, and device type. Multimodal sensing equipment should be used to simultaneously acquire data from all four modalities, with each data set provided with the target device's category label and bounding box location.

[0066] The RGB images were cropped, scaled, and normalized to make them of the same size, and data augmentation (random flipping, rotation, cropping, and brightness adjustment) was applied to increase data diversity. The torque signal was normalized, and the length of its time series was aligned with the RGB image data through interpolation. The initial laser point cloud data was downsampled and spatial coordinates were aligned to ensure the consistency of the laser point cloud data in space.

[0067] Finally, based on the calibration information provided by the acquisition device, the three modal data are precisely aligned, and the target's category label and bounding box position are formatted into a unified data format.

[0068] Step 2: Use a lightweight convolutional network (MobileNet) to extract features from different modal data.

[0069] RGB image feature extraction:

[0070] F RGB =CNN RGB (I RGB )

[0071] Among them, F RGB is the feature of RGB image, I RGB For the input of RGB image, CNN RGB It is a convolutional neural network for RGB images;

[0072] Torque signal feature extraction:

[0073] A one-dimensional convolutional network is used to encode the torque signal of the robotic arm:

[0074] F T =CNN 1D (T)

[0075] Among them, F T is the encoding feature of the moment, CNN 1D is a single-layer convolutional network, T is the time series data of the torque signal;

[0076] Laser point cloud feature extraction:

[0077] Use the point cloud neural network PointNet to extract the structural features of the laser point cloud:

[0078] F P =PointNet(P)

[0079] Among them, F P is the structural feature of the laser point cloud, PointNet is the point cloud neural network, and P is the laser point cloud data.

[0080] In step 3, a cross-modal domain adversarial network (CM-DAN) is used to perform adversarial training on the processed data features to fuse the data features of different modalities.

[0081] During adversarial training, the generator generates more “realistic” features, while the discriminator becomes better at identifying fake data. This adversarial process forces the generator to optimize its output, gradually aligning features across modalities and eliminating differences between devices.

[0082] Through this adversarial training mechanism, the features learned by the feature encoder are able to have a similar distribution across different modalities, thus addressing the modality bias caused by sensor heterogeneity. While the domain discriminator attempts to distinguish the modality of the input features, the feature encoder attempts to generate features that confuse the domain discriminator, preventing it from accurately distinguishing between modalities. This adversarial training process enables the feature encoder to learn more discriminative and robust features.

[0083] In the previous step, different attention mechanisms were used for data from different modalities, allowing the system to dynamically adjust the contribution of each modality in feature fusion. The attention mechanisms of different modalities are optimized based on their characteristics and performance in the current environment, ensuring that the most important information is highlighted during the fusion process. During the training of the adversarial network, these features enhanced by different attention mechanisms (enhanced features are derived from the adversarially optimized features of the generator, whose distribution in the feature space tends to be consistent, resulting in better cross-modal portability) are input into the cross-modal adversarial network. Through adversarial training, the parameters of the generator and discriminator are optimized, reducing the deviation in the feature distribution of different modalities, thereby improving the accuracy and stability of cross-modal data fusion.

[0084] In adversarial training, the adversarial loss function L adv for:

[0085]

[0086] Among them, θ F is the parameter of the feature encoder, θ D is the parameter of the domain discriminator, Ds and D t Represent the data distribution of the source domain and the target domain respectively, F(x) is the feature extracted by the feature encoder, D is the domain discriminator, To minimize the parameters θ of the generator F F , is the parameter θ of the discriminator D D is optimized to maximize the loss function.

[0087] The final fused feature expression is obtained by weighted summation:

[0088]

[0089] Among them, α m represents the attention weights of different modalities, F m Represents the characteristics of each mode, F fused is the final fused feature. Through this multi-modal attention mechanism, the disassembly sequence decision model can automatically adjust the contribution of each modality based on the current environmental state (such as noise intensity and occlusion level). For example, in good lighting conditions with no occlusion, the RGB image may be given a higher weight; however, in the presence of occlusion or insufficient lighting, the depth image or laser point cloud may be given a higher weight. This dynamic adjustment mechanism ensures the stability and accuracy of the model in different scenarios.

[0090] For RGB image modality, a spatial attention-based mechanism is adopted to calculate the weights of different areas in the image using convolution operations to focus on the key areas in the image. The attention weight expression is as follows:

[0091]

[0092] Among them, α RGB is the attention weight of the RGB image modality; represents the feature of the Cth channel of the RGB image modality; in the image, the RGB color information of each pixel is converted into features, C is the feature channel index, indicating the dimension of the feature map; is the decision to disassemble the sequence, called the target feature, which is the feature related to the input data; ‖·‖2 is the L2 norm, which represents the Euclidean distance of the feature vector and is used to measure the difference between two feature vectors;

[0093] For the torque signal modality, channel-based attention is used to enhance the signal related to physical depth; the channel attention weight formula of the torque signal is:

[0094]

[0095] Among them, α Tis the attention weight of the torque signal; is the characteristic of the Cth channel representing the torque signal;

[0096] For the laser point cloud data modality, a combination of spatial attention and temporal attention is used to highlight the spatial distribution characteristics of depth information; the attention weight formula of the laser point cloud is:

[0097]

[0098] Among them, α P is the attention weight of the laser point cloud image; is the feature of the Cth channel of the laser point cloud.

[0099] Step 4: Based on the reinforcement learning algorithm, the disassembly order and execution intensity are dynamically adjusted according to the real-time perception data to generate the optimal disassembly action sequence.

[0100] The dynamically fused features are fed into the reinforcement learning strategy network, which dynamically adjusts the weights of different modalities in different environments based on the importance of the modality or step. Through interactive learning with the environment using a weighted loss function, the disassembly strategy is optimized, and the optimal disassembly action sequence is output. Reinforcement learning can adjust the action sequence to suit different disassembly tasks based on real-time environmental feedback.

[0101] Specifically, in step 3, the data from different modalities has been fused. Next, to handle the interaction and information flow between multimodal features, this paper introduces the Inter-Modal Information Flow Adaptive Attention Mechanism (IMAF). The core goal of this mechanism is to dynamically adjust the contribution of different modalities in the decision-making process by calculating the relative information flow strength between the modalities.

[0102] IMAF Mechanism

[0103] Inter-modal information flow: We calculate the strength of the information flow between each pair of modalities to determine which modality has the greatest influence on the other. By adjusting the strength of these information flows, we can flexibly increase or decrease the influence of certain modalities, ensuring that the most important modalities dominate decision-making.

[0104] Dynamic weight adjustment: The IMAF mechanism automatically adjusts the weights of each modality based on environmental changes and task requirements to avoid redundant information or unnecessary noise interference between modalities. The system can adaptively optimize the contribution of each modality and improve the effect of multimodal fusion.

[0105] The system is first initialized based on the information flow intensity of each sensor modality, and dynamically adjusts the weight of each modality according to the current task requirements and environmental changes; the IMAF mechanism evaluates the contribution of each sensor modality through continuous feedback information; when the system obtains new task requirements or environmental data, the IMAF mechanism adjusts the weight of each modality based on this information, enhancing the focus on key modalities and suppressing reliance on unimportant modalities.

[0106] In the reinforcement learning algorithm, the system uses Q-value to evaluate the reward of each action; the Q-value calculation is based on the current state and optional actions, indicating the expected benefit of choosing a certain action in a specific state; at each decision moment, the system evaluates the Q-values ​​of different actions and selects the action with the largest Q-value to execute; the IMAF mechanism intervenes in this process, and it will adjust the calculation of the Q-value based on the current modal information. The IMAF mechanism dynamically adjusts the data flow weights of the modalities so that the contribution of each modality in the current task is fully considered when calculating the Q-value. The system will use IMAF feedback information to update the influence of each modality in the Q-value calculation to ensure the optimal action selection.

[0107] The formula for the relative information flow intensity between modalities is:

[0108]

[0109] in, Indicates that from mode m i To mode m j The information flow intensity, and They are mode m i and m j is the feature representation, and C is the feature channel index, representing the dimension of the feature map. This information flow adjustment mechanism ensures that important modal information is highlighted and dynamically enhances or weakens the contribution of specific modalities through adjustment. This mechanism ensures that the information flow between modalities influences every decision-making process in the reinforcement learning model, further improving decision accuracy and system adaptability.

[0110] Modes with stronger information flow are given higher weight in the Q-value calculation, thus influencing action selection during reinforcement learning. In this way, the strength of information flow directly influences the Q-value of each state-action pair. The system uses information flow strength to optimize Q-values ​​based on environmental changes and task requirements, ultimately leading to more accurate decisions.

[0111] In reinforcement learning, the IMAF mechanism provides the ability to dynamically adjust modal weights, allowing the system to flexibly adjust the contribution of each modality based on the current environment and task requirements. By combining the IMAF mechanism, the system can dynamically adjust the weight of each modality based on the information flow intensity of each modality, thereby affecting the state representation and Q-value calculation in reinforcement learning. After combining the IMAF mechanism, the state representation and Q-value calculation formula of reinforcement learning are:

[0112] Q(s t ,a t )=R(s t ,a t )+γmax a′ Q(s t+1 ,a t )

[0113] Among them, s t is the current state, a t is the current disassembly action, R(s t ,a t ) is the immediate reward, γ is the discount factor Q(s t ,a t ) is based on the state s t and action a t The calculated Q value represents the expected long-term reward of the action in the current state, Q(s t+1 ,a t ) is in the new state s t+1 Next select action a t The IMAF mechanism adjusts the weight of each modality to ensure that the reinforcement learning model can make optimal decisions based on real-time environmental feedback. Through this combination, reinforcement learning no longer relies solely on information from a single modality, but can extract useful information from multiple modalities to comprehensively evaluate and select the optimal disassembly action.

[0114] Based on the generated Q value, the system selects the action that maximizes the Q value as the optimal disassembly action. Specifically, the system outputs the action with the highest Q value as the disassembly strategy at the current moment:

[0115]

[0116] Among them, a* is the disassembly strategy output when the Q value is the highest. This step ensures that the system selects the optimal disassembly action at each time step, making the disassembly process more efficient and accurate. During the actual disassembly process, the system will adjust the disassembly action in real time based on feedback signals (such as force perception, visual information, etc.). For example, during the disassembly process, if the shape or position of a component changes, the system will recalculate the Q value through real-time perception data and dynamically adjust the disassembly action. Through this dynamic adjustment, the system can cope with uncertainties in complex environments and maintain high disassembly accuracy and efficiency.

[0117] Case 1: Under normal circumstances

[0118] The clear imagery provided by the visual sensor allows for precise location of the device's disassembly points (e.g., screw locations). The force sensor accurately senses the hardness and material of the disassembled components and adjusts the disassembly force based on real-time feedback. The 3D point cloud data provided by the LiDAR helps determine the spatial position and depth of objects, complementing the limitations of vision and force sensing.

[0119] In this environment, the information flow between the various sensors (vision, force sensing, and lidar) is high (meaning each modality contributes significantly). The vision modality provides accurate component location, the force sensing modality provides precise physical contact feedback, and the lidar provides additional spatial information. The system is able to fully utilize these information flows to ensure efficient disassembly operations.

[0120] Case 2: Low light environment

[0121] The effectiveness of visual sensors declines. Due to low light or reflection issues, images become blurred and cannot accurately identify the position of parts. Force sensors can still provide accurate physical contact feedback. LiDAR continues to provide spatial depth data.

[0122] At this point, the visual sensor's information flow intensity decreases because the image is unclear and cannot provide valuable position information. The system automatically reduces the weight of the visual modality (that is, reduces its influence on decision-making) and increases the weight of the force sensor and lidar modalities. The lidar and force sensors provide a higher intensity of information flow, so the system relies more on them to continue the disassembly task.

[0123] Case 3: Occlusion or complex interference

[0124] Both visual sensors and lidar are affected by obstructions (such as piled objects or complex equipment structures), making it difficult to obtain clear images and point cloud data. Force sensors can still sense physical contact and pressure changes, especially when disassembling harder components.

[0125] In this case, the information flow intensity of the LiDAR and vision sensors decreases, while the information flow intensity of the force sensor remains high. Based on the IMAF mechanism, the system automatically increases the weight of the force modality to ensure that the system relies on force data to complete the disassembly.

[0126] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating optimal sequence for household appliance disassembly based on cross-modal data fusion and reinforcement learning, characterized by: The steps include: Step 1: collect different modal data of used household appliances through different sensors at the front end of the robotic arm and pre-process the different modal data; Step 2: Use lightweight convolutional networks to extract features from different modal data; Step 3: Use a cross-modal adversarial network to conduct adversarial training on the processed data features to fuse the data features of different modalities; Step 4: Based on the reinforcement learning algorithm, the disassembly order and execution intensity are dynamically adjusted according to the real-time perception data to generate the optimal disassembly action sequence.

2. The method for generating an optimal sequence for disassembly of household appliances based on cross-modal data fusion and reinforcement learning according to claim 1 is characterized in that: In step 1, the RGB images captured by the depth camera, the torque signals measured by the torque meter, and the initial laser point cloud data from the laser point cloud collector are collected. The RGB images are cropped, scaled, and normalized to make them of the same size, and data enhancement is applied to increase data diversity. The torque signals are normalized, and the length of their time series is aligned with the RGB image data through interpolation. The initial laser point cloud data is downsampled and the spatial coordinates are aligned to ensure the consistency of the laser point cloud data in space.

3. The method for generating an optimal sequence for disassembly of household appliances based on cross-modal data fusion and reinforcement learning according to claim 2, characterized in that: The specific method in step 2 is as follows; RGB image feature extraction: F RGB =CNN RGB (I RGB ) Among them, F RGB is the feature of RGB image, I RGB For the input of RGB image, CNN RGB It is a convolutional neural network for RGB images; Torque signal feature extraction: A one-dimensional convolutional network is used to encode the torque signal of the robotic arm: F T =CNN 1D (T) Among them, F T is the encoding feature of the moment, CNN 1D is a single-layer convolutional network, T is the time series data of the torque signal; Laser point cloud feature extraction: Use the point cloud neural network PointNet to extract the structural features of the laser point cloud: F P =PointNet(P) Among them, F P is the structural feature of the laser point cloud, PointNet is the point cloud neural network, and P is the laser point cloud data.

4. The method for generating an optimal sequence for disassembly of household appliances based on cross-modal data fusion and reinforcement learning according to claim 1, characterized in that: In step 3, in adversarial training, the adversarial loss function L adv for: Among them, θ F is the parameter of the feature encoder, θ D is the parameter of the domain discriminator, D s and D t Represent the data distribution of the source domain and the target domain respectively, F(x) is the feature extracted by the feature encoder, D is the domain discriminator, To minimize the parameters θ of the generator F F , is the parameter θ of the discriminator D D is optimized to maximize the loss function.

5. The method for generating an optimal sequence for disassembly of household appliances based on cross-modal data fusion and reinforcement learning according to claim 3 is characterized in that: In step 3, the final fused feature expression is obtained by weighted summation: Among them, α m represents the attention weights of different modalities, F m Represents the characteristics of each mode, F fused is the final fusion feature; For RGB image modality, a spatial attention-based mechanism is adopted to calculate the weights of different areas in the image using convolution operations to focus on the key areas in the image. The attention weight expression is as follows: Among them, α RGB is the attention weight of the RGB image modality; represents the feature of the Cth channel of the RGB image modality; in the image, the RGB color information of each pixel is converted into features, C is the feature channel index, indicating the dimension of the feature map; is the decision to disassemble the sequence, called the target feature, which is the feature related to the input data; ‖·‖2 is the L2 norm, which represents the Euclidean distance of the feature vector and is used to measure the difference between two feature vectors; For the torque signal modality, channel-based attention is used to enhance the signal related to physical depth; the channel attention weight formula of the torque signal is: Among them, α T is the attention weight of the torque signal; is the characteristic of the Cth channel representing the torque signal; For the laser point cloud data modality, a combination of spatial attention and temporal attention is used to highlight the spatial distribution characteristics of depth information; the attention weight formula of the laser point cloud is: Among them, α P is the attention weight of the laser point cloud image; is the feature of the Cth channel of the laser point cloud.

6. The method for generating an optimal sequence for disassembly of household appliances based on cross-modal data fusion and reinforcement learning according to claim 1, characterized in that: In step 4, the system is first initialized based on the information flow intensity of each sensor modality, and dynamically adjusts the weight of each modality according to the current task requirements and environmental changes; the IMAF mechanism evaluates the contribution of each sensor modality through continuous feedback information; when the system obtains new task requirements or environmental data, the IMAF mechanism adjusts the weight of each modality based on this information, enhancing the focus on key modalities and suppressing reliance on unimportant modalities.

7. The method for generating an optimal sequence for disassembly of household appliances based on cross-modal data fusion and reinforcement learning according to claim 1, characterized in that: In step 4, in the reinforcement learning algorithm, the system uses Q-value to evaluate the reward of each action; the Q-value calculation is based on the current state and optional actions, indicating the expected benefit of selecting an action in a specific state; at each decision moment, the system evaluates the Q-values ​​of different actions and selects the action with the largest Q-value to execute; the IMAF mechanism intervenes in this process, and it adjusts the calculation of the Q-value based on the current modal information. The IMAF mechanism dynamically adjusts the data flow weights of the modalities so that the contribution of each modality in the current task is fully considered when calculating the Q-value. The system uses IMAF feedback information to update the influence of each modality in the Q-value calculation to ensure the optimal action selection.

8. The method for generating an optimal sequence for disassembly of household appliances based on cross-modal data fusion and reinforcement learning according to claim 6, characterized in that: In step 4, the formula for the relative information flow strength between modalities is: in, Indicates that from mode m i To mode m j The information flow intensity, and They are mode m i and m j The feature representation of , C is the feature channel index, which represents the dimension of the feature map.

9. The method for generating an optimal sequence for disassembly of household appliances based on cross-modal data fusion and reinforcement learning according to claim 7, characterized in that: The state representation and Q-value calculation formula of reinforcement learning are: Q(s t ,a t )=R(s t ,a t )+γmax a′ Q(s t+1 ,a t ) Among them, s t is the current state, a t is the current disassembly action, R(s t ,a t ) is the immediate reward, γ is the discount factor Q(s t ,a t ) is based on the state s t and action a t The calculated Q value represents the expected long-term reward of the action in the current state, Q(s t+1 ,a t ) is in the new state s t+1 Next select action a t Q value.

10. The method for generating an optimal sequence for disassembling household appliances based on cross-modal data fusion and reinforcement learning according to claim 9, characterized in that: The system outputs the action with the highest Q value as the disassembly strategy at the current moment: Among them, a * This is the disassembly strategy output when the Q value is the highest.

Images