Data acquisition methods, equipment, and media for humanoid robot training fields

Abstract

This application provides a data acquisition method, device, and medium for a humanoid robot training field. The method determines the welding task corresponding to the equipment to be welded, acquires multi-sensor data, performs path planning and trajectory generation based on the multi-sensor data and the welding task, obtains an initial trajectory, injects random perturbations into the humanoid robot during the welding task based on the initial trajectory, adjusts the initial trajectory according to the random perturbations, and continues to execute the welding task based on the adjusted initial trajectory, recording the state data during the welding task execution process. This method can be applied to welding operation scenarios in the industrial manufacturing field and can be used in a humanoid robot training field to achieve trajectory generation based on multi-sensor data, automatically record data during task execution, realize automated data acquisition, improve data acquisition efficiency, and enhance the richness of data by adding perturbations, thus helping to improve the generalization ability of the humanoid robot under different production conditions.

Description

Technical Field

[0001] This application relates to the field of humanoid robot technology, and in particular to a data acquisition method, device and medium for humanoid robot training field. Background Technology

[0002] In the production of modern refrigeration equipment (such as refrigerators and air conditioners), welding is a crucial process for ensuring product quality and performance. For example, the process pipes, suction pipes, and exhaust pipes of a refrigerator compressor need to be welded to other components to ensure refrigerant sealing and proper equipment operation. In related technologies, welding operations can be automated using fixed robotic arms. However, this method offers limited flexibility.

[0003] In order to apply humanoid robots to humanoid robot training fields or industrial welding scenarios, how to efficiently collect rich data for humanoid robot training is an urgent technical problem to be solved. Summary of the Invention

[0004] This application provides a data acquisition method, device, and medium for humanoid robot training fields, to improve the richness and efficiency of data acquisition by humanoid robots in welding scenarios.

[0005] In a first aspect, embodiments of this application provide a data acquisition method for a humanoid robot training field, including:

[0006] Determine the welding task corresponding to the equipment to be welded;

[0007] Acquire multi-sensor data;

[0008] Based on the multi-sensor data and the welding task, path planning and trajectory generation are performed to obtain an initial trajectory;

[0009] During the process of controlling the humanoid robot to perform the welding task according to the initial trajectory, random disturbances are injected, the initial trajectory is adjusted according to the random disturbances, and the welding task is continued based on the adjusted initial trajectory.

[0010] Record the status data during the execution of the welding task.

[0011] In one possible design, the random perturbation includes at least one of the following: joint trajectory perturbation of the humanoid robot, contact force perturbation between the welding torch and the equipment to be welded, visual occlusion perturbation, welding torch posture perturbation, arc discontinuity perturbation, center of gravity shift perturbation of the humanoid robot, parallel multitasking perturbation, welding machine current perturbation, and welding speed perturbation.

[0012] In one possible design, the injection of random perturbations during the process of controlling the humanoid robot to perform the welding task according to the initial trajectory includes at least one of the following:

[0013] Motion control commands are generated based on the initial trajectory. A sinusoidal perturbation signal is superimposed on the motion control commands and injected into the joint trajectory perturbation. The motion control commands are used to control the humanoid robot to hold a welding torch and move it closer to the equipment to be welded.

[0014] After the welding torch comes into contact with the equipment to be welded, the admittance control of the humanoid robot is turned off for a first preset time, and the contact force disturbance is injected.

[0015] During the welding process, the arm of the humanoid robot is controlled to block the field of view of the visual sensor for a second preset time period, and the visual occlusion disturbance is injected.

[0016] The initial trajectory includes a welding path; random deflection angles are superimposed in the normal plane of the welding path to inject welding gun attitude disturbance;

[0017] The arc interruption of a third preset duration is triggered through the welding machine communication interface, injecting the intermittent arc disturbance.

[0018] During the welding process, the humanoid robot is controlled to lean forward at a preset angle, and the center of gravity shift disturbance is injected.

[0019] During the welding process, a voice broadcast task is initiated, injecting parallel multi-task disturbances.

[0020] In one possible design, the welding task corresponding to the equipment to be welded is determined, including:

[0021] Acquire image data of the equipment to be welded;

[0022] The current workpiece to be welded in the equipment to be welded is determined based on the process flow of the equipment to be welded and the image data.

[0023] Based on the current workpiece to be welded, the corresponding process parameters are retrieved to obtain the welding task.

[0024] In one possible design, the step of performing path planning and trajectory generation based on the multi-sensor data and the welding task to obtain an initial trajectory includes:

[0025] The target work location is randomly determined from multiple work locations in the work area of ​​the equipment to be welded;

[0026] Based on the target work location, the multi-sensor data, and the welding task, path planning and trajectory generation are performed to obtain an initial trajectory; the initial trajectory includes the first trajectory data of the humanoid robot moving from the waiting area to the target work location.

[0027] In one possible design, the step of performing path planning and trajectory generation based on the multi-sensor data and the welding task to obtain an initial trajectory includes:

[0028] The location of the starting point in the welding device is determined based on the image data in the multi-sensor data.

[0029] Based on the image data, determine the first coordinate of the starting point location in the first coordinate system of the equipment to be welded;

[0030] Based on the first and second transformation relationships, the second coordinates of the starting point in the robotic arm coordinate system are determined according to the first coordinates; the first transformation relationship is the transformation relationship between the humanoid robot's camera coordinate system and the robotic arm coordinate system; the second transformation relationship is the transformation relationship between the first coordinate system and the camera coordinate system.

[0031] Based on the second coordinates, the multi-sensor data, and the welding task, path planning and trajectory generation are performed to obtain an initial trajectory; the initial trajectory includes the second trajectory data of the humanoid robot moving the welding torch to the starting point position.

[0032] In one possible design, the state data includes recovery data in response to the random disturbance.

[0033] In one possible design, the multi-sensor data includes at least one of the following: image data of the workpiece to be welded, force sensing data of the clamping tool holding the workpiece to be welded, and temperature data of the welding area; based on the multi-sensor data and the welding task, path planning and trajectory generation are performed to obtain an initial trajectory, including:

[0034] The image data is preprocessed to obtain preprocessed image data; the preprocessing includes at least one of the following: Gaussian filtering, contrast enhancement;

[0035] Based on the preprocessed image data, determine the spatial axis position of the workpiece to be welded in the welding equipment;

[0036] The tilt angle of the workpiece to be welded is determined based on the position of the spatial axis.

[0037] Based on the tilt angle, the force sensing data, the temperature data, and the welding task, path planning and trajectory generation are performed to obtain an initial trajectory.

[0038] Secondly, embodiments of this application provide a data acquisition device for a humanoid robot training field, comprising:

[0039] The determination module is used to determine the welding task corresponding to the equipment to be welded;

[0040] The planning module is used to acquire multi-sensor data, perform path planning and trajectory generation based on the multi-sensor data and the welding task, and obtain an initial trajectory.

[0041] The disturbance module is used to inject random disturbances into the process of controlling the humanoid robot to perform the welding task according to the initial trajectory, adjust the initial trajectory according to the random disturbances, and continue to perform the welding task based on the adjusted initial trajectory.

[0042] The data acquisition module is used to record the status data during the execution of the welding task.

[0043] Thirdly, embodiments of this application provide a data acquisition device for a humanoid robot training field, comprising: at least one processor and a memory;

[0044] The memory stores computer-executed instructions;

[0045] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method described in the first aspect above and various possible designs of the first aspect.

[0046] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in the first aspect and various possible designs of the first aspect.

[0047] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect and various possible designs of the first aspect.

[0048] This embodiment provides a data acquisition method, device, and medium for a humanoid robot training field. The method determines the welding task corresponding to the equipment to be welded, acquires multi-sensor data, performs path planning and trajectory generation based on the multi-sensor data and the welding task, obtains an initial trajectory, injects random perturbations into the humanoid robot during the welding task based on the initial trajectory, adjusts the initial trajectory according to the random perturbations, and continues to execute the welding task based on the adjusted initial trajectory, recording the state data during the welding task execution process. The data acquisition method provided in this embodiment achieves automated data acquisition by generating a trajectory based on multi-sensor data and automatically recording data during task execution, improving data acquisition efficiency. Furthermore, by adding perturbations, it enhances the richness of the data, helping to improve the generalization ability of the humanoid robot under different production conditions. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0050] Figure 1 This is a schematic diagram illustrating an application scenario of the data acquisition method for a humanoid robot training field provided in an embodiment of this application.

[0051] Figure 2 A flowchart illustrating a data acquisition method for a humanoid robot training field provided in an embodiment of this application;

[0052] Figure 3 A schematic diagram illustrating the principle of humanoid robot hand-eye calibration in the data acquisition method for a humanoid robot training field provided in this application embodiment;

[0053] Figure 4 This is a schematic diagram of the structure of a data acquisition device for a humanoid robot training field provided in an embodiment of this application;

[0054] Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0055] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] It should be noted that the data acquisition method, equipment and medium for humanoid robot training fields provided in this application can be used in the field of humanoid robot technology, or in any field other than the field of humanoid robot technology. The application fields of the data acquisition method, equipment and medium for humanoid robot training fields provided in this application are not limited.

[0058] In modern refrigeration equipment manufacturing, welding is a crucial process for ensuring product quality and reliability. Taking refrigerators as an example, the process tubes, suction pipes, exhaust pipes, and other extended copper pipes on the compressor must be reliably connected to other components through welding to ensure the refrigerant system is sealed and the equipment operates normally.

[0059] Currently, welding operations on refrigerator production lines mainly rely on manual operation or fixed robotic arms. While manual welding is flexible, it depends on the worker's experience and hand-eye coordination, resulting in long training periods, high labor intensity, and fluctuating welding quality due to worker fatigue and differences in skill level, making it difficult to guarantee consistent product quality. To improve production efficiency and product quality, some factories have introduced automated welding equipment. However, traditional automated welding equipment is mostly composed of fixed robotic arms, which, limited by their structural design, can typically only work along preset welding paths, lacking flexibility and struggling to adapt to welding tasks involving parts of different specifications or with uncertain positions on the production line. Especially in the dynamic operating environment of a production line, the position and posture of the workpiece may shift, and fixed robotic arms lack adaptive adjustment capabilities, making it difficult to cope with dynamically changing production environments. Therefore, the application of humanoid robots in welding is a feasible solution to these problems.

[0060] Currently, there are no mature humanoid robot application solutions for dynamic welding scenarios in production environments. Designing training scenarios suitable for actual production needs to verify the applicability and reliability of humanoid robots is crucial for achieving industrial applications.

[0061] To address the aforementioned technical challenges, the inventors of this application have discovered that humanoid robots used for data acquisition can comprehensively simulate the production process in an industrial workshop, including operational details at each stage, potential unexpected situations, and various real-world environmental scenarios. Data acquisition in welding operations is not simply the repetition of a single action, but encompasses complex factors such as the interaction of various materials and equipment during production, and environmental interference. For example, the position and orientation of the refrigerator compressor pipes on an assembly line may deviate, and the humanoid robot can perceive and adapt to these changes in real time. However, existing data acquisition methods largely rely on technologies such as motion capture teleoperation, master-slave arm teleoperation, and virtual reality (VR) teleoperation. These methods typically require human demonstration or manual control, resulting in high manpower input, low efficiency in the acquisition process, and difficulty in ensuring data accuracy. Especially in delicate operations, it is difficult to capture subtle differences in robot movements, and the generalization ability is limited. Therefore, by constructing dynamic training scenarios that meet the needs of actual production lines and adopting automated data acquisition solutions, robots can repeatedly train their welding skills in a real production environment, improving their ability to handle delicate movements, enhancing their generalization ability, and providing high-quality data support for subsequent model training. This gradually improves the robot's performance and adaptability in complex welding tasks. Based on this, embodiments of this application provide a data acquisition method for a humanoid robot training field.

[0062] Figure 1 This is a schematic diagram illustrating an application scenario for the data acquisition method used in a humanoid robot training field, as provided in an embodiment of this application. Figure 1 As shown, the equipment to be welded (e.g., a refrigerator) is conveyed on a conveyor belt, and multiple humanoid robots are used to move to the work area around the conveyor belt to perform welding tasks on the refrigerator.

[0063] In the specific implementation process, after receiving the start command, the humanoid robot can move from the waiting area to the working area around the conveyor belt. As the conveyor belt transports the refrigerator to be welded, the humanoid robot determines the welding task corresponding to the equipment to be welded, acquires multi-sensor data, and performs path planning and trajectory generation based on the multi-sensor data and the welding task to obtain an initial trajectory. Based on the initial trajectory, random perturbations are injected into the humanoid robot during the welding task execution process, and the initial trajectory is adjusted according to the random perturbations. Based on the adjusted initial trajectory, the welding task continues to be executed, and the state data during the welding task execution process is recorded. The data acquisition method provided in this application embodiment automatically records data during task execution by generating a trajectory based on multi-sensor data, achieving automated data acquisition, improving data acquisition efficiency, and enhancing data richness by adding perturbations, which helps improve the generalization ability of the humanoid robot under different production conditions.

[0064] It should be noted that, Figure 1 The schematic diagram shown is merely an example. The data acquisition method and scenario for humanoid robot training field described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of the system and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0065] The technical solutions of this application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0066] Figure 2 This is a flowchart illustrating a data acquisition method for a humanoid robot training field provided in an embodiment of this application. Figure 2 As shown, the method includes:

[0067] 201. Determine the welding task corresponding to the equipment to be welded.

[0068] The executing entity in this embodiment can be a humanoid robot.

[0069] Specifically, the system may include a task planning module, which can receive task instructions and thus determine the welding task. For example, this module can be used for global planning, module scheduling, and exception management of welding tasks, undertaking the top-level control function of the entire system. After receiving the task instruction, the task planning module will decompose the task according to the welding process requirements, formulate a detailed execution process, and coordinate the collaborative work of various functional modules.

[0070] Taking the task of welding refrigerator compressor pipes as an example, the specific task breakdown can include: first, it can be broken down into three sub-tasks according to the welding process stage, including pre-processing sub-tasks (such as welding torch preheating and welding point positioning marking), core welding sub-tasks (broken down according to pipe type, such as process pipe welding, suction pipe welding, and exhaust pipe welding, and the welding sequence of each pipe should follow the spatial avoidance principle of "short first, long second, bottom first, top first"), and post-processing sub-tasks (such as welding quality inspection and temperature cooling monitoring).

[0071] Each subtask is further broken down into executable operation units. For example, the "process pipe welding" subtask in the core welding subtask is further broken down into four operation units: "welding torch posture adjustment → welding torch flame adjustment → uniform speed welding trajectory execution → arc termination control". Each unit has a clear triggering condition (such as the judgment criterion for the completion of posture adjustment is that the visual sensor detects that the welding torch flame comes into contact with the weld) and an execution time threshold (such as the flame contact time with the weld needs to be completed within 1-3 seconds).

[0072] The task planning module can also be used to monitor the progress of welding tasks, identify abnormal situations and trigger fault handling mechanisms in a timely manner to ensure the stable and efficient operation of the system and realize automated welding task management and control.

[0073] In some embodiments, determining the welding task corresponding to the equipment to be welded may include: acquiring image data of the equipment to be welded; determining the current workpiece to be welded in the equipment based on the process flow and image data of the equipment to be welded; and retrieving the corresponding process parameters based on the current workpiece to be welded to obtain the welding task. In this embodiment, through intelligent matching of image data and process flow, the workpiece to be welded can be automatically and accurately identified and located, and the corresponding process parameters can be quickly retrieved, significantly improving the automation level and response efficiency of welding task planning, and reducing manual intervention and preparation time.

[0074] Specifically, multi-sensor data can include image data, which can acquire image data of the equipment to be welded. Then, based on the process flow of the equipment to be welded and the image data, the current task progress of the equipment to be welded can be determined. For example, if the process flow includes three workpieces A, B and C to be welded in sequence, and multiple humanoid robots are working together, assuming that the previous robot has completed the welding task of workpiece A, then the current workpiece to be welded can be determined as B, and the current welding task is the welding task for B.

[0075] 202. Acquire multi-sensor data, and based on the multi-sensor data and welding task, perform path planning and trajectory generation to obtain the initial trajectory.

[0076] Specifically, the system may also include a sensing system module and a motion control module.

[0077] For example, the sensing system module integrates multimodal sensors, which may include high-resolution vision sensors, force control sensors, and temperature sensors, to comprehensively sense dynamic changes in the welding environment.

[0078] Given that compressor copper pipes are mostly connected by circular pipes, the vision sensor in the perception system focuses on optimizing the processing of weld seams on these circular pipes. In the oxyacetylene gas welding scenario, the vision sensor first preprocesses the image using Gaussian filtering to remove noise from potential interference such as smoke and sparks in the welding environment. Then, the CLAHE algorithm is used to enhance the contrast between the weld seam area and the surrounding pipe surface, making the weld seam features easier to identify. For circular pipes, images are acquired from multiple perspectives, and the Hough circle transform algorithm is used to extract the circular contours from each perspective to estimate the spatial axis position. Subsequently, point cloud data is combined, and point cloud fitting technology is used to calculate the tilt angle of the pipe relative to the horizontal plane, providing a reliable spatial attitude basis for the accurate planning of the subsequent welding trajectory.

[0079] In weld treatment, two common types of welds for connecting circular pipes are butt welds and lap welds. For butt welds, subpixel-level edge detection technology is used to accurately obtain the depth and width parameters of the joint by finely identifying the pixels at the weld edge. At the same time, the SIFT algorithm is used to extract key feature points at the joint, and based on this, a straight welding trajectory that fits the joint direction is planned. For lap welds, the gray value difference method is used to compare the image gray value difference between the overlapping area and the non-overlapping area to clearly identify the boundary of the overlapping area. The feature information of the overlapping boundary is extracted by the contour matching algorithm, and then a welding trajectory with a certain swing amplitude is planned according to the boundary range. The swing amplitude is dynamically adjusted according to the actual size of the lap width to ensure that the trajectory can completely cover the lap area and adapt to the weld shape of connecting circular pipes.

[0080] Based on the parallax principle of binocular vision, the vision system converts two-dimensional image information into three-dimensional spatial coordinates, reconstructs the three-dimensional structure of the weld area, and finally maps the three-dimensional data from the camera coordinate system to the robot end effector coordinate system through a coordinate transformation algorithm, outputting the precise position and orientation information of the weld. Considering the easy oxidation of copper pipe surface, the vision system also uses a color recognition algorithm to detect the oxide layer on the pipe surface (the oxide layer is usually green or dark brown), and prioritizes areas with no oxidation or light oxidation as the welding starting point to ensure the bonding strength of the weld.

[0081] Force control sensors are applied to the physical interactions before and after welding. During the copper pipe adjustment stage before welding, some copper pipes may shift position or tilt due to vibrations on the production line, causing the weld seam to be obscured or in a blind spot. The robot needs to use an auxiliary gripper to gently clamp the copper pipe to adjust the weld seam's posture, ensuring it is fully exposed within the detection range of the vision sensor, preventing welding quality defects caused by copper pipe obstruction. Given the relatively soft nature of copper pipes, excessive clamping force can cause pipe deformation, affecting welding accuracy. Therefore, the force control sensor monitors the contact force between the gripper and the copper pipe in real time, controlling it within a certain range. Through closed-loop feedback, it dynamically adjusts the gripping force and angle of the gripper, ensuring smooth adjustment of the copper pipe's posture while avoiding deformation or wear. If the robot arm encounters slight obstructions from other surrounding pipes or equipment during the adjustment process, the force control sensor immediately feeds back to the control system, driving the robotic arm to fine-tune its movement path, ensuring the copper pipe's posture adjustment is completed within a safe range. After welding is completed, if it is necessary to clean the spatter around the weld, the force control sensor will control the contact force between the cleaning tool (such as a soft brush) and the surface of the copper tube within a certain range to avoid scratching the oxide layer of the copper tube or leaving indentations.

[0082] Temperature sensors monitor the temperature of the welding area in real time. The raw temperature data is processed by moving average filtering and compared with the optimal temperature range for oxyacetylene welding of copper pipes (usually 800-1000℃). When the temperature exceeds the upper limit, the robot will reduce the flame power (reduce the gas flow rate) to lower the welding temperature and prevent the copper pipe from developing problems such as coarse grains and increased brittleness due to overheating. When the temperature is below the lower limit, the flame power will be increased (increase the gas flow rate) to ensure that the copper pipe reaches a sufficient melting temperature and guarantee the strength of the weld joint.

[0083] The motion control algorithm module provides functions such as path planning, trajectory generation, compliant control, and adaptive admittance control. The path planning module, combined with environmental information provided by the perception system, generates a welding path with the shortest time and lowest energy consumption. The trajectory generation module, based on spline interpolation algorithms, generates smooth, high-precision motion trajectories for the robot, avoiding sudden speed changes and unnecessary acceleration during motion. Compliant control and admittance control handle external disturbances and non-rigid contact issues during welding, enabling the robot to adjust the posture and force of its welding tools in real time, thereby maintaining welding stability and accuracy in complex or dynamic environments.

[0084] In some embodiments, the process of fusing multi-sensor data may include: firstly, acquiring RGB-D point cloud data (containing the geometric features of the pipe surface) through a vision sensor in the multi-sensor system; acquiring the welding torch contact force / torque vector ($F_x, F_y, F_z, T_x, T_y, T_z$) through a six-dimensional force sensor in the multi-sensor system; and acquiring the welding current / voltage waveform through an arc monitoring module in the multi-sensor system. Then, the point cloud data can be input into a weld feature extraction algorithm to obtain the actual pipe pose deviation (Δx, Δy, Δz, Δθ); the contact force vector can be input into a contact state classifier to obtain a contact type identifier (normal contact / collision / adhesion); and a fast Fourier transform (FFT) can be performed on the current waveform to obtain an arc stability index (0-1.0).

[0085] This allows for thermal deformation compensation fusion. Specifically, it involves receiving the actual positional deviation of the pipeline and combining it with a material thermal expansion coefficient database (copper pipe: 1.7×10⁻). 5 The thermal deformation compensation is calculated by overlaying infrared thermometer data (real-time pipeline temperature T) onto the molten pool ( / ℃). The positional deviation, thermal deformation compensation, and welding process constraints are then integrated and input into a dynamic RRT* algorithm to obtain a path point sequence. This sequence is then optimized, and the original trajectory is generated through fifth-order pH curve spline interpolation. Real-time visual monitoring data of the molten pool (melt width W) is received, and the speed adjustment coefficient is calculated to obtain an initial trajectory with speed constraints.

[0086] During the welding process, the arc energy integral value is received and input into the LSTM stage classifier. The welding stage identifier (arc initiation / steady state / arc termination) is used to call the basic parameter library (arc initiation stage: M=0.3, B=200, K=2500) and superimpose the contact state classification results to trigger parameter adjustment rules (e.g., if a collision is detected, the critical damping mode is activated; if adhesion is detected, high-frequency micro-vibration is superimposed).

[0087] The admittance controller outputs joint torque commands to drive the robot to perform welding actions. A laser tracker collects the actual trajectory in real time, calculates the trajectory tracking error, and inputs the tracking error, weld pool quality score, and energy consumption data into a reinforcement learning agent to update the parameter rule base for path planning and admittance control, completing a self-evolutionary closed loop. Multiple sensor data and process data such as joint torque are also recorded.

[0088] In this embodiment, by deeply integrating multi-source sensor data such as vision, force, arc, and temperature, and introducing thermal deformation compensation and adaptive parameter adjustment mechanisms, the accuracy and robustness of the welding trajectory under complex thermal disturbances can be significantly improved. At the same time, through reinforcement learning, the parameter library is optimized online, ultimately achieving a synergistic improvement in welding quality, efficiency, and system autonomy, and enhancing the quality and richness of data acquisition.

[0089] In some embodiments, path planning and trajectory generation are performed based on multi-sensor data and the welding task to obtain an initial trajectory. This may include: randomly determining a target work position from multiple work positions in the work area of ​​the equipment to be welded; performing path planning and trajectory generation based on the target work position, multi-sensor data, and the welding task to obtain an initial trajectory; the initial trajectory includes first trajectory data of the humanoid robot moving from the waiting area to the target work position. In this embodiment, by randomly determining the target position from multiple work positions and planning an initial trajectory including a movement segment, the flexible operation mode of manual welding can be effectively simulated, improving the robot's adaptability to non-fixed workstations and dynamic production environments. Simultaneously, the efficiency of the connection between movement and welding is optimized, facilitating the enrichment of data acquisition.

[0090] Specifically, the motion control module can determine the first trajectory data for the process of the humanoid robot moving from the waiting area to the work area, and the second trajectory data for the process of the robotic arm end moving to the starting point position after the humanoid robot reaches the work area and stops. It can also include the third trajectory data from the starting point position to the ending position.

[0091] In some embodiments, path planning and trajectory generation are performed based on multi-sensor data and the welding task to obtain an initial trajectory. This may include: determining the starting point position in the welding equipment based on image data from the multi-sensor data; determining the first coordinates of the starting point position in a first coordinate system of the welding equipment based on the image data; determining the second coordinates of the starting point position in the robotic arm coordinate system based on the first coordinates, according to a first transformation relationship and a second transformation relationship; the first transformation relationship is the transformation relationship between the humanoid robot's camera coordinate system and the robotic arm coordinate system; the second transformation relationship is the transformation relationship between the first coordinate system and the camera coordinate system; and performing path planning and trajectory generation based on the second coordinates, multi-sensor data, and the welding task to obtain the initial trajectory. The initial trajectory includes second trajectory data of the humanoid robot moving the welding torch to the starting point position. In this embodiment, by accurately locating the coordinates of the starting point in the robotic arm coordinate system through multi-coordinate system transformation, the accuracy and efficiency of the initial positioning of the welding torch can be significantly improved, reducing manual teaching steps and enhancing the robot's adaptive welding capability for complex workpieces and different installation postures.

[0092] The first coordinate system is a coordinate system established based on the equipment to be welded, such as a coordinate system established based on the compressor of the refrigerator to be welded.

[0093] For example, such as Figure 3 As shown, before the humanoid robot performs welding tasks, the system first performs hand-eye calibration to unify the coordinate systems of the camera, robot base, and tool end effector. The core of calibration lies in establishing precise transformation relationships, ensuring that the visual information acquired by the camera can be accurately mapped to the motion control of the robot end effector, thereby achieving precise positioning of the welding point and trajectory planning. Assume the humanoid robot's base coordinate system is... The coordinate system of the robotic arm's end effector is The camera coordinate system is The welding torch coordinate system is The coordinate system of the refrigerator compressor solder joints is The current end pose. The fixed relationship of the camera at the end of the robotic arm, obtained from the robot's forward kinematics. As a result of hand-eye calibration, the position of the solder joint observed by the camera in the camera coordinate system. Calculated by visual algorithms The fixed coordinate transformation from the robotic arm's end effector to the welding torch tip is a fundamental coordinate transformation. The core of the welding operation is to make the welding torch coordinate system... Coordinate system of welding point Overlap, that is The target pose that the end effector of the robotic arm should move to can be obtained by the following formula (1). Target pose in the base coordinate system :

[0094] (1)

[0095] After calculating the target pose that the robot's end effector needs to reach, the system transmits this pose information to the task planning module. The task planning module coordinates and initiates the motion control algorithm module to generate a trajectory plan that meets the welding task requirements. The core of trajectory planning is to ensure that the robot's end effector moves to the target position along a smooth and efficient path, while achieving precise attitude adjustment and welding point alignment in a dynamic environment.

[0096] In some embodiments, the multi-sensor data includes at least one of the following: image data of the workpiece to be welded, force sensing data of the workpiece being clamped by the clamping tool, and temperature data of the welding area. Based on the multi-sensor data and the welding task, path planning and trajectory generation are performed to obtain an initial trajectory, which may include: preprocessing the image data to obtain preprocessed image data; the preprocessing includes at least one of the following: Gaussian filtering and contrast enhancement; determining the spatial axis position of the workpiece to be welded in the welding equipment based on the preprocessed image data; determining the tilt angle of the workpiece to be welded based on the spatial axis position; and performing path planning and trajectory generation based on the tilt angle, force sensing data, temperature data, and the welding task to obtain the initial trajectory. In this embodiment, by preprocessing the image data and extracting the workpiece's spatial axis and tilt angle, the adaptability of the initial trajectory planning to complex workpiece postures can be significantly improved. Combined with force and temperature sensing data, the accuracy and process matching of the welding path are further enhanced, and the dependence on fixed tooling fixtures is reduced.

[0097] In this embodiment, the multi-sensor data may include at least one of image data, welding torch contact force sensing data, temperature data, inertial sensing data, foot force sensing data, joint encoder data, heat distribution data, arc current data, and visual images of the molten pool.

[0098] 203. During the process of controlling the humanoid robot to perform welding tasks according to the initial trajectory, random disturbances are injected, the initial trajectory is adjusted according to the random disturbances, and the welding task continues to be performed based on the adjusted initial trajectory.

[0099] Specifically, the system may also include a drive module, which is used for real-time joint control and feedback, including executing motion commands, collecting motor operating status, and providing real-time data support to the control system. This module uses a closed-loop control strategy to ensure high-precision consistency between the robot's movements and the expected path, while also supporting multi-joint coordinated motion to achieve complex welding posture adjustments. Real-time data from the drive module (such as joint position, velocity, and torque) is transmitted to the data module, serving as a crucial foundation for optimizing the robot's welding behavior.

[0100] In some embodiments, random perturbations include at least one of the following: joint trajectory perturbations of the humanoid robot, contact force perturbations between the welding torch and the equipment to be welded, visual occlusion perturbations, welding torch posture perturbations, arc discontinuity perturbations, center of gravity shift perturbations of the humanoid robot, parallel multitasking perturbations, welding machine current perturbations, and welding speed perturbations. In this application embodiment, by injecting various types of random perturbations, uncertainties in the real welding environment can be effectively simulated, enhancing the humanoid robot's anti-interference capability and trajectory robustness, improving its adaptability and welding quality stability in complex dynamic scenarios, and increasing the richness of data acquisition.

[0101] In some embodiments, random disturbances are injected during the process of controlling a humanoid robot to perform a welding task according to an initial trajectory, including at least one of the following: First, generating motion control commands based on the initial trajectory, superimposing a sinusoidal disturbance signal on the motion control commands, and injecting joint trajectory disturbances; the motion control commands are used to control the humanoid robot to hold a welding torch and approach the equipment to be welded; Second, after the welding torch contacts the equipment to be welded, closing the admittance control of the humanoid robot for a first preset duration, and injecting contact force disturbances; Third, during the welding process, controlling the arm of the humanoid robot to obstruct the field of view of the visual sensor for a second preset duration, and injecting visual obstruction disturbances; Fourth, the initial trajectory includes a welding path; superimposing a random deflection angle in the normal plane of the welding path, and injecting welding torch posture disturbances; Fifth, triggering an arc interruption for a third preset duration through the welding machine communication interface, and injecting arc discontinuity disturbances; Sixth, during the welding process, controlling the humanoid robot to lean forward by a preset angle, and injecting center of gravity offset disturbances; Seventh, during the welding process, initiating a voice broadcast task, and injecting parallel multitasking disturbances. In this embodiment, by simulating multi-dimensional real disturbances such as joints, force perception, vision, electric arcs, and center of gravity, and by adopting an active injection strategy, the adaptability and robustness of humanoid robots under extreme working conditions can be systematically improved, providing high-quality adversarial examples for algorithm training and accelerating their autonomous intelligent evolution.

[0102] 204. Record the status data during the execution of the welding task.

[0103] Specifically, the system may also include a data module, which is used for real-time acquisition, processing, storage, and uploading of multi-dimensional sensor data. The acquired data includes the robot's joint pose, joint torque, acceleration, visual information, force feedback, and heat distribution images during the welding process.

[0104] In some embodiments, the state data may further include recovery data for random disturbances. In this application embodiment, by recording recovery data for random disturbances, the anti-interference capability and adaptive performance of the robot system can be quantitatively evaluated, providing crucial data support for subsequent control algorithm optimization and reliability verification, and accelerating system iteration and upgrades.

[0105] The data acquisition method for humanoid robot training grounds provided in this embodiment can significantly improve the anti-interference ability and trajectory accuracy of humanoid robots in dynamic welding environments by combining multi-sensor data fusion and task-adaptive path planning with an online trajectory adjustment mechanism that injects random disturbances. At the same time, it provides high-quality data support for welding process optimization and robot autonomous learning by recording the status data throughout the entire process.

[0106] Figure 4 This is a schematic diagram of the structure of a data acquisition device for a humanoid robot training field provided in an embodiment of this application. Figure 4 As shown, the data acquisition device 40 for the humanoid robot training field includes: a determination module 401, a planning module 402, a disturbance module 403, and an acquisition module 404.

[0107] The determination module 401 is used to determine the welding task corresponding to the equipment to be welded.

[0108] The planning module 402 is used to acquire multi-sensor data, perform path planning and trajectory generation based on the multi-sensor data and welding task, and obtain an initial trajectory.

[0109] The disturbance module 403 is used to inject random disturbances into the process of controlling the humanoid robot to perform welding tasks according to the initial trajectory, adjust the initial trajectory according to the random disturbances, and continue to perform welding tasks based on the adjusted initial trajectory.

[0110] The data acquisition module 404 is used to record the status data during the execution of the welding task.

[0111] The data acquisition device for humanoid robot training grounds provided in this application embodiment generates trajectories based on multi-sensor data and automatically records data during task execution, thereby achieving automated data acquisition, improving data acquisition efficiency, and enhancing data richness by adding perturbations, which helps improve the generalization ability of humanoid robots under different production conditions.

[0112] In some embodiments, random disturbances include at least one of the following: joint trajectory disturbances of the humanoid robot, contact force disturbances between the welding torch and the equipment to be welded, visual occlusion disturbances, welding torch posture disturbances, arc discontinuity disturbances, center of gravity shift disturbances of the humanoid robot, parallel multitasking disturbances, welding machine current disturbances, and welding speed disturbances.

[0113] In some embodiments, the disturbance module 403 is specifically configured to perform at least one of the following:

[0114] Motion control commands are generated based on the initial trajectory. Sine perturbation signals are superimposed on the motion control commands, and joint trajectory perturbations are injected. The motion control commands are used to control the humanoid robot to hold the welding torch and approach the equipment to be welded.

[0115] After the welding torch comes into contact with the equipment to be welded, the admittance control of the humanoid robot is turned off for a first preset time, and a contact force disturbance is injected.

[0116] During the welding process, the arm of the humanoid robot is controlled to block the field of view of the visual sensor for a second preset time period, thus injecting a visual occlusion disturbance.

[0117] The initial trajectory includes the welding path; random deflection angles are superimposed in the normal plane of the welding path to inject welding gun attitude disturbances.

[0118] The arc interruption of the third preset duration is triggered through the welding machine communication interface to inject intermittent arc disturbance.

[0119] During the welding process, the humanoid robot is tilted forward at a preset angle, and a center of gravity shift disturbance is injected.

[0120] During the welding process, a voice broadcast task is initiated, injecting parallel multi-task disturbances.

[0121] In some embodiments, the determining module 401 is specifically used to: acquire image data of the equipment to be welded; determine the current workpiece to be welded in the equipment to be welded based on the process flow and image data of the equipment to be welded; and retrieve the corresponding process parameters based on the current workpiece to be welded to obtain the welding task.

[0122] In some embodiments, the planning module 402 is specifically used to: randomly determine a target work position from multiple work positions in the work area of ​​the equipment to be welded; perform path planning and trajectory generation based on the target work position, multi-sensor data and welding task to obtain an initial trajectory; the initial trajectory includes the first trajectory data of the humanoid robot moving from the waiting area to the target work position.

[0123] In some embodiments, the planning module 402 is specifically used to: determine the starting point position in the welding device based on image data from multi-sensor data; determine the first coordinates of the starting point position in the first coordinate system of the welding device based on the image data; determine the second coordinates of the starting point position in the robotic arm coordinate system based on the first coordinates according to the first transformation relationship and the second transformation relationship; the first transformation relationship is the transformation relationship between the camera coordinate system of the humanoid robot and the robotic arm coordinate system; the second transformation relationship is the transformation relationship between the first coordinate system and the camera coordinate system; perform path planning and trajectory generation based on the second coordinates, multi-sensor data and welding task to obtain an initial trajectory; the initial trajectory includes the second trajectory data of the humanoid robot moving the welding torch to the starting point position.

[0124] In some embodiments, the state data includes recovery data in response to random disturbances.

[0125] In some embodiments, the multi-sensor data includes at least one of the following: image data of the workpiece to be welded, force sensing data of the workpiece being clamped by the clamping tool, and temperature data of the welding area; the planning module 402 is specifically used for: preprocessing the image data to obtain preprocessed image data; the preprocessing includes at least one of the following: Gaussian filtering, contrast enhancement; determining the spatial axis position of the workpiece to be welded in the welding equipment based on the preprocessed image data; determining the tilt angle of the workpiece to be welded based on the spatial axis position; and performing path planning and trajectory generation based on the tilt angle, force sensing data, temperature data, and welding task to obtain an initial trajectory.

[0126] The data acquisition device for humanoid robot training field provided in this application embodiment can be used to execute the above method embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0127] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the electronic device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus.

[0128] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0129] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0130] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0131] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0132] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0133] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0134] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0135] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0136] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0137] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0138] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0139] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0140] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0142] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A data collection method for a humanoid robot training field, characterized by, The method comprises the following steps: determining a welding task corresponding to a device to be welded; obtaining multi-sensor data; based on the multi-sensor data and the welding task, performing path planning and trajectory generation to obtain an initial trajectory; in the process of controlling the humanoid robot to perform the welding task according to the initial trajectory, injecting random disturbance, adjusting the initial trajectory according to the random disturbance, and continuing to perform the welding task based on the adjusted initial trajectory; recording state data in the process of performing the welding task; the random disturbance comprises at least one of the following: joint trajectory disturbance of the humanoid robot, contact force disturbance between the welding torch and the device to be welded, visual occlusion disturbance, welding torch posture disturbance, electric arc interruption disturbance, center of gravity offset disturbance of the humanoid robot, parallel multi-task disturbance, welding machine current disturbance, and welding speed disturbance; the injection of random disturbance in the process of controlling the humanoid robot to perform the welding task according to the initial trajectory comprises at least one of the following: generating a motion control instruction according to the initial trajectory, superimposing a sinusoidal disturbance signal on the motion control instruction, and injecting the joint trajectory disturbance; the motion control instruction is used to control the humanoid robot to hold the welding torch close to the device to be welded; after the welding torch contacts the device to be welded, the admittance control of the humanoid robot is turned off for a first preset time length, and the contact force disturbance is injected; in the welding process, the visual sensor field of view of the humanoid robot arm is controlled to be shielded within a second preset time length, and the visual occlusion disturbance is injected; the initial trajectory comprises a welding path; a random angle is superimposed in the normal plane of the welding path, and the welding torch posture disturbance is injected; an electric arc interruption of a third preset time length is triggered through a welding machine communication interface, and the electric arc interruption disturbance is injected; in the welding process, the humanoid robot is controlled to lean forward by a preset angle, and the center of gravity offset disturbance is injected; determining a welding task corresponding to a device to be welded comprises: obtaining image data of the device to be welded; determining a current workpiece to be welded in the device to be welded according to the process flow of the device to be welded and the image data; obtaining a welding task by retrieving corresponding process parameters according to the current workpiece to be welded; the initial trajectory is obtained by performing path planning and trajectory generation based on the second coordinate, the multi-sensor data, and the welding task; the initial trajectory comprises second trajectory data of the humanoid robot moving the welding torch to the welding starting point position. ​ ​ ​ ​ 2. The method of claim 1, wherein, The path planning and trajectory generation are performed according to the multi-sensing data and the welding task to obtain an initial trajectory, including: randomly determining a target work position from a plurality of work positions in a work area of the equipment to be welded; performing path planning and trajectory generation according to the target work position, the multi-sensing data and the welding task to obtain an initial trajectory; the initial trajectory includes first trajectory data of the humanoid robot moving from the waiting area to the target work position.

3. The method according to any of claims 1-2, characterized in that, The state data includes recovery data for the random disturbance.

4. The method according to any one of claims 1-2, characterized in that, The multi-sensing data includes at least one of the following: image data of the workpiece to be welded, force sensing data when the clamping tool clamps the workpiece to be welded, temperature data of the welding area; The path planning and trajectory generation are performed according to the multi-sensing data and the welding task to obtain an initial trajectory, including: preprocessing the image data to obtain preprocessed image data; the preprocessing includes at least one of the following: Gaussian filtering, contrast enhancement; determining the spatial axis position of the workpiece to be welded in the equipment to be welded according to the preprocessed image data; determining the inclination angle of the workpiece to be welded according to the spatial axis position; performing path planning and trajectory generation according to the inclination angle, the force sensing data, the temperature data and the welding task to obtain an initial trajectory.

5. A data acquisition device for a humanoid robot training arena, characterized by, including: at least one processor and a memory; The memory stores computer execution instructions; The at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the data acquisition method for the humanoid robot training field as claimed in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and when the processor executes the computer execution instructions, the data acquisition method for the humanoid robot training field as claimed in any one of claims 1 to 4 is realized.

Images