A dual-robot brazing collaborative control method and system based on AI vision guidance

Abstract

This application discloses an AI vision-guided dual-robot brazing collaborative control method and system, relating to the field of industrial automation welding technology. The method involves: acquiring a workpiece image at the target workstation; processing the workpiece image, using an AI model to identify feature points on the workpiece image, and outputting the 3D coordinates and normal vectors of the feature points; based on the visual positioning results, the current state of each robot, and the state of each workstation, assigning robots to the welding task at the target workstation based on a dynamic load balancing algorithm, and generating task assignment instructions; generating robot motion sequences based on the task assignment instructions and visual positioning results, and coupling welding process parameters with the robot motion sequences to generate coupled control instructions; and controlling the assigned robots to perform welding operations based on the coupled control instructions. This method solves the problems of dynamic task scheduling and motion collaborative control in parallel dual-robot operations, improving the production efficiency and welding accuracy of the dual-robot collaborative welding system.

Description

Technical Field

[0001] This application relates to the field of industrial automation welding technology, specifically to a dual-robot brazing collaborative control method and system based on AI vision guidance. Background Technology

[0002] This application relates to the field of industrial automated welding technology, and more specifically, to an automated brazing control system and method based on machine vision and industrial robots, which is particularly suitable for high-quality and high-consistency automated high-frequency induction brazing operations on workpieces such as pipes and shells under tight production cycles.

[0003] In existing automated brazing systems, using machine vision to guide robots in welding path planning is an important means to improve flexibility and precision. For example, Chinese patent application CN121515195A discloses a dynamic path planning method and system for embodied intelligent industrial robots based on data acquisition. This technical solution collects workpiece point cloud data through multimodal sensors (such as LiDAR and structured light) and dynamically adjusts the parameters of the iterative nearest point registration algorithm based on the curvature of the point cloud to achieve high-precision path planning for complex curved surfaces (such as artificial joints). This solution represents a technical approach in the prior art to achieve precise robot trajectory planning through visual perception and adaptive algorithms.

[0004] However, directly applying the aforementioned existing technologies to mass production scenarios with extremely high cycle time requirements (e.g., ≤4.6 seconds / piece), such as high-frequency brazing of compressor exhaust pipes, reveals significant shortcomings. The core of this technical solution lies in reconstructing and fitting complex curved surface geometry through point cloud registration. Its data processing flow is lengthy and computationally burdensome, making it difficult to meet the real-time requirements of millisecond-level response. More importantly, this solution is designed as a single-robot, single-task system, completely neglecting dynamic task scheduling, motion coordination, and real-time synchronous control with core brazing process parameters such as high-frequency heating and wire feeding in a dual-robot parallel operation environment. Therefore, directly applying this solution in dual-robot, dual-station brazing production lines that pursue extremely high production efficiency will lead to low robot utilization, substandard cycle time, and even downtime or quality defects due to coordination conflicts, failing to meet the demands of modern, efficient, and flexible brazing production. Summary of the Invention

[0005] The purpose of this application is to provide a dual-robot brazing collaborative control method and system based on AI vision guidance to solve the problems mentioned in the background art.

[0006] In a first aspect, one embodiment of this application provides an AI vision-guided dual-robot brazing collaborative control method, applied to a production line including a first station and a second station. The method includes: acquiring a workpiece image of the target station, the workpiece image including an image captured by a top camera and an image captured by a side camera; processing the workpiece image, using an AI model to identify feature points on the workpiece image, and outputting the three-dimensional coordinates and normal vectors of the feature points in the robot base coordinate system to form a visual positioning result; dynamically allocating a robot to the welding task of the target station based on a dynamic load balancing algorithm according to the visual positioning result, the current state of each robot, and the state of each station, and generating a task allocation instruction; generating a robot motion sequence according to the task allocation instruction and the visual positioning result, and coupling the welding process parameters with the robot motion sequence to generate a coupling control instruction, the welding process parameters including at least heating power and wire feeding speed; and controlling the allocated robot to move to the target station to perform the welding operation based on the coupling control instruction.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, acquiring a workpiece image of the target workstation, processing the workpiece image, and using an AI model to identify feature points on the workpiece image includes: responding to a target workstation arrival signal, synchronously triggering a top camera and a side camera positioned above the target workstation to capture images, obtaining a top image and a side image; preprocessing the top image and the side image respectively to obtain standardized image data; inputting the standardized image data into a pre-trained AI model, and the AI ​​model outputting a heatmap corresponding to the feature points; determining the pixel coordinates of the feature points in the top image and the side image based on the heatmap; and calculating the three-dimensional coordinates of the feature points in the robot base coordinate system based on the pixel coordinates and the intrinsic and extrinsic parameters of the top camera and the side camera.

[0008] In conjunction with the first aspect, in certain implementations of the first aspect, standardized image data is input into a pre-trained AI model, and the AI ​​model outputs a heatmap corresponding to the feature points. This includes: the AI ​​model employing a convolutional neural network model; inputting standardized image data into the convolutional layer of the convolutional neural network model to obtain a convolutional layer output feature map; applying a non-linear activation function to the convolutional layer output feature map to obtain activated feature values; expanding the receptive field through the pooling layer of the convolutional neural network model based on the activated feature values ​​to obtain pooled feature values; inputting the pooled feature values ​​into multiple alternating stacks of convolutional layers, activation functions, and pooling layers to obtain a higher-order feature map; setting welding points in the higher-order feature map as feature points; and for each feature point, the convolutional neural network model outputting a corresponding heatmap.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, a robot is dynamically allocated to the welding task at the target workstation based on a dynamic load balancing algorithm. This includes: creating a dynamic task queue containing welding tasks; obtaining the status information of all robots, which includes at least the robot ID, current position, current status, and estimated task completion time; determining a list of available robots based on the status information; for the welding task at the head of the dynamic task queue, traversing the list of available robots and calculating the comprehensive cost of allocating the welding task to each robot in the list; the comprehensive cost is a weighted sum of the travel time cost and the current load cost; selecting the robot that minimizes the comprehensive cost as the selected robot and allocating the welding task to the selected robot.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, welding process parameters are coupled with the robot motion sequence to generate coupled control instructions, including: discretizing the robot motion sequence into multiple path points and assigning a path progress value to each path point; assigning a corresponding heating power setting value and wire feeding speed setting value to each path point according to the path progress value corresponding to each path point; and updating the heating power setting value and wire feeding speed setting value as additional parameters to the corresponding path points in the robot motion sequence to generate coupled control instructions.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, a robot motion sequence is generated based on the task allocation instructions and visual positioning results, including: planning a trajectory from the robot's starting position to the welding target point based on the three-dimensional coordinates and normal vectors in the visual positioning results, the trajectory including at least an approach segment, an introduction segment, a welding segment, and a retraction segment; discretizing the trajectory into multiple path points, each path point including coordinates, attitude, and path progress value; calculating the instantaneous linear velocity at each path point based on the coordinates, attitude, and path progress value of the path points; and assembling the coordinates, attitude, path progress value, and instantaneous linear velocity of the path points into a robot motion sequence.

[0012] In conjunction with the first aspect, some implementations of the first aspect also include: obtaining the angles of each joint axis of the robot corresponding to each path point through inverse kinematics calculation; generating joint position, joint velocity, and joint acceleration through trajectory interpolation based on the timestamp of each path point and the corresponding joint axis angle; and constructing a joint space control sequence from the joint position, joint velocity, and joint acceleration, which is used to control the posture of each joint of the robot at the path point in real time.

[0013] Secondly, this application provides an AI vision-guided dual-robot brazing collaborative control system, applied to a production line including a first station and a second station, comprising: a vision positioning module, used to acquire workpiece images of the target station, process the workpiece images, use an AI model to identify feature points on the workpiece images, and output the three-dimensional coordinates and normal vectors of the feature points in the robot base coordinate system to form a vision positioning result; a dynamic scheduling module, used to dynamically allocate a robot to the welding task of the target station based on the vision positioning result, the current state of each robot, and the state of each station, and generate a task allocation instruction; a trajectory and process planning module, used to generate a robot motion sequence based on the task allocation instruction and the vision positioning result, and couple the welding process parameters with the robot motion sequence to generate a coupling control instruction; the welding process parameters include at least heating power and wire feeding speed; and an execution module, used to control the allocated robot to move to the target station to perform the welding operation based on the coupling control instruction.

[0014] Thirdly, this application provides an electronic device, including: a processor; and a memory, in which computer program instructions are stored, which, when executed by the processor, implement the steps in the AI ​​vision-guided dual-robot brazing collaborative control method mentioned in the first aspect above.

[0015] Fourthly, this application provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, cause the processor to perform the steps in the AI ​​vision-guided dual-robot brazing collaborative control method mentioned in the first aspect.

[0016] The AI ​​vision-guided dual-robot brazing collaborative control method provided in this application solves the problems of dynamic task scheduling, motion coordination, and real-time synchronization control of core brazing process parameters such as high-frequency heating and wire feeding in high-cycle (≤4.6 seconds / piece) dual-station production scenarios. This is achieved by employing an AI vision-guided system with top and side cameras for high-precision 3D workpiece positioning, combined with dynamic task allocation based on a dynamic load balancing algorithm, and coupling control of welding process parameters with robot motion sequences. This improves the overall production efficiency, welding accuracy, and process stability of the dual-robot collaborative welding system. Attached Figure Description

[0017] Figure 1 The diagram shown is a flowchart illustrating an AI vision-guided dual-robot brazing collaborative control method provided in an exemplary embodiment of this application.

[0018] Figure 2The diagram shown is a flowchart of a dual-robot brazing collaborative control method based on AI vision guidance, provided in another exemplary embodiment of this application.

[0019] Figure 3 The diagram shown is a flowchart of a dual-robot brazing collaborative control method based on AI vision guidance, provided in another exemplary embodiment of this application.

[0020] Figure 4 The diagram shown is a flowchart of a dual-robot brazing collaborative control method based on AI vision guidance, provided in another exemplary embodiment of this application.

[0021] Figure 5 The diagram shown is a flowchart of a dual-robot brazing collaborative control method based on AI vision guidance, provided in another exemplary embodiment of this application.

[0022] Figure 6 The diagram shown is a flowchart of a dual-robot brazing collaborative control method based on AI vision guidance, provided in another exemplary embodiment of this application.

[0023] Figure 7 The diagram shown is a flowchart of a dual-robot brazing collaborative control method based on AI vision guidance, provided in another exemplary embodiment of this application.

[0024] Figure 8 The diagram shown is an architectural schematic of an AI vision-guided dual-robot brazing collaborative control system provided in an exemplary embodiment of this application.

[0025] Figure 9 The diagram shown is a structural schematic of an electronic device provided in an exemplary embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The following is combined Figures 1 to 7 This application provides a detailed description of a dual-robot brazing collaborative control method based on AI vision guidance.

[0028] Figure 1 The diagram shown is a flowchart illustrating an AI vision-guided dual-robot brazing collaborative control method provided in an exemplary embodiment of this application. Figure 1As shown in the embodiments of this application, the AI ​​vision-guided dual-robot brazing collaborative control method includes the following steps.

[0029] Step 100: Obtain the workpiece image of the target station.

[0030] For example, the workpiece image refers to an image obtained by the top camera and an image obtained by the side camera.

[0031] Step 102: Process the workpiece image, use the AI ​​model to identify feature points on the workpiece image, and output the three-dimensional coordinates and normal vectors of the feature points in the robot base coordinate system to form a visual positioning result.

[0032] For example, a feature point refers to a specific location on a workpiece image that needs to be brazed at high frequency, as identified by a vision system.

[0033] For example, the robot base coordinate system refers to a coordinate system with the robot as the base, the robot base as the origin, the x-axis pointing forward of the robot as the positive direction, the y-axis pointing to the left of the robot as the positive direction, and the z-axis pointing upward as the positive direction.

[0034] For example, the normal vector refers to the ideal direction vector perpendicular to the welding surface of the workpiece at the feature point, which together with the three-dimensional coordinates of the feature point constitutes a complete spatial pose description.

[0035] Step 104: Based on the visual positioning results, the current status of each robot and the status of each workstation, dynamically allocate a robot to the welding task of the target workstation based on the dynamic load balancing algorithm, and generate task allocation instructions.

[0036] For example, the robot's current state includes the robot ID, current state, current location, and estimated completion time of the current task.

[0037] For example, the status of a workstation includes workstation ID, lifting status, and stopper status.

[0038] For example, a dynamic load balancing algorithm refers to making decisions by calculating an evaluation value called overall cost, always assigning new tasks to the robot that minimizes the overall cost.

[0039] Step 106: Generate a robot motion sequence based on the task allocation instructions and visual positioning results, and couple the welding process parameters with the robot motion sequence to generate coupled control instructions.

[0040] For example, a robot motion sequence refers to a set of spatial trajectory data planned by the control system that describes the robot's movement from the starting point to the welding point and the completion of the welding path.

[0041] For example, welding process parameters include heating power and wire feed speed.

[0042] Step 108: Based on the coupling control command, control the assigned robot to move to the target workstation to perform welding operations.

[0043] For example, coupled control instructions refer to the set of executable instructions that are ultimately sent to the robot and the high-frequency welding equipment, deeply integrating spatial motion with temporal processes.

[0044] The AI ​​vision-guided dual-robot brazing collaborative control method provided in this application solves the problems of dynamic task scheduling, motion coordination, and real-time synchronization control of core brazing process parameters such as high-frequency heating and wire feeding in high-cycle (≤4.6 seconds / piece) dual-station production scenarios. This is achieved by employing an AI vision-guided system with top and side cameras for high-precision 3D workpiece positioning, combined with dynamic task allocation based on a dynamic load balancing algorithm, and coupling control of welding process parameters with robot motion sequences. This improves the overall production efficiency, welding accuracy, and process stability of the dual-robot collaborative welding system.

[0045] Figure 2 The diagram shown is a flowchart illustrating an AI vision-guided dual-robot brazing collaborative control method provided in another exemplary embodiment of this application. Figure 1 This application extends from the embodiments shown. Figure 2 The illustrated embodiment will be described in detail below. Figure 2 The illustrated embodiments and Figure 1 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0046] like Figure 2 As shown in the embodiment of this application, the dual-robot brazing collaborative control method based on AI vision guidance obtains the workpiece image of the target station, processes the workpiece image, and uses an AI model to identify feature points on the workpiece image, including the following steps.

[0047] Step 200: In response to the target workstation arrival signal, the top camera and the side camera set above the target workstation are synchronously triggered to take pictures, and the top image and the side image are obtained.

[0048] Specifically, firstly, the Boolean signals from the photoelectric sensors on the dual-station conveyor line are continuously monitored. When the pallet carrying the tooling plate moves to the predetermined position and is locked by the lifting mechanism, the sensor status changes from FALSE to TRUE. When the sensor status is TRUE, a structured command message is immediately sent to the vision module via the industrial Ethernet. This command message includes a command word (such as TRIGGER_CAPTURE), the target station ID (such as STATION_1), and the task sequence number.

[0049] For example, the central control PLC is the core of the production line cycle, and its signals are used to ensure that the visual actions are precisely synchronized with the material flow, thus avoiding invalid shooting by the camera when the workpiece is not in place or has been moved away.

[0050] Next, the vision module interprets the instruction and sends a high-level pulse signal to both the top and side cameras simultaneously via a dedicated trigger output signal line. Upon receiving the high-level pulse signal, both cameras immediately perform a shutter exposure to capture the scene image at the current moment.

[0051] It should be understood that the parameters of the two cameras are preset, including exposure time, gain, and other parameters.

[0052] For example, a high-level pulse signal is used to control two cameras to ensure that the two perspectives capture the state of the workpiece at the same physical instant, thus eliminating the micro-motion error of the object caused by taking pictures one after the other.

[0053] Finally, the scene images captured by the two cameras are converted into raw image data and pushed to the image buffer of the vision module via an interface protocol. The raw image data includes Image_Top, an image data block captured by the top camera, and Image_Side, an image data block captured by the side camera. The two images have the same timestamp and are pixel-aligned, and are in RGB format.

[0054] Step 202: Preprocess the top and side images respectively to obtain standardized image data.

[0055] Specifically, firstly, based on the image data blocks Image_Top and Image_Side, the same normalization operation is performed independently on each image. Then, using a bilinear interpolation algorithm, the images are scaled to the input size preset by the AI ​​model, resulting in scaled image data blocks Image_Top' and Image_Side'. The input size can be 640 pixels (width) × 480 pixels (height).

[0056] It should be understood that the AI ​​model is used to convert the standard input image of the position to be welded into pixel coordinates that can be recognized by the central control PLC. The AI ​​model can be various lightweight neural network models. In this embodiment, a convolutional neural network (CNN) model is used.

[0057] Next, based on the scaled image data blocks Image_Top' and Image_Side', the RGB channel intensity values ​​of each pixel are converted into floating-point numbers and then normalized to obtain the normalized floating-point values ​​for the RGB channels. value ( i ), i ={R, G, B}. Where the RGB channel intensity values ​​are integers in the original range of 0-255.

[0058] ; in, value ( i ) represents the original value of the R, G, or B channel of the corresponding pixel in the scaled image; value ( i () represents the normalized floating-point value of the R, G, or B channels of the corresponding pixel; mean and std These are the pre-calculated mean and standard deviation, derived from the statistical values ​​of the dataset used during AI model training.

[0059] Finally, the scaled and normalized image data blocks Image_Top' and Image_Side' are encapsulated into an image tensor Tensor_input. The dimensions of the image tensor can be [...]. x , 3, H, W], x Indicates the first x ( x =1, 2) images, 3 represents the color (RGB) channel, H represents the image height in pixels, for example 480 pixels; W represents the image width in pixels, for example 640 pixels.

[0060] Step 204: Input the standardized image data into the pre-trained AI model, and the AI ​​model outputs a heatmap corresponding to the feature points.

[0061] Step 206: Determine the pixel coordinates of the feature points in the top and side images based on the heatmap.

[0062] Specifically, based on the coordinates of the feature points ( u k , v k), scaled up to the original input image size to obtain the final coordinates of the feature points. u original , v original ).

[0063] u original = u k * (W / W k ); v original = v k * (H / H k ); Step 208: Calculate the three-dimensional coordinates of the feature points in the robot's base coordinate system based on the pixel coordinates and the intrinsic and extrinsic parameters of the top and side cameras.

[0064] Specifically, firstly, based on the final coordinates of the feature points ( u original , v original The algorithm pairs the pixel coordinates of the same feature point in the two images, forming matching pairs (u_top_i, v_top_i) and (u_side_i, v_side_i). Here, (u_top_i, v_top_i) represents the coordinates of the i-th feature point in the image taken by the top camera; (u_side_i, v_side_i) represents the coordinates of the i-th feature point in the image taken by the side camera.

[0065] Next, for the coordinates of each pair of matched feature points, a system of linear equations is constructed by combining the projection matrices of the two cameras to solve for its three-dimensional coordinates P_target = (X, Y, Z).

[0066] s * [ u ; v ; 1] = K * [R | t]* [X_target; Y_target; Z_target; 1]; in, s Indicates a non-zero scaling factor; u ; v [1] represents the homogeneous coordinates of the feature point; K represents the intrinsic parameter matrix of the camera; [R | t] represents the rotation and translation matrix of the camera relative to the world coordinate system, i.e., the extrinsic parameter matrix; [X_target;Y_target; Z_target; 1] represents the homogeneous coordinates of the feature point in the world coordinate system.

[0067] It should be understood that the world coordinate system refers to a coordinate system with the robot as the base, the robot base as the origin, the x-axis pointing in front of the robot (along the horizontal direction of the workbench) as the positive direction, the y-axis pointing to the left of the robot (along the vertical direction of the workbench) as the positive direction, and the z-axis pointing upward (vertically away from the ground) as the positive direction.

[0068] For example, by simultaneously solving the linear equations of the two cameras, the non-zero scale factor can be eliminated. s And solve for (X_target, Y_target, Z_target).

[0069] Secondly, based on multiple feature points that are not on the same straight line, the local plane is fitted by the three-dimensional coordinates of the feature points to obtain the unit normal vector N_target = (Nx, Ny, Nz) of the local plane.

[0070] It should be understood that the unit normal vector is used to represent the robot's welding torch posture, ensuring that the robot approaches the welding point at the correct angle.

[0071] Finally, a visual data packet is constructed based on the three-dimensional coordinates P_target and unit normal vector N_target of multiple feature points, as well as the station ID_STA and timestamp T_vision of the dual-station conveyor line.

[0072] The AI-guided dual-robot brazing collaborative control method provided in this application solves the problems of incomplete feature point recognition and insufficient positioning accuracy caused by single-viewpoint or asynchronous shooting by synchronously triggering dual-camera shooting in response to the workstation arrival signal, performing standardized preprocessing on the images, outputting feature point heatmaps using an AI model, and then calculating three-dimensional coordinates using camera parameters. This achieves rapid and high-precision three-dimensional visual positioning of welding feature points on the workpiece, providing a reliable basis for subsequent precise robot operations.

[0073] Figure 3 The diagram shown is a flowchart illustrating a dual-robot brazing collaborative control method based on AI vision guidance, provided in another exemplary embodiment of this application. Figure 1 This application extends from the embodiments shown. Figure 3 The illustrated embodiment will be described in detail below. Figure 3 The illustrated embodiments and Figure 1 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0074] like Figure 3 As shown in the embodiment of this application, the dual-robot brazing collaborative control method based on AI vision guidance involves inputting standardized image data into a pre-trained AI model, and the AI ​​model outputting a heat map corresponding to the feature points, including the following steps.

[0075] The AI ​​model uses a convolutional neural network model, including: Step 300: Input the standardized image data into the convolutional layer of the convolutional neural network model to obtain the output feature map of the convolutional layer.

[0076] Specifically, the image tensor (Tensor_input) is set to a matrix X with height H and width W. The weighted sum of local regions is calculated using the convolutional layers of a CNN model, including the following steps. The convolutional layers have a total of... l Each layer has K convolutional kernels, K0. l Indicates the first l There are 1 convolutional kernel, and each layer has 3 channels. i , j ) represents the position in matrix X. i , j (pixels).

[0077] It should be understood that the number of channels in a convolutional layer is the same as the number of R, G, or B channels in a pixel, and the calculation process for each channel is also the same. Therefore, we will take processing one channel of a single image (such as Image_Top) as an example.

[0078] For a given input image, the () l -1) Layer, Matrix X (l-1) Upper position ( i , j The pixel of ) after the first l K convolutional kernels l After calculation, the first number is obtained. l Layer feature map Z l At position ( i , j The value of ) is calculated using the following formula: ; Among them, Z l ( i , j ) indicates the first l Layer output feature map at position ( i , j The value of ); b l Indicates the first l K convolution kernel of layer l Corresponding bias term; K l ( m , n ) indicates the first l Each convolutional kernel is located at ( m , n The weighting parameters of X; (l-1) ( i + m, j + n ) indicates that the feature map of the previous layer is in the local window ( i , j Offset ( m , n The value at the position.

[0079] It should be understood that the feature map of the previous layer is matrix X. (l-1) The convolution kernel slides across the entire spatial domain of the input matrix X. Each convolution kernel generates an independent feature map, and multiple convolution kernels generate multiple feature maps to capture different features (such as edges and textures).

[0080] For example, the convolutional kernel K is a set of small templates learned from the data. The computation process is similar to comparing the templates on an image, outputting high values ​​at locations with high matching scores, thereby activating the feature.

[0081] Step 302: Based on the feature map output by the convolutional layer, apply a non-linear activation function to obtain the activated feature values.

[0082] Specifically, based on the convolution operation, the first... l Layer feature map Z l By applying a non-linear activation function ReLU, the activated eigenvalues ​​A are obtained. l ( i , j ).

[0083] A l ( i , j )=ReLU[Z l ( i , j )]= max (0, Z l ( i , j )); It should be understood that the activation function ReLU represents taking values ​​of 0 and Z. l ( i , j The maximum value between ) is used to set all negative values ​​in the feature map to zero, introducing non-linearity so that the network can approximate arbitrarily complex functions.

[0084] Step 304: Based on the activated feature values, expand the receptive field through the pooling layer of the convolutional neural network model to obtain the pooled feature values.

[0085] Specifically, based on the activated eigenvalue A l ( i , jThe receptive field is expanded through pooling layers in the CNN model to obtain pooled feature values. P l ( i , j ).

[0086] P l ( i , j ) = max (A) l ( i *S + p, j *S + q) ), {p,q ∈ [0, P-1]}; in, P l ( i , j ) indicates that the output after pooling is at position ( i , j eigenvalues; max This indicates taking the maximum value of all activation values ​​within the P×P region. The P×P region is preset within the pooling layer and applied to the input feature values ​​to expand the receptive field. S represents the pooling step size, which can be equal to the pooling window size P, for example, P=2.

[0087] It should be understood that pooling operations are used to output local regions ( i *S + p, j The most significant feature of *S + q) is that it reduces the output size to about 1 / S of the input.

[0088] For example, the pooling layer does not care about the precise location of the feature within the P×P region, but only whether there is a strong feature in the region. This makes the CNN model less sensitive to small changes in the position of the feature, such as slight deviations in the placement of the workpiece, thus improving robustness.

[0089] It should be understood that multiple convolutional layers, activation layers, and pooling layers are stacked alternately. That is, after the input matrix X is processed through convolutional layers, activation layers, and pooling layers, the output feature values ​​are input into convolutional layers, activation layers, and pooling layers again for cyclical processing.

[0090] Step 306: The pooled feature values ​​are input into multiple alternating stacks of convolutional layers, activation functions, and pooling layers to obtain a high-order feature map. The welding points in the high-order feature map are set as feature points. For each feature point, the convolutional neural network model outputs the corresponding heatmap.

[0091] Specifically, after alternating stacking of multiple convolutional layers, activation layers, and pooling layers, a high-order feature map is obtained. Welding points in this high-order feature map are set as feature points. For the k-th predefined feature point, the CNN model outputs a heatmap H proportional to the size of the input image. k The coordinates of the feature points ( u k , v k This is obtained by finding the maximum value of the heatmap. Each pixel value on the heatmap represents the probability that the point is feature point k.

[0092] ; Among them, H k ( i , j ) represents the value of the predicted heatmap at the k-th channel and position (i, j); argmax This operation retrieves the position of the maximum value and returns a value that makes H equal to the maximum value. k ( i , j The largest coordinate ( i , j ).

[0093] It should be understood that the output heatmap is scaled down proportionally to the input image. Therefore, subsequent steps require scaling up the feature points in the heatmap to the size of the original input image in order to determine the coordinates of the feature points.

[0094] For example, a CNN model is trained to illuminate a heatmap where feature points are likely to appear. The pixel coordinates of the feature points are obtained by finding the brightest points.

[0095] The AI ​​vision-guided dual-robot brazing collaborative control method provided in this application solves the technical problem of accurate and stable identification of feature points in complex workpiece backgrounds by using a convolutional neural network as the AI ​​model and performing multiple alternating processes such as convolution, activation, and pooling to extract high-order features and output feature point heatmaps. This achieves the effect of significantly improving the accuracy and anti-interference capability of identifying key features such as welding points by leveraging the powerful feature extraction capabilities of deep learning models.

[0096] Figure 4 The diagram shown is a flowchart illustrating a dual-robot brazing collaborative control method based on AI vision guidance, provided in another exemplary embodiment of this application. Figure 1 This application extends from the embodiments shown. Figure 4 The illustrated embodiment will be described in detail below. Figure 4 The illustrated embodiments and Figure 1 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0097] like Figure 4 As shown in the embodiment of this application, the AI ​​vision-guided dual-robot brazing collaborative control method dynamically allocates a robot to the welding task of the target station based on a dynamic load balancing algorithm, including the following steps.

[0098] Step 400: Create a dynamic task queue containing welding tasks.

[0099] Step 402: Obtain the status information of all robots. The status information includes at least the robot ID, current location, current status, and estimated task completion time.

[0100] Step 404: Determine the list of available robots based on the status information.

[0101] Specifically, first, based on the visual data packet, obtain the workstation ID_STA, 3D coordinates P_target, unit normal vector N_target, and timestamp T_vision; By monitoring the robot's status, the robot ID_RB, current state_RB, current position P_current_RB, and estimated completion time T_complete_RB for the current task are obtained; among them, the current state_RB includes idle, moving, welding, and fault.

[0102] The station status signals, including station ID_STA, lifting status, and stopper status, are obtained through the central control PLC.

[0103] It should be understood that the workstation ID_STA corresponds to different conveyor lines. In this embodiment, a dual-workstation conveyor line is set, so the workstation ID_STA = {1, 2}. It can also be a multi-workstation setting. The current position P_current_RB refers to the coordinates of the robot's current position in the robot's base coordinate system. The lifting state and the blocking state each have only two states: working or idle, which are used to confirm whether the workstation can be used for operation.

[0104] Next, the central control PLC continuously monitors the three data sources mentioned above (visual data packets, robot status, and workstation status signals). When a new task information is received (visual data packet received), a new welding task is immediately created, and two real-time updated data tables are set: the robot status table and the workstation status table. The data in the robot status table is the robot status information, and the data in the workstation status table is the workstation status information.

[0105] For example, acquiring data from multiple sources ensures that the necessary up-to-date and comprehensive data is available for decision-making. All information (tasks, robot positions, workstation status) is synchronized to a unified time base to avoid decision-making errors due to information delays, such as assigning a task to a robot that is actually about to collide with it.

[0106] Secondly, the new welding task is constructed into a structured welding task object T_i, which includes task number, workstation ID_STA, welding target point (P_target, N_target), task priority, generation timestamp, and allocation status.

[0107] It should be understood that the task number is an index of the task number automatically set when a new welding task is created, using an auto-incrementing sequence; the workstation ID_STA indicates which workstation on the conveyor line the new welding task originates from; the welding target point includes three-dimensional coordinates and a unit normal vector, used to determine the welding point position and welding posture; the task priority is set by the worker, and defaults to normal if not set; the generation timestamp refers to the time when the new welding task was created; the allocation status indicates whether the new welding task has been assigned to the robot for execution. Since it is currently in the task creation stage, the allocation status = pending allocation.

[0108] Finally, the welding task object T_i is inserted at the end of the preset dynamic task queue. This dynamic task queue is a priority queue based on the generation timestamp and task priority, and it operates on a first-in, first-out (FIFO) basis by default, unless a higher-priority task is inserted.

[0109] Step 406: For the welding task at the head of the dynamic task queue, traverse the list of available robots and calculate the comprehensive cost of assigning the welding task to each robot in the list; the comprehensive cost is the weighted sum of the travel time cost and the current load cost.

[0110] Specifically, firstly, based on the welding task object T_i in the dynamic task queue, the robot status table, and the workstation status table, the central control PLC makes scheduling decisions to select the optimal robot to execute the welding task, which includes the following steps.

[0111] Next, traverse the robot state table. If the current state State_RB is idle, or the current state State_RB is moving or welding and the estimated completion time T_complete_RB of the current task is less than the current time plus the estimated moving time, then add the corresponding robot ID_RB to the list of available robots L_available.

[0112] Secondly, for the welding task object T_i at the head of the dynamic task queue, iterate through each robot RB_j in the available robot list L_available and calculate the comprehensive cost C_ij for assigning it to RB_j. The cost function is typically a weighted average of the travel time cost and the load balancing cost. C_ij = α × T_move(RB_j, ID_STA) + β × Load_RB_j; Where C_ij represents the estimated total cost of assigning welding task object T_i to robot RB_j; T_move(RB_j, ID_STA) represents the estimated movement time (in seconds) for robot RB_j to move from its current position P_current_RB to the workstation ID_STA corresponding to welding task object T_i; Load_RB_j represents the current load factor of robot RB_j, which is set to the number of tasks that robot RB_j has been assigned but not completed; α and β This represents the weighting coefficient, used to balance efficiency and load balancing. (Settings are needed.) α >> β That is, the shortest travel time should be given priority.

[0113] Step 408: Select the robot that minimizes the overall cost as the selected robot, and assign the welding task to the selected robot.

[0114] Specifically, firstly, based on the comprehensive cost C_ij, the robot RB_j that minimizes the comprehensive cost C_ij is selected as the selected robot RB_assign for the welding task object T_i.

[0115] RB_assign = argmin (C_ij), {RB_j ∈ L_available}; in, argmin This represents selecting the robot that minimizes the cost function value.

[0116] Next, the welding task object T_i is removed from the dynamic task queue, and its assignment status is updated to assigned. Simultaneously, the count of the current load factor Load_RB_assign for the selected robot RB_assign is incremented by 1.

[0117] Finally, based on the welding task object T_i and the selected robot RB_assign, the task assignment instruction is output in the format {welding task object T_i, selected robot RB_assign}. Simultaneously, the current state State_RB of robot RB_assign in the robot state table is updated to "moving".

[0118] For example, the decision-making mechanism of the central control PLC does not adopt a simple polling or fixed mapping rule. The introduction of the cost function makes each allocation of the scheduling decision a local optimal solution in the current global state. By adjusting the weights, it can adapt to different production strategies.

[0119] The AI ​​vision-guided dual-robot brazing collaborative control method provided in this application overcomes the problems of uneven robot workload, system-wide cycle time limitation due to fixed or unreasonable task allocation, and low resource utilization in dual-robot systems. This method dynamically allocates robots by creating a dynamic task queue, acquiring robot status information in real time, and calculating a weighted sum of movement time cost and current load cost as the comprehensive cost. It achieves load balancing between the two robots, maximizes overall system efficiency, and ensures high-cycle production requirements.

[0120] Figure 5 The diagram shown is a flowchart illustrating a dual-robot brazing collaborative control method based on AI vision guidance, provided in another exemplary embodiment of this application. Figure 1 This application extends from the embodiments shown. Figure 5 The illustrated embodiment will be described in detail below. Figure 5 The illustrated embodiments and Figure 1 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0121] like Figure 5 As shown in the embodiment of this application, the AI ​​vision-guided dual-robot brazing collaborative control method couples the welding process parameters with the robot motion sequence to generate coupled control commands, including the following steps.

[0122] Step 500: Discretize the robot motion sequence into multiple path points and assign a path progress value to each path point.

[0123] Specifically, trajectory A is discretized into M path points k. Each point contains preset path point coordinates P_k, tool posture Q_k, and path progress S_k.

[0124] The tool posture Q_k is determined by the unit normal vector N_target of the welding target point in combination with the welding process requirements: for straight welds, Q_k remains constant for all welding segments; for curved welds, Q_k varies with the path tangent and normal; the path progress S_k represents the normalized arc length parameter. For the entire welding trajectory from the starting position to the welding target point, let the total length be L_total. For any path point P_k on trajectory A, let the cumulative arc length from the starting position to P_k be L_k, then S_k = L_k / L_total.

[0125] It should be understood that the path progress S_k takes different values ​​for different segments depending on the different segments of trajectory A. In this embodiment, the approach segment is set to S_k∈[0, 0.6], the introduction segment to S_k∈(0.6, 1), the welding segment to S_k=1, and the withdrawal segment to S_k>1.

[0126] Step 502: Based on the path progress value corresponding to each path point, assign a corresponding heating power setting value and wire feeding speed setting value to each path point.

[0127] Specifically, based on the robot motion sequence Waypoint_k, the basic heating power P_base, and the wire feeding speed W_base, corresponding process parameters are assigned to each robot motion sequence Waypoint_k, including the following steps.

[0128] If the path progress S_k∈[0, 0.6], it indicates that the current path point k belongs to the approach segment of trajectory A, and the heating power P_set = 0% is set; If the path progress S_k∈(0.6, 1), it indicates that the current path point k belongs to the introduction segment of trajectory A, and the heating power P_set = 30% × P_base is set. If the path progress S_k=1, it indicates that the current path point k belongs to the welding segment of trajectory A, and the heating power P_set= 100% × P_base is set. If the path progress S_k>1, it indicates that the current path point k belongs to the pullback segment of trajectory A, and the heating power P_set= 20% × P_base is set. If the path progress S_k≠1, it indicates that the current path point k does not belong to the welding segment of trajectory A, and the wire feed speed W_set = 0 mm / s is set. If the path progress S_k=1, it indicates that the current path point k belongs to the welding segment of trajectory A. Set the wire feeding speed W_set= W_base mm / s.

[0129] Step 504: The heating power setting value and the wire feeding speed setting value are used as additional parameters to update the corresponding path points in the robot motion sequence to generate coupling control commands.

[0130] Specifically, the robot motion sequence Waypoint_k={t_k, P_k, Q_k, V_k, S_k, P_set, W_set} is updated based on the heating power P_set and wire feeding speed W_set at different path points.

[0131] Based on the updated robot motion sequence Waypoint_k, the central control PLC sends control commands to the actuators to complete the welding task.

[0132] The AI ​​vision-guided dual-robot brazing collaborative control method provided in this application overcomes process quality problems such as overheating, under-welding, and poor solder formation caused by the asynchronous or mismatched timing of robot motion and welding process parameters. This is achieved by discretizing the robot motion sequence into path points with path progress values ​​and synchronously allocating heating power and wire feeding speed setpoints according to the progress values ​​to generate coupled control commands. This realizes precise synchronization and collaborative control between the welding process and the robot motion trajectory, improving welding quality and process reliability.

[0133] Figure 6 The diagram shown is a flowchart illustrating a dual-robot brazing collaborative control method based on AI vision guidance, provided in another exemplary embodiment of this application. Figure 1 This application extends from the embodiments shown. Figure 6 The illustrated embodiment will be described in detail below. Figure 6 The illustrated embodiments and Figure 1 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0134] like Figure 6 As shown in the embodiment of this application, the AI ​​vision-guided dual-robot brazing collaborative control method generates a robot motion sequence based on the task allocation instructions and visual positioning results, including the following steps.

[0135] Step 600: Based on the three-dimensional coordinates and normal vectors in the visual positioning results, plan a trajectory from the robot's starting position to the welding target point.

[0136] Specifically, based on the task number and workstation ID_STA in the task allocation instruction, a preset set of process parameters is loaded from the process parameter database. This set includes the base heating power P_base, wire feed speed W_base, welding speed V_weld, and preheating time T_preheat. Based on the welding target points (P_target, N_target) and the preset starting pose (P_current, Q_current), a trajectory A from P_current to P_target is planned. Trajectory A is divided into multiple segments: Approaching phase: Rapidly move from the starting position P_current to the safe point P_safe directly above the welding target point; wherein, the Euclidean distance between the safe point P_safe and the welding target point P_target is controlled within the range of [2, 5cm]; Introduction: From the safe point P_safe at a lower speed v 0. Move along the unit normal vector N_target to the welding target point P_target; where the lower speedv 0 is less than 10% of the normal moving speed; Welding segment: Starting from the welding target point P_target, it moves along the welding joint path defined by P_target and N_target at a welding speed V_weld; Retreat segment: After welding is completed, it rises along the unit normal vector N_target and returns to the safe point P_safe.

[0137] Step 602: Discretize the trajectory into multiple path points, each path point including coordinates, attitude and path progress value.

[0138] Specifically, trajectory A is discretized into M path points k. Each point contains preset path point coordinates P_k, tool posture Q_k, and path progress S_k.

[0139] The tool posture Q_k is determined by the unit normal vector N_target of the welding target point in combination with the welding process requirements: for straight welds, Q_k remains constant for all welding segments; for curved welds, Q_k varies with the path tangent and normal; the path progress S_k represents the normalized arc length parameter. For the entire welding trajectory from the starting position to the welding target point, let the total length be L_total. For any path point P_k on trajectory A, let the cumulative arc length from the starting position to P_k be L_k, then S_k = L_k / L_total.

[0140] It should be understood that the path progress S_k takes different values ​​for different segments depending on the different segments of trajectory A. In this embodiment, the approach segment is set to S_k∈[0, 0.6], the introduction segment to S_k∈(0.6, 1), the welding segment to S_k=1, and the withdrawal segment to S_k>1.

[0141] Step 604: Calculate the instantaneous linear velocity at each path point based on the coordinates, attitude, and path progress value of the path points.

[0142] Specifically, based on the {P_k, Q_k, S_k} sequence of discrete path points, the total motion time T_total, and the timestamp t_k obtained by interpolation through the motion curve, the approximate instantaneous linear velocity V_k at each path point is calculated using the position difference and time difference between adjacent path points.

[0143] V_k ≈ (P_{k+1} - P_{k-1}) / (t_{k+1} - t_{k-1}); Step 606: The coordinates, attitude, path progress value and instantaneous linear velocity of the path points are combined to form a robot motion sequence.

[0144] Specifically, based on the timestamp t_k, path point coordinates P_k, tool posture Q_k, instantaneous linear velocity V_k, and path progress S_k, all calculated and planned parameters are used to form a robot motion sequence Waypoint_k={t_k, P_k, Q_k, V_k, S_k}, where k = {0, 1, 2, ..., M}, according to the path point k.

[0145] The AI ​​vision-guided dual-robot brazing collaborative control method provided in this application solves the problems of unstable robot motion trajectories and inaccurate speed control in the welding segment that affect welding quality. This is achieved by planning a trajectory encompassing multiple stages, including approach, introduction, welding, and retraction, based on visual positioning results, and discretizing it into a path point sequence containing coordinates, attitude, progress value, and instantaneous linear velocity. This results in the generation of smooth, continuous, and speed-controllable robot motion trajectories, ensuring a stable welding process and excellent welding quality.

[0146] Figure 7 The diagram shown is a flowchart illustrating a dual-robot brazing collaborative control method based on AI vision guidance, provided in another exemplary embodiment of this application. Figure 1 This application extends from the embodiments shown. Figure 7 The illustrated embodiment will be described in detail below. Figure 7 The illustrated embodiments and Figure 1 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0147] like Figure 7 As shown in the embodiments of this application, the AI ​​vision-guided dual-robot brazing collaborative control method, after generating the robot motion sequence, further includes: Step 700: The angles of each joint axis of the robot corresponding to each path point are obtained by inverse kinematics calculation.

[0148] Specifically, for the pose {P_k, Q_k} of each path point k, the angles θ_k of each joint axis when the robot reaches that path point k are calculated using inverse kinematics formulas. k , θ_2 k , ..., θ_n k ] T .

[0149] {P_k, Q_k} = forward_kinematics(θ_1 k , θ_2 k , ..., θ_n k ); Wherein, forward_kinematics is a known robot forward kinematics model; θ_n represents the angle of the nth joint axis. In this embodiment, the robot used is six-axis, and n=6 is set.

[0150] It should be understood that for a six-axis robot, it is necessary to establish nonlinear equations through inverse kinematics formulas and obtain the angles of each joint axis by using the Jacobian matrix inversion method.

[0151] Step 702: Based on the timestamps of each path point and the corresponding joint axis angles, generate joint position, joint velocity and joint acceleration through trajectory interpolation.

[0152] Specifically, firstly, based on the joint axis angles {θ_k} and the total motion time T_total, trajectory interpolation is performed individually for each joint of the robot, transforming the discrete joint axis angles into continuous and smooth joint positions θ_desired(t), joint velocities ω_j(t), and joint accelerations α_j(t) with respect to time t. This process includes the following steps.

[0153] Next, based on the total motion time T_total and the path progress S_k, a timestamp t_k is assigned to each path point k, such that t_k = T_total × S_k. The total motion time is determined by the required welding speed V_weld and the total path length.

[0154] Secondly, using (t_k, θ_k) as known points, we construct the interpolation function for each joint through cubic spline interpolation. That is, for the j-th joint, we construct the function θ_j(t) satisfying θ_j(t_k) = θ_j k .

[0155] Finally, taking the first derivative of the obtained continuous function θ_j(t), we get the joint velocity ω_j(t) = dθ_j(t) / dt; taking the second derivative, we get the joint acceleration α_j(t) = d 2 θ_j(t) / dt 2 .

[0156] Step 704: The joint position, joint velocity, and joint acceleration are combined to form a joint space control sequence.

[0157] Specifically, the joint position θ_desired(t), joint velocity ω_j(t), and joint acceleration α_j(t) are used to construct a joint space control sequence, which is used to control the posture of each joint of the robot at the path points in real time.

[0158] For example, kinematic solving and motion curve interpolation are essential processes for generating the underlying control commands—joint commands θ_desired(t). The final output robot motion sequence is used for coupling with welding process parameters and system monitoring, but it does not directly drive the robot joints. The two data commands are generated in parallel within the system, jointly completing the transformation from spatial path description to executable motion commands.

[0159] The AI ​​vision-guided dual-robot brazing collaborative control method provided in this application effectively solves the problem of trajectory tracking errors that may arise from the inability to directly control the movement of each robot joint in high-level motion planning. This method converts the path point sequence into joint axis angles through inverse kinematics and further generates joint position, velocity, and acceleration control sequences through trajectory interpolation. It achieves real-time, high-precision control of robot motion at the joint space level, ensuring that the robot's end effector can accurately and smoothly reproduce the planned trajectory.

[0160] Figure 8 The diagram shown is an exemplary embodiment of an AI vision-guided dual-robot brazing collaborative control system provided in this application. Figure 8 As shown in the embodiment of this application, the AI ​​vision-guided dual-robot brazing collaborative control system includes: a vision positioning module 800, a dynamic scheduling module 802, a trajectory and process planning module 804, and an execution module 806.

[0161] The visual positioning module 800 is used to acquire workpiece images of the target workstation, process the workpiece images, identify feature points on the workpiece images using an AI model, and output the three-dimensional coordinates and normal vectors of the feature points in the robot base coordinate system to form the visual positioning result. The dynamic scheduling module 802 is used to dynamically allocate a robot to the welding task of the target workstation based on the visual positioning result, the current state of each robot, and the state of each workstation, and generate a task allocation instruction. The trajectory and process planning module 804 is used to generate a robot motion sequence based on the task allocation instruction and the visual positioning result, and couple the welding process parameters with the robot motion sequence to generate a coupling control instruction. The welding process parameters include at least heating power and wire feeding speed. The execution module 806 is used to control the allocated robot to move to the target workstation to perform the welding operation based on the coupling control instruction.

[0162] It should be understood that the operation and functions of the relevant modules mentioned in the AI ​​vision-guided dual-robot brazing collaborative control system can be referenced above. Figures 1 to 7 The AI ​​vision-guided dual-robot brazing collaborative control method provided will not be elaborated upon here to avoid repetition.

[0163] In another embodiment, the system hardware configuration includes: a dual-station conveyor line, two lifting mechanisms, a vision module, two Nachi MZ12 robots, a high-frequency induction heating power supply, a wire feeder, a chiller unit, and a central control PLC. The dual-station conveyor line includes photoelectric sensors, trays, tooling plates, workstations, and blocking mechanisms. The vision module includes a top camera and a side camera. The dual-station conveyor line acts as a material carrier, continuously transporting the tooling plates loaded with workpieces to two preset workstations. The photoelectric sensors and blocking mechanisms ensure accurate stopping and positioning of the tooling plates within the workstations; the lifting mechanisms vertically lift and lock the tooling plates after they are in place; the vision module takes pictures of the welding points on the workpieces from both top and side angles, processes the images using AI vision software, and accurately identifies and calculates the coordinates and orientation of the welding points in three-dimensional space; the two Nachi MZ12 robots... The MZ12 robot adopts a side-mounted layout, receiving positioning information from the vision system and scheduling instructions from the central control system. It is responsible for carrying the high-frequency heating head to the welding point and performing welding operations according to a preset trajectory. The high-frequency induction heating power supply, wire feeder, and chiller unit constitute a complete welding process unit. The high-frequency induction heating power supply provides energy to the heating head installed at the end of the robot, enabling it to instantly generate high temperature to melt the solder. The wire feeder accurately delivers the welding wire to the welding point. The chiller unit continuously provides circulating cooling for the high-frequency heating head to prevent it from overheating and being damaged. The central control PLC is responsible for processing the input signals from all sensors, coordinating the timing and interlocking of the entire process, including conveying, lifting, vision triggering, robot scheduling, welding start and stop, etc., to ensure that the entire system operates safely and reliably automatically according to the predetermined rhythm.

[0164] In operation, the fixture plate carrying the workpiece flows into the target workstation via the conveyor line. Upon arrival, a photoelectric sensor triggers, a blocking mechanism precisely positions the workpiece, and a lifting mechanism actuates, stabilizing the fixture plate to the working height. Simultaneously, top and side vision cameras capture images, and the AI ​​vision system quickly identifies and calculates the 3D coordinates of the welding point. The control system, based on dynamic scheduling logic, allocates an idle robot to the current workstation. The selected robot receives the visual positioning data, moves to the welding point, and simultaneously, the high-frequency heating head begins preheating. The robot moves along the planned trajectory, and the wire feeder feeds wire synchronously. Brazing is completed under the high temperature generated by the heating head. After welding, the robot moves away, the lifting mechanism descends, and the fixture plate is returned to the conveyor line and flows out of the workstation, ready for the next workpiece. During this process, two robots are intelligently scheduled between two workstations, working closely with the conveying, vision, and welding subsystems to achieve high-cycle continuous production of less than 4.6 seconds per piece.

[0165] Figure 9 The diagram shown is a structural schematic of an electronic device provided in an exemplary embodiment of this application. Figure 9 As shown, the electronic device 90 includes one or more processors 901 and memory 902.

[0166] The processor 901 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions.

[0167] The memory 902 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 901 may execute the program instructions to implement the three-dimensional coordinates, synthesis costs, and / or other desired functions of the various embodiments of this application described above. Various contents such as image data blocks Image_Top and Image_Side may also be stored in the computer-readable storage medium.

[0168] In one example, the electronic device 90 may also include an input device 903 and an output device 904, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0169] The input device 903 may include, for example, a keyboard, a mouse, etc.

[0170] The output device 904 can output various information to the outside, including robot motion sequences such as Waypoint_k. The output device 904 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0171] Of course, for the sake of simplicity, Figure 9 Only some of the components of the electronic device 90 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 90 may include any other suitable components depending on the specific application.

[0172] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the AI ​​vision-guided dual-robot brazing collaborative control method according to various embodiments of this application described above.

[0173] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0174] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the AI ​​vision-guided dual-robot brazing collaborative control method according to various embodiments of this application described above.

[0175] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0176] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0177] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0178] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0179] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0180] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A dual-robot brazing collaborative control method based on AI vision guidance, characterized in that, Applied to production lines including a first station and a second station, including: Acquire workpiece images at the target workstation, including images captured by a top camera and images captured by a side camera; The workpiece image is processed, and an AI model is used to identify feature points on the workpiece image. The three-dimensional coordinates and normal vectors of the feature points in the robot base coordinate system are output to form a visual positioning result. Based on the visual positioning results, the current state of each robot, and the state of each workstation, a robot is dynamically allocated to the welding task of the target workstation based on the dynamic load balancing algorithm, and a task allocation instruction is generated. Based on the task allocation instructions and the visual positioning results, a robot motion sequence is generated, and welding process parameters are coupled with the robot motion sequence to generate a coupling control instruction; the welding process parameters include at least heating power and wire feeding speed. Based on the coupling control command, the assigned robot is controlled to move to the target workstation to perform welding operations.

2. The dual-robot brazing collaborative control method according to claim 1, characterized in that, The process of acquiring the workpiece image of the target workstation, processing the workpiece image, and using an AI model to identify feature points on the workpiece image includes: In response to the target workstation arrival signal, the top camera and the side camera set above the target workstation are synchronously triggered to take pictures, and the top image and the side image are obtained. The top and side images are preprocessed to obtain standardized image data; The standardized image data is input into a pre-trained AI model, and the AI ​​model outputs a heatmap corresponding to the feature points; The pixel coordinates of the feature points in the top image and the side image are determined based on the heat map; Based on the pixel coordinates and the intrinsic and extrinsic parameters of the top and side cameras, the three-dimensional coordinates of the feature point in the robot's base coordinate system are calculated.

3. The dual-robot brazing collaborative control method according to claim 2, characterized in that, The step of inputting the standardized image data into a pre-trained AI model, and the AI ​​model outputting a heatmap corresponding to the feature points, includes: The AI ​​model uses a convolutional neural network model. The standardized image data is input into the convolutional layer of the convolutional neural network model to obtain the output feature map of the convolutional layer; Based on the output feature map of the convolutional layer, a non-linear activation function is applied to obtain the activated feature values; Based on the activated feature values, the receptive field is expanded through the pooling layer of the convolutional neural network model to obtain the pooled feature values. The pooled feature values ​​are input into multiple alternating stacks of convolutional layers, activation functions, and pooling layers to obtain a high-order feature map. The welding points in the high-order feature map are set as feature points. For each feature point, the convolutional neural network model outputs the corresponding heatmap.

4. The dual-robot brazing collaborative control method according to any one of claims 1-3, characterized in that, The method of dynamically allocating a robot to the welding task at the target workstation based on a dynamic load balancing algorithm includes: Create a dynamic task queue containing the welding tasks; Obtain the status information of all robots, including at least the robot ID, current position, current status, and estimated task completion time; A list of available robots is determined based on the status information; For the welding task at the head of the dynamic task queue, traverse the list of available robots and calculate the comprehensive cost of assigning the welding task to each robot in the list; the comprehensive cost is a weighted sum of the travel time cost and the current load cost. The robot that minimizes the overall cost is selected as the chosen robot, and the welding task is assigned to the selected robot.

5. The dual-robot brazing collaborative control method according to any one of claims 1-3, characterized in that, The process of coupling welding process parameters with the robot motion sequence to generate coupled control commands includes: The robot motion sequence is discretized into multiple path points, and a path progress value is assigned to each path point; Based on the path progress value corresponding to each path point, assign a corresponding heating power setting value and wire feeding speed setting value to each path point; The heating power setting value and the wire feeding speed setting value are used as additional parameters to update the corresponding path points in the robot motion sequence to generate the coupling control command.

6. The dual-robot brazing collaborative control method according to any one of claims 1-3, characterized in that, The step of generating a robot motion sequence based on the task allocation instruction and the visual positioning result includes: Based on the three-dimensional coordinates and normal vectors in the visual positioning results, a trajectory is planned from the robot's starting position to the welding target point. The trajectory includes at least an approach segment, an introduction segment, a welding segment, and a retraction segment. The trajectory is discretized into multiple path points, each path point including coordinates, attitude and path progress value; Calculate the instantaneous linear velocity at each path point based on the coordinates, attitude, and path progress value of the path points; The coordinates, orientation, path progress value, and instantaneous linear velocity of the path points are used to form the robot motion sequence.

7. The dual-robot brazing collaborative control method according to claim 6, characterized in that, Also includes: The angles of each joint axis of the robot corresponding to each path point are obtained by inverse kinematics calculation. Based on the timestamps of each path point and the corresponding joint axis angles, joint position, joint velocity, and joint acceleration are generated through trajectory interpolation. The joint position, joint velocity, and joint acceleration constitute a joint space control sequence, which is used to control the posture of each joint of the robot at path points in real time.

8. A dual-robot brazing collaborative control system based on AI vision guidance, characterized in that, Applied to production lines including a first station and a second station, including: The visual positioning module is used to acquire workpiece images of the target workstation, process the workpiece images, use an AI model to identify feature points on the workpiece images, and output the three-dimensional coordinates and normal vectors of the feature points in the robot base coordinate system to form a visual positioning result. The dynamic scheduling module is used to dynamically allocate a robot to the welding task of the target workstation based on the visual positioning results, the current state of each robot and the state of each workstation, and to generate a task allocation instruction. The trajectory and process planning module is used to generate a robot motion sequence based on the task allocation instruction and the visual positioning result, and to couple the welding process parameters with the robot motion sequence to generate a coupling control instruction; the welding process parameters include at least heating power and wire feeding speed; The execution module is used to control the assigned robot to move to the target workstation to perform welding operations based on the coupled control commands.

9. An electronic device, characterized in that, include: Processor; and A memory that stores computer program instructions, which, when executed by a processor, cause the processor to perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores computer program instructions that, when executed by a processor, cause the processor to perform the steps of the method as described in any one of claims 1 to 7.

Images