Target identification and trajectory planning method and system for slag adding robot

Abstract

According to the target identification and trajectory planning method and system for the slag adding robot, by improving the YOLO11s model, the influence of shape and size factors on a regression result in bounding box regression is better processed, the bounding box extraction precision is improved, and the false drop rate and the omission rate of each category are effectively reduced; besides, by recognizing the positions of the crystallizer and the worker in the working environment image, the walking track of the slag adding robot is planned through visual guidance, so that the slag adding robot accurately captures the position of the crystallizer, and obstacle avoidance and worker mistaken entering alarm in the slag adding process are effectively achieved; in addition, comprehensive analysis is carried out from multiple characteristics in the pose track, and the energy consumption of the slag adding robot in the slag adding process is reduced with the purpose of optimal energy consumption.

Description

Technical Field

[0001] This invention relates to the field of industrial robot trajectory planning, and more specifically, to a method and system for target recognition and trajectory planning of a slag-adding robot. Background Technology

[0002] Currently, most enterprises use manual slag addition, which is highly arbitrary. While robotic automatic slag addition technology can meet the principles of frequent, small-volume, and uniform addition, it is difficult to guarantee in actual operation. Manual slag addition is constrained by subjective experience, making precise control difficult. Operators need to push protective slag into the crystallizer at irregular intervals, which can easily cause instantaneous agitation of the molten steel surface within the crystallizer, resulting in slag entrapment. Because the control of parameters such as the amount and rate of slag added in manual slag addition largely depends on the operator's experience, it is difficult to produce a stable liquid slag layer. Furthermore, manual slag addition cannot solve the problem of moisture reabsorption during storage, and the slag material easily generates dust during the addition process, resulting in waste and environmental pollution. Summary of the Invention

[0003] To address at least one of the aforementioned technical problems, the present invention aims to provide a method and system for target recognition and trajectory planning of a slag-adding robot, which improves the accuracy of bounding box extraction, effectively reduces the false detection rate and false negative rate of various categories, and also improves the accuracy and convenience of slag addition.

[0004] The first aspect of this invention provides a method for target recognition and trajectory planning of a slag-adding robot, comprising: During the movement of the slag-adding robot, images of the working environment along the current path are acquired in real time using a pre-set imaging device. The working environment image is preprocessed to obtain a preprocessed feature image; Extract bounding boxes from the preprocessed feature image; The bounding box is compared and analyzed with the first bounding box in the preset first bounding box set in turn to obtain the first similarity value set; If there is a first similarity value in the first similarity value set that is greater than or equal to the preset first similarity threshold, the slag-adding robot will stop moving and issue an alarm. If all the first similarity values ​​in the first similarity value set are less than the preset first similarity threshold, then the bounding box and the second bounding box in the preset second bounding box set are compared and analyzed in turn to obtain the second similarity value set. If all second similar values ​​in the second similarity value set are less than the preset second similarity threshold, the travel trajectory route is switched; if there is a second similar value in the second similarity value set that is greater than or equal to the preset second similarity threshold, slag is added to the crystallizer based on the preset slag addition command.

[0005] In this solution, the step of preprocessing the working environment image to obtain a preprocessed feature image specifically includes: Based on a preset algorithm, image feature points are extracted from the working environment image to obtain a working environment feature map; The working environment feature map is grouped according to the number of channels to obtain the feature map of each group; The feature map of each group is split into and The two parts are calculated separately for channel attention and spatial attention, resulting in a weighted product that incorporates both channel and spatial attention. and ; Weighted by channel attention and spatial attention and The images are then stitched together to obtain the processed feature map.

[0006] In this scheme, after extracting the bounding boxes from the preprocessed feature image, the bounding boxes are optimized using an improved SNDS-YOLO model, with the bounding box regression loss function being: ,in , N represents the number of predicted bounding boxes, and C1 and C2 represent the weight coefficients, ensuring that C1 + C2 = 1 (e.g., C1 = 0.8, C2 = 0.2). middle b and These are the absolute center coordinates of the bounding box and the preset element box in the feature image, respectively. , ; 'scale' is the scaling factor, and 'ww' and 'hh' represent the weighting coefficients in the horizontal and vertical directions, respectively. Their values ​​are related to the shape of the ground truth box. and These represent the x and y coordinates of the center point of the bounding box in the feature image, respectively. and represents the x and y coordinates of the center point of the preset element box, respectively; w and h represent the width and height of the bounding box in the feature image, respectively. and These represent the width and height of the preset element box, respectively.

[0007] In this scheme, the similarity value The formula is Where C represents a constant. , Indicates weight, ; , and These represent the length and width of the calculation box, respectively. and This indicates the length and width of the preset element box.

[0008] In this solution, the step of obtaining the slag-adding command specifically includes: Obtain the location coordinates of the slag-adding robot; The crystallizer position is extracted from the processed feature map and converted into three-dimensional spatial coordinates in the coordinate system of the slag-adding robot base to obtain a three-dimensional model of the crystallizer. Based on the motion optimization function of the slag-adding robot, the optimal slag-adding posture trajectory of the slag-adding robot is determined according to the position coordinates of the slag-adding robot and the three-dimensional model of the crystallizer. Based on a pre-defined dynamic model, the optimal slag-adding posture trajectory is converted into slag-adding instructions for the slag-adding robot.

[0009] In this scheme, the step of obtaining the optimal slag-adding pose trajectory specifically includes: Based on the three-dimensional model of the crystallizer, determine the slag addition and coverage point above the crystallizer; Based on the slag coverage point, determine the corresponding spray gun pose of the slag-adding robot, i.e., the slag-adding pose; After traversing all the points covered by slag, we obtain the set of slag poses; Starting from any slag-adding pose in the set of slag-adding poses, connect it with other slag-adding poses to construct a slag-adding pose trajectory; After traversing all the slag-adding poses, we obtain the set of slag-adding pose trajectories; Extract the features and feature values ​​from any slag-adding pose trajectory in the slag-adding pose trajectory set; Based on the motion optimization function of the slag-adding robot, the energy consumption assessment index is determined according to the features and feature values ​​in the slag-adding pose trajectory. The optimal slag-adding posture trajectory is defined as the one that minimizes the energy consumption assessment index. The features in the slag-adding pose trajectory include at least the length of the slag-adding pose trajectory, the energy consumption when completing the entire slag-adding pose, and the smoothness of the preceding and following slag-adding poses.

[0010] In this scheme, after adding slag to the crystallizer based on a preset slag addition command, the method further includes: Based on a pre-set vision sensor, an image of the crystallizer surface is acquired; Extract pixels from the crystallizer surface image; By comparing and analyzing the pixels in the crystallizer surface image, the pixel difference between adjacent positions is determined; If the pixel difference between adjacent positions is greater than the preset pixel difference threshold, an alert message will be triggered.

[0011] A second aspect of the present invention provides a target recognition and trajectory planning system for a slag-adding robot, comprising a memory and a processor. The memory stores a program for a target recognition and trajectory planning method for a slag-adding robot. When the processor executes the program for a target recognition and trajectory planning method for a slag-adding robot, it performs the following steps: During the movement of the slag-adding robot, images of the working environment along the current path are acquired in real time using a pre-set imaging device. The working environment image is preprocessed to obtain a preprocessed feature image; Extract bounding boxes from the preprocessed feature image; The bounding box is compared and analyzed with the first bounding box in the preset first bounding box set in turn to obtain the first similarity value set; If there is a first similarity value in the first similarity value set that is greater than or equal to the preset first similarity threshold, the slag-adding robot will stop moving and issue an alarm. If all the first similarity values ​​in the first similarity value set are less than the preset first similarity threshold, then the bounding box and the second bounding box in the preset second bounding box set are compared and analyzed in turn to obtain the second similarity value set. If all second similar values ​​in the second similarity value set are less than the preset second similarity threshold, the travel trajectory route is switched; if there is a second similar value in the second similarity value set that is greater than or equal to the preset second similarity threshold, slag is added to the crystallizer based on the preset slag addition command.

[0012] In this solution, the step of preprocessing the working environment image to obtain a preprocessed feature image specifically includes: Based on a preset algorithm, image feature points are extracted from the working environment image to obtain a working environment feature map; The working environment feature map is grouped according to the number of channels to obtain the feature map of each group; The feature map of each group is split into and The two parts are calculated separately for channel attention and spatial attention, resulting in a weighted product that incorporates both channel and spatial attention. and ; Weighted by channel attention and spatial attention and The images are then stitched together to obtain the processed feature map.

[0013] In this scheme, after extracting the bounding boxes from the preprocessed feature image, the bounding boxes are optimized using an improved SNDS-YOLO model, with the bounding box regression loss function being: ,in , N represents the number of predicted bounding boxes, and C1 and C2 represent the weight coefficients, ensuring that C1 + C2 = 1 (e.g., C1 = 0.8, C2 = 0.2). middle b and These are the absolute center coordinates of the bounding box and the preset element box in the feature image, respectively. , ; 'scale' is the scaling factor, and 'ww' and 'hh' represent the weighting coefficients in the horizontal and vertical directions, respectively. Their values ​​are related to the shape of the ground truth box. and These represent the x and y coordinates of the center point of the bounding box in the feature image, respectively. and represents the x and y coordinates of the center point of the preset element box, respectively; w and h represent the width and height of the bounding box in the feature image, respectively. and These represent the width and height of the preset element box, respectively. In this context, C represents a constant. , Indicates weight, ; , and These represent the length and width of the calculation box, respectively. and This indicates the length and width of the preset element box.

[0014] One or more technical solutions proposed in this application have at least the following technical effects: By improving the YOLO11s model, the influence of shape and size factors on the regression results in bounding box regression is better handled, improving the accuracy of bounding box extraction and effectively reducing the false detection rate and false negative rate of each category. In addition, by identifying the positions of crystallizers and workers in the working environment image, visual guidance is used to plan the slag-adding robot's walking trajectory, enabling the slag-adding robot to accurately capture the position of the crystallizer and effectively realize obstacle avoidance and personnel accident alarm during the slag-adding process. Furthermore, by comprehensively analyzing multiple features in the pose trajectory, with the goal of optimal energy consumption, the energy consumption of the slag-adding robot during the slag-adding process is reduced. Attached Figure Description

[0015] Figure 1 A flowchart of a target recognition and trajectory planning method for a slag-adding robot according to the present invention is shown; Figure 2 A comparison chart of the loss function before and after the improvement of this invention is shown; Figure 3 A comparison diagram of the confusion matrix of the present invention is shown; Figure 4 A comparison chart showing the changes in Precision, Recall, and PR curves for the YOLO11s model and the SNDS-YOLO model is presented. Figure 5 A block diagram of a target recognition and trajectory planning system for a slag-adding robot according to the present invention is shown. Detailed Implementation

[0016] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0017] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0018] Figure 1 A flowchart of a target recognition and trajectory planning method for a slag-adding robot according to the present invention is shown.

[0019] like Figure 1 As shown, this invention discloses a target recognition and trajectory planning method for a slag-adding robot, comprising: During the movement of the slag-adding robot, images of the working environment along the current path are acquired in real time using a pre-set imaging device. The working environment image is preprocessed to obtain a preprocessed feature image; Extract bounding boxes from the preprocessed feature image; The bounding box is compared and analyzed with the first bounding box in the preset first bounding box set in turn to obtain the first similarity value set; If there is a first similarity value in the first similarity value set that is greater than or equal to the preset first similarity threshold, the slag-adding robot will stop moving and issue an alarm. If all the first similarity values ​​in the first similarity value set are less than the preset first similarity threshold, then the bounding box and the second bounding box in the preset second bounding box set are compared and analyzed in turn to obtain the second similarity value set. If all second similar values ​​in the second similarity value set are less than the preset second similarity threshold, the travel trajectory route is switched; if there is a second similar value in the second similarity value set that is greater than or equal to the preset second similarity threshold, slag is added to the crystallizer based on the preset slag addition command.

[0020] According to embodiments of the present invention, uniformly adding protective slag into the crystallizer is crucial in continuous casting processes. However, traditional robots with fixed slag-addition paths cannot cope with dynamic changes such as adjustments to equipment layout, sudden appearances of other mobile devices or personnel, etc. Fixed paths are not only prone to collisions with obstacles, but also lead to slag-addition failure if the crystallizer is not accurately identified. Furthermore, safety issues are particularly prominent in human-machine collaborative scenarios. A multi-path exploration strategy breaks the rigidity of fixed paths. Pre-setting multiple paths increases system redundancy and coverage, while "random selection" is an effective deadlock prevention strategy. When the target (crystallizer) cannot be effectively identified on a certain path or encounters a blockage, the system does not stagnate but lays the foundation for subsequent "path switching" logic, greatly improving the system's robustness in unstructured environments.

[0021] It should be noted that the first similarity value set includes multiple first similarity values; the second similarity value set includes multiple second similarity values; the preset first bounding box set includes element boxes of workers in different states; the preset second bounding box set includes element boxes of crystallizers in different states; and the travel trajectory route.

[0022] According to an embodiment of the present invention, the step of preprocessing the working environment image to obtain a preprocessed feature image specifically includes: Based on a preset algorithm, image feature points are extracted from the working environment image to obtain a working environment feature map; The working environment feature map is grouped according to the number of channels to obtain the feature map of each group; The feature map of each group is split into and The two parts are calculated separately for channel attention and spatial attention, resulting in a weighted product that incorporates both channel and spatial attention. and ; Weighted by channel attention and spatial attention and The images are then stitched together to obtain the processed feature map.

[0023] It should be noted that the number of channels in the working environment feature map is grouped by G to obtain multiple sub-feature maps. This process is implemented by y.view(b * G, -1, h, w), which reshapes the feature map by grouping the number of channels by G; then, the feature map of each group is split into two parts ( and ), calculate channel attention and spatial attention separately, where the calculation steps for channel attention are as follows: for Global average pooling is used to obtain global information for each group. A linear transformation (through learned weights and biases) is then applied to this global information to calculate the importance of each channel. Channel attention is obtained using the sigmoid activation function and applied to... Above, adjust the weights of the channel features to obtain the channel attention-weighted features. The specific steps for calculating spatial attention are as follows: ... Spatial attention is computed using group normalization, adjusted using learned weights and biases, and then generated using a sigmoid activation function. This spatial attention is then applied to... Above, the spatial dimension features are adjusted to obtain spatial attention-weighted features. G refers to the number of groups in Group Convolution. y represents the feature map, b*G represents the batch size multiplied by the number of groups G, combined into a single dimension, h and w represent the height and width respectively; view represents a method in PyTorch.

[0024] According to an embodiment of the present invention, after extracting the bounding boxes from the preprocessed feature image, the bounding boxes are optimized using an improved SNDS-YOLO model, and the bounding box regression loss function is: ,in , N represents the number of predicted bounding boxes, and C1 and C2 represent the weight coefficients, ensuring that C1 + C2 = 1 (e.g., C1 = 0.8, C2 = 0.2). middle b and These are the absolute center coordinates of the bounding box and the preset element box in the feature image, respectively. , ; 'scale' is the scaling factor, and 'ww' and 'hh' represent the weighting coefficients in the horizontal and vertical directions, respectively. Their values ​​are related to the shape of the ground truth box. and These represent the x and y coordinates of the center point of the bounding box in the feature image, respectively. and represents the x and y coordinates of the center point of the preset element box, respectively; w and h represent the width and height of the bounding box in the feature image, respectively. and These represent the width and height of the preset element box, respectively. In this context, C represents a constant. , Indicates weight, ; , and These represent the length and width of the calculation box, respectively. and This represents the length and width of the preset element bounding box; after extracting the bounding boxes from the preprocessed feature image, the number of bounding boxes in the corresponding feature image is predicted to obtain the number of predicted boxes.

[0025] It's important to note that the CIOU loss function improves the accuracy and convergence speed of bounding box prediction by comprehensively considering factors such as overlapping areas, center point distance, and aspect ratio, but it neglects the influence of the bounding box's shape and size. To address this, the Shape-NWD loss function is introduced. Shape-NWD is a method for bounding box regression in object detection, an improvement on Shape-IoU and NWD (Normalized Wasserstein Distance). It primarily aims to better handle the impact of shape and scale factors on regression results, especially in small object detection tasks.

[0026] According to an embodiment of the present invention, the similarity value The formula is Where C represents a constant. , Indicates weight, ; , and These represent the length and width of the calculation box, respectively. and This indicates the length and width of the preset element box.

[0027] It should be noted that the bounding boxes and preset element boxes in the feature image are modeled as Gaussian distributions, and then the Wasserstein distance is used to measure the similarity between these two distributions. The advantage of this distance is that even if two boxes do not overlap at all, or have very little overlap, the similarity can still be measured. The similarity is set to... Its formula is: Where C represents a constant, where , , Indicates weight, ww and hh represent the weighting coefficients in the horizontal and vertical directions, respectively. and These represent the x and y coordinates of the center point of the calculation box, respectively. and These represent the x and y coordinates of the center point of the preset element box, respectively. and These represent the length and width of the calculation box, respectively. and This indicates the length and width of the preset element frame; the preset element frame includes the element frame of the worker in different states and the element frame of the crystallizer in different states.

[0028] According to an embodiment of the present invention, the step of obtaining the slag addition command specifically includes: Obtain the location coordinates of the slag-adding robot; The crystallizer position is extracted from the processed feature map and converted into three-dimensional spatial coordinates in the coordinate system of the slag-adding robot base to obtain a three-dimensional model of the crystallizer. Based on the motion optimization function of the slag-adding robot, the optimal slag-adding posture trajectory of the slag-adding robot is determined according to the position coordinates of the slag-adding robot and the three-dimensional model of the crystallizer. Based on a pre-defined dynamic model, the optimal slag-adding posture trajectory is converted into slag-adding instructions for the slag-adding robot.

[0029] It should be noted that the trajectory planning description of the slag-adding robot includes the robot's travel trajectory and its slag-adding posture trajectory. The robot's travel trajectory is the path it takes to move to the crystallizer, and its posture trajectory is the trajectory it takes when adjusting the position of the robotic arm or robot after it has stopped at the crystallizer position and is controlling the spray gun to add slag. The preset dynamic model stores a large number of instructions and codes, which can convert the robot's motion trajectory or slag-adding posture trajectory into corresponding trajectory codes, and then convert the corresponding trajectory codes into corresponding instructions.

[0030] According to an embodiment of the present invention, the step of obtaining the optimal slag-adding pose trajectory specifically includes: Based on the three-dimensional model of the crystallizer, determine the slag addition and coverage point above the crystallizer; Based on the slag coverage point, determine the corresponding spray gun pose of the slag-adding robot, i.e., the slag-adding pose; After traversing all the points covered by slag, we obtain the set of slag poses; Starting from any slag-adding pose in the set of slag-adding poses, connect it with other slag-adding poses to construct a slag-adding pose trajectory; After traversing all the slag-adding poses, we obtain the set of slag-adding pose trajectories; Extract the features and feature values ​​from any slag-adding pose trajectory in the slag-adding pose trajectory set; Based on the motion optimization function of the slag-adding robot, the energy consumption assessment index is determined according to the features and feature values ​​in the slag-adding pose trajectory. The optimal slag-adding posture trajectory is defined as the one that minimizes the energy consumption assessment index. The features in the slag-adding pose trajectory include at least the length of the slag-adding pose trajectory, the energy consumption when completing the entire slag-adding pose, and the smoothness of the preceding and following slag-adding poses.

[0031] It should be noted that the motion optimization function of the slag-grabbing robot includes the energy consumption assessment index calculation formula, which is as follows: ,in These represent the corresponding weight coefficients. These represent the feature values ​​corresponding to the slag-adding pose trajectory length, the energy consumption when completing the entire slag-adding pose, and the smoothness of the preceding and following slag-adding poses, respectively. The slag-adding pose trajectory length is the total displacement length of the spray gun. The energy consumption when completing the entire slag-adding pose is determined based on the torque of the robotic arm when completing each slag-adding pose, and its formula is as follows: ,in This is the torque vector of each robotic arm at time t; the smoothness of the preceding and following slag-adding poses is evaluated and determined by the jerk of the robotic arm during the transition process, and its formula is: ,in This represents the jerk of the robotic arm, measured in units of... The smaller the accelerometer, the smoother the movement of the robotic arm, reducing wear between robotic arms and thus reducing energy consumption.

[0032] Furthermore, during the spraying process, the speed of the robotic arm must not exceed the set speed threshold, the acceleration must not exceed the set acceleration threshold, the torque output by the robotic arm must not exceed the preset maximum torque threshold, and the joint angle of the corresponding robotic arm must be controlled within the set angle range.

[0033] According to an embodiment of the present invention, after adding slag to the crystallizer based on a preset slag addition command, the method further includes: Based on a pre-set vision sensor, an image of the crystallizer surface is acquired; Extract pixels from the crystallizer surface image; By comparing and analyzing the pixels in the crystallizer surface image, the pixel difference between adjacent positions is determined; If the pixel difference between adjacent positions is greater than the preset pixel difference threshold, an alert message will be triggered.

[0034] It should be noted that the step of comparing and analyzing pixels in the crystallizer surface image to determine the pixel difference between adjacent positions specifically involves: extracting any one pixel; using this pixel as a reference point, determining the positions of the other eight pixels in a period; calculating the difference between the pixel value of the reference point and the pixel values ​​of the other eight pixel positions sequentially; and taking the absolute value to obtain an initial set of pixel differences. The average of these initial pixel differences is then calculated to obtain the pixel difference between adjacent positions. The uniformity of the protective slag thickness on the crystallizer surface is judged by the pixel difference between adjacent positions. If the pixel difference between adjacent positions is greater than a preset pixel difference threshold, it indicates that the thickness of the protective slag at that pixel position differs significantly from the thickness of the surrounding protective slag, thus triggering a warning message.

[0035] Figure 2 A comparison chart of the loss function before and after the improvement of this invention is shown.

[0036] like Figure 2 As shown in the figure, after a set number of training iterations, the loss function before and after the improvement is compared as follows: Figure 2 As shown, it is clear that the improved model has a lower loss value, indicating that by introducing the Shape-NWD loss function, the influence of shape and scale factors on the regression results in bounding box regression is better handled, achieving the desired effect.

[0037] Figure 3 A comparison diagram of the confusion matrix of the present invention is shown.

[0038] like Figure 3 As shown, the confusion matrices of YOLO11s (a) and SNDS-YOLO (b) are presented. Compared to YOLO11s, the confusion matrix of SNDS-YOLO typically has higher values ​​in the diagonal region. Furthermore, the values ​​for the background category are generally lower. This indicates that SNDS-YOLO can effectively reduce the false positive and false negative rates for each category.

[0039] Figure 4 A comparison chart showing the changes in Precision, Recall, and PR curves for the YOLO11s model and the SNDS-YOLO model is presented.

[0040] like Figure 4 As shown, a1, a2, and a3 are the curves showing the changes in Precision, Recall, and PR of the YOLO11s model, while b1, b2, and b3 are the curves showing the changes in Precision, Recall, and PR of the SNDS-YOLO model. It can be seen that the SNDS-YOLO model has better detection accuracy.

[0041] Figure 5 A block diagram of a target recognition and trajectory planning system for a slag-adding robot according to the present invention is shown.

[0042] like Figure 5 As shown, a second aspect of the present invention provides a target recognition and trajectory planning system 5 for a slag-adding robot, comprising a memory 51 and a processor 52. The memory stores a program for a target recognition and trajectory planning method for a slag-adding robot. When the processor executes the program for a target recognition and trajectory planning method for a slag-adding robot, it performs the following steps: During the movement of the slag-adding robot, images of the working environment along the current path are acquired in real time using a pre-set imaging device. The working environment image is preprocessed to obtain a preprocessed feature image; Extract bounding boxes from the preprocessed feature image; The bounding box is compared and analyzed with the first bounding box in the preset first bounding box set in turn to obtain the first similarity value set; If there is a first similarity value in the first similarity value set that is greater than or equal to the preset first similarity threshold, the slag-adding robot will stop moving and issue an alarm. If all the first similarity values ​​in the first similarity value set are less than the preset first similarity threshold, then the bounding box and the second bounding box in the preset second bounding box set are compared and analyzed in turn to obtain the second similarity value set. If all second similar values ​​in the second similarity value set are less than the preset second similarity threshold, the travel trajectory route is switched; if there is a second similar value in the second similarity value set that is greater than or equal to the preset second similarity threshold, slag is added to the crystallizer based on the preset slag addition command.

[0043] In this solution, the step of preprocessing the working environment image to obtain a preprocessed feature image specifically includes: Based on a preset algorithm, image feature points are extracted from the working environment image to obtain a working environment feature map; The working environment feature map is grouped according to the number of channels to obtain the feature map of each group; The feature map of each group is split into and The two parts are calculated separately for channel attention and spatial attention, resulting in a weighted product that incorporates both channel and spatial attention. and ; Weighted by channel attention and spatial attention and The images are then stitched together to obtain the processed feature map.

[0044] In this scheme, after extracting the bounding boxes from the preprocessed feature image, the bounding boxes are optimized using an improved SNDS-YOLO model, with the bounding box regression loss function being: ,in , N represents the number of predicted bounding boxes, and C1 and C2 represent the weight coefficients, ensuring that C1 + C2 = 1 (e.g., C1 = 0.8, C2 = 0.2). middle b and These are the absolute center coordinates of the bounding box and the preset element box in the feature image, respectively. , ; 'scale' is the scaling factor, and 'ww' and 'hh' represent the weighting coefficients in the horizontal and vertical directions, respectively. Their values ​​are related to the shape of the ground truth box. and These represent the x and y coordinates of the center point of the bounding box in the feature image, respectively. and represents the x and y coordinates of the center point of the preset element box, respectively; w and h represent the width and height of the bounding box in the feature image, respectively. and These represent the width and height of the preset element box, respectively. In this context, C represents a constant. , Indicates weight, ; , and These represent the length and width of the calculation box, respectively. and This indicates the length and width of the preset element box.

[0045] This invention discloses a target recognition and trajectory planning method and system for a slag-adding robot. By improving the YOLO11s model, it better handles the influence of shape and size factors on the regression results in bounding box regression, thereby improving the accuracy of bounding box extraction and effectively reducing the false detection rate and false negative rate of various categories. In addition, by identifying the positions of crystallizers and workers in the working environment image, it realizes visual guidance to plan the walking trajectory of the slag-adding robot, enabling the slag-adding robot to accurately capture the position of the crystallizer, effectively realizing obstacle avoidance and personnel accident alarm during the slag-adding process. Furthermore, by comprehensively analyzing multiple features in the pose trajectory, with the aim of optimizing energy consumption, it reduces the energy consumption of the slag-adding robot during the slag-adding process.

[0046] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0047] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0048] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0049] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0050] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method for target recognition and trajectory planning of a slag-adding robot, characterized in that, include: During the movement of the slag-adding robot, images of the working environment along the current path are acquired in real time using a preset imaging device; The working environment image is preprocessed to obtain a preprocessed feature image; Extract bounding boxes from the preprocessed feature image; The bounding box is compared and analyzed with the first bounding box in the preset first bounding box set in turn to obtain the first similarity value set; If there is a first similarity value in the first similarity value set that is greater than or equal to the preset first similarity threshold, the slag-adding robot will stop moving and issue an alarm. If all the first similarity values ​​in the first similarity value set are less than the preset first similarity threshold, then the bounding box and the second bounding box in the preset second bounding box set are compared and analyzed in turn to obtain the second similarity value set. If all the second similar values ​​in the second similarity value set are less than the preset second similarity threshold, then switch the travel trajectory route; If there is a second similarity value in the second similarity value set that is greater than or equal to the preset second similarity threshold, then slag is added to the crystallizer based on the preset slag addition command.

2. The target recognition and trajectory planning method for a slag-adding robot according to claim 1, characterized in that, The step of preprocessing the working environment image to obtain a preprocessed feature image specifically includes: Based on a preset algorithm, image feature points are extracted from the working environment image to obtain a working environment feature map; The working environment feature map is grouped according to the number of channels to obtain the feature map of each group; The feature map of each group is split into and The two parts are calculated separately for channel attention and spatial attention, resulting in a weighted product that incorporates both channel and spatial attention. and ; Weighted by channel attention and spatial attention and The images are then stitched together to obtain the processed feature map.

3. The target recognition and trajectory planning method for a slag-adding robot according to claim 1, characterized in that, After extracting the bounding boxes from the preprocessed feature image, the bounding boxes are optimized using an improved SNDS-YOLO model, with the bounding box regression loss function being: ,in , N represents the number of predicted bounding boxes, and C1 and C2 represent the weight coefficients, ensuring that C1 + C2 = 1 (e.g., C1 = 0.8, C2 = 0.2). middle b and These are the absolute center coordinates of the bounding box and the preset element box in the feature image, respectively. , ; 'scale' is the scaling factor, and 'ww' and 'hh' represent the weighting coefficients in the horizontal and vertical directions, respectively. Their values ​​are related to the shape of the ground truth box. and These represent the x and y coordinates of the center point of the bounding box in the feature image, respectively. and represents the x and y coordinates of the center point of the preset element box, respectively; w and h represent the width and height of the bounding box in the feature image, respectively. and These represent the width and height of the preset element box, respectively. In this context, C represents a constant. , Indicates weight, ; , and These represent the length and width of the calculation box, respectively. and This indicates the length and width of the preset element box.

4. The target recognition and trajectory planning method for a slag-adding robot according to claim 1, characterized in that, The similarity value The formula is Where C represents a constant. , Indicates weight, ; , and These represent the length and width of the calculation box, respectively. and This indicates the length and width of the preset element box.

5. The target recognition and trajectory planning method for a slag-adding robot according to claim 1, characterized in that, The steps for obtaining the slag-adding instruction specifically include: Obtain the location coordinates of the slag-adding robot; The crystallizer position is extracted from the processed feature map and converted into three-dimensional spatial coordinates in the coordinate system of the slag-adding robot base to obtain a three-dimensional model of the crystallizer. Based on the motion optimization function of the slag-adding robot, the optimal slag-adding posture trajectory of the slag-adding robot is determined according to the position coordinates of the slag-adding robot and the three-dimensional model of the crystallizer. Based on a pre-defined dynamic model, the optimal slag-adding posture trajectory is converted into slag-adding instructions for the slag-adding robot.

6. The target recognition and trajectory planning method for a slag-adding robot according to claim 5, characterized in that, The steps for obtaining the optimal slag-adding pose trajectory specifically include: Based on the three-dimensional model of the crystallizer, determine the slag addition and coverage point above the crystallizer; Based on the slag coverage point, determine the corresponding spray gun pose of the slag-adding robot, i.e., the slag-adding pose; After traversing all the points covered by slag, we obtain the set of slag poses; Starting from any slag-adding pose in the set of slag-adding poses, connect it with other slag-adding poses to construct a slag-adding pose trajectory; After traversing all the slag-adding poses, we obtain the set of slag-adding pose trajectories; Extract the features and feature values ​​from any slag-adding pose trajectory in the slag-adding pose trajectory set; Based on the motion optimization function of the slag-adding robot, the energy consumption assessment index is determined according to the features and feature values ​​in the slag-adding pose trajectory. The optimal slag-adding posture trajectory is defined as the one that minimizes the energy consumption assessment index. The features in the slag-adding pose trajectory include at least the length of the slag-adding pose trajectory, the energy consumption when completing the entire slag-adding pose, and the smoothness of the preceding and following slag-adding poses.

7. The target recognition and trajectory planning method for a slag-adding robot according to claim 1, characterized in that, After adding slag to the crystallizer based on a preset slag addition command, the process further includes: Based on a pre-set vision sensor, an image of the crystallizer surface is acquired; Extract pixels from the crystallizer surface image; By comparing and analyzing the pixels in the crystallizer surface image, the pixel difference between adjacent positions is determined; If the pixel difference between adjacent positions is greater than the preset pixel difference threshold, an alert message will be triggered.

8. A target recognition and trajectory planning system for a slag-adding robot, characterized in that, The system includes a memory and a processor. The memory stores a program for a target recognition and trajectory planning method for a slag-adding robot. When the processor executes the program, the method performs the following steps: During the movement of the slag-adding robot, images of the working environment along the current path are acquired in real time using a preset imaging device; The working environment image is preprocessed to obtain a preprocessed feature image; Extract bounding boxes from the preprocessed feature image; The bounding box is compared and analyzed with the first bounding box in the preset first bounding box set in turn to obtain the first similarity value set; If there is a first similarity value in the first similarity value set that is greater than or equal to the preset first similarity threshold, the slag-adding robot will stop moving and issue an alarm. If all the first similarity values ​​in the first similarity value set are less than the preset first similarity threshold, then the bounding box and the second bounding box in the preset second bounding box set are compared and analyzed in turn to obtain the second similarity value set. If all the second similar values ​​in the second similarity value set are less than the preset second similarity threshold, then switch the travel trajectory route; If there is a second similarity value in the second similarity value set that is greater than or equal to the preset second similarity threshold, then slag is added to the crystallizer based on the preset slag addition command.

9. The target recognition and trajectory planning system for a slag-adding robot according to claim 8, characterized in that, The step of preprocessing the working environment image to obtain a preprocessed feature image specifically includes: Based on a preset algorithm, image feature points are extracted from the working environment image to obtain a working environment feature map; The working environment feature map is grouped according to the number of channels to obtain the feature map of each group; The feature map of each group is split into and The two parts are calculated separately for channel attention and spatial attention, resulting in a weighted product that incorporates both channel and spatial attention. and ; Weighted by channel attention and spatial attention and The images are then stitched together to obtain the processed feature map.

10. The target recognition and trajectory planning system for a slag-adding robot according to claim 8, characterized in that, After extracting the bounding boxes from the preprocessed feature image, the bounding boxes are optimized using an improved SNDS-YOLO model, with the bounding box regression loss function being: ,in , N represents the number of predicted bounding boxes, and C1 and C2 represent the weight coefficients, ensuring that C1 + C2 = 1 (e.g., C1 = 0.8, C2 = 0.2). middle b and These are the absolute center coordinates of the bounding box and the preset element box in the feature image, respectively. , ; 'scale' is the scaling factor, and 'ww' and 'hh' represent the weighting coefficients in the horizontal and vertical directions, respectively. Their values ​​are related to the shape of the ground truth box. and These represent the x and y coordinates of the center point of the bounding box in the feature image, respectively. and represents the x and y coordinates of the center point of the preset element box, respectively; w and h represent the width and height of the bounding box in the feature image, respectively. and These represent the width and height of the preset element box, respectively. In this context, C represents a constant. , Indicates weight, ; , and These represent the length and width of the calculation box, respectively. and This indicates the length and width of the preset element box.

Images