Flexible ton bag lug positioning method based on neural network model

Abstract

This invention discloses a method for locating flexible ton bag lifting lugs based on a neural network model, relating to the field of computer systems based on specific computational models. It includes simultaneously acquiring and aligning color and depth images; inputting the CLAHE-enhanced color image into a neural network model, which introduces a depth-guided attention module into the backbone network to generate an attention weight map by fusing depth gradient information, and then performing element-wise multiplication with the color feature map to output the first 2D bounding box of the lifting lug; extracting depth pixel regions from the bounding box and converting them into point cloud data; denoising, calculating normal vectors and curvature of the point cloud, and using a clustering algorithm to remove bag body interference to obtain a second 2D bounding box; further analyzing image moments and depth gradients to generate grasping instructions for a robotic arm. This invention effectively solves the challenges of weak texture recognition, flexible deformation positioning, and real-time edge calculation against a background of the same color, significantly improving the robustness and efficiency of grasping.

Description

Technical Field

[0001] This invention relates to the field of computer systems based on specific computational models, specifically to a method for positioning flexible ton bag lifting lugs based on a neural network model. Background Technology

[0002] In the transportation of bulk materials such as fertilizers, feed, plastic granules, and building materials, flexible container bags (FIBCs, commonly known as "ton bags") have become the mainstream transshipment packaging form due to their strong loading capacity and high logistic adaptability. Currently, upstream processes in the ton bag production process, such as handling and palletizing, have largely achieved automation. However, in the terminal transfer and loading / unloading stages, especially the "hooking" process, most factories still rely on manual operation. The typical operating mode is as follows: after the ton bag is transported to the lifting point by a conveyor belt, the position of the lifting lugs is identified manually, and the hook is completed with the assistance of a vehicle or robotic arm. This "semi-automated" mode not only creates a disconnect between information flow and logistics in smart factories, but also becomes a bottleneck restricting the overall production efficiency. At the same time, manual hooking work is arduous, repetitive, and limited by blind spots in the vehicle's line of sight, leading to frequent safety accidents such as crushing and squeezing injuries. There is an urgent need to introduce machine vision technology to achieve "unmanned, fully automated hooking."

[0003] While machine vision has been widely used in the identification and grasping of rigid industrial workpieces (such as automotive parts and cardboard boxes), its direct application to the grasping of flexible mega-bags still faces significant technical challenges. Current mainstream solutions can be categorized into three types, all of which struggle to meet the demands of real-world operating conditions: (1) Recognition schemes based on traditional 2D vision utilize industrial cameras to acquire color images and locate targets through algorithms such as edge detection, template matching, or color segmentation. However, the lifting lugs and bag bodies of ton bags are usually made of the same material (such as white polypropylene woven fabric). In industrial settings, it is common to see scenarios where "white lifting lugs are located on white bag bodies" or even "white bag bodies are placed on white conveyor belts," resulting in severe "same-color visual camouflage." Under conditions of weak or no texture, the contrast between the lifting lugs and the background is extremely low, making it difficult for traditional 2D algorithms based on gradient or color features to extract effective contours, which can easily lead to missed or false detections.

[0004] (2) The positioning scheme based on traditional 3D geometric registration uses an RGB-D depth camera to acquire point cloud data and employs ICP (Iterative Closest Point) or CAD model matching algorithms to register the measured point cloud with a standard geometric model to estimate the pose. Such methods are usually based on the "rigid body assumption," that is, the shape of the target object is fixed. However, as a typical flexible object, the ton bag is affected by gravity compression, stacking state, and material flow during the transportation process, resulting in a highly random shape (such as collapse, wrinkles, and twisting). The lifting lugs may appear upright, collapsed, or even stuck to the bag body. Such nonlinear flexible deformation causes the standard geometric template to fail, making it difficult for the ICP algorithm to converge and unable to stably output the grasping pose.

[0005] (3) 3D point cloud solutions based on high-performance deep learning employ deep neural network models such as PointNet++ and GPD (Grasp PoseDetection) to perform feature learning and semantic segmentation on massive unordered point clouds, thereby improving the ability to grasp and detect complex objects. Although it is highly adaptable to irregular objects, the network structure is complex, the number of parameters is huge (millions or more), and the computational resources are consumed in a high manner. Due to cost and deployment environment constraints, industrial sites often use embedded industrial control computers (such as NVIDIA Jetson series) as edge computing nodes, which are difficult to support high-performance servers. Under the condition of limited computing power, such algorithms are difficult to achieve real-time processing of high-density point clouds, and the system response delay is significant, which cannot meet the high-speed operation requirements of industrial production lines.

[0006] In summary, existing visual positioning technologies cannot simultaneously address the three core challenges: weak texture recognition against a background of the same color, non-rigid deformation positioning of flexible objects, and real-time performance on low-computing-power edge devices. Therefore, there is an urgent need to develop a lightweight hook positioning method optimized for edge computing environments, integrating 2D image texture and 3D depth information, to achieve highly robust and real-time fully automated hooking operations. Summary of the Invention

[0007] The purpose of this invention is to provide a flexible ton bag lifting lug positioning method based on a neural network model, which aims to overcome the above-mentioned problems existing in the prior art.

[0008] To achieve the objective, the present invention provides the following technical solution: A method for positioning flexible tonne bag lifting lugs based on a neural network model includes the following steps: Step S100: Simultaneously acquire color and depth images of the flexible ton bag using a camera; map the depth image onto the pixel coordinate system of the color image, so that the pixels in the color image correspond one-to-one with the depth values ​​in the depth image; Step S200: Perform contrast-limited adaptive histogram equalization on the color image to enhance the texture features of the lug edges; Step S300: Input the depth image and the color image processed by step S200 into the neural network model; In the backbone network feature extraction stage, the neural network model extracts the gradient information of the aligned depth map through the depth-guided attention module to generate a spatial attention weight map, and at the same time extracts the color feature map; The neural network model performs element-wise multiplication of the attention weight map and the color feature map, and outputs the first 2D bounding box of the lug in the image pixel coordinate system. Step S400: Based on the pixel coordinates of the first 2D bounding box, extract the corresponding first depth pixel region from the original depth image; using the camera intrinsic parameter matrix, convert the first depth pixel region into three-dimensional point cloud data; Step S500: Denoise the 3D point cloud data, and calculate the normal vector and curvature value of each point in the denoised 3D point cloud data; Step S600: Cluster the denoised 3D point cloud data using both normal vector difference and spatial distance as dual measures; after clustering, retain the point cloud cluster with the largest number of points that conforms to the characteristics of the hanging ear and has the curvature change as the target result, thereby eliminating the interference points caused by the planar bag body; based on the retained point cloud cluster, further extract the corresponding second depth pixel region from the first depth pixel region in step S400; based on the second depth pixel region, further select the corresponding second 2D bounding box from the inside of the first 2D bounding box.

[0009] Furthermore, in step S300 above, the formula for calculating the gradient magnitude is: ; in, This indicates that the aligned depth image is located at point... The depth value at that location; The formula for calculating the attention weight map is:

[0010]

[0011] in, Represents a depth feature map; σ It is the Sigmoid activation function, which normalizes the weights to (0, 1); Conv Indicates the convolution operation; This represents a feature map of a certain layer in the backbone network; This represents element-wise multiplication.

[0012] Furthermore, the aforementioned neural network model is based on YOLOv8, and the original C2f module is replaced with the Ghost module in the backbone network.

[0013] Furthermore, the specific steps of S500 described above are as follows: Input the above 3D point cloud data into the statistical outlier filter; set the number of neighborhood points and the standard deviation multiple, traverse the point cloud to calculate the average distance, and remove flying point noise that exceeds the statistical range; A KD-Tree index is established for the denoised 3D point cloud data, the covariance matrix of each point is calculated, and then the eigenvalues ​​are solved to obtain the normal vector and curvature value of the point.

[0014] Furthermore, two sets of RGB-D depth cameras are pre-configured to capture images of the flexible ton bag from different directions; in step S300 above, the neural network model also outputs the confidence score of the first 2D bounding box; Using one of the RGB-D depth cameras, perform steps S100 to S300. If the output confidence level is not lower than the preset threshold, continue with the subsequent steps. If the output confidence level is lower than the preset threshold, switch to another RGB-D depth camera and re-execute steps S100 to S300. If the output confidence level still does not reach the preset threshold, it is judged as an abnormal situation and manual intervention is required.

[0015] Furthermore, the curvature change described above conforms to the characteristics of a hanging ear, specifically: the curvature exhibits a significant jump in space; if there are a large number of curvature differences between neighboring points within a point cloud cluster that exceed a set threshold, then the curvature change of the point cloud cluster is considered significant.

[0016] Furthermore, it also includes the following steps: Step S700: Extract the center point pixel coordinates of the second 2D bounding box from step S600 above, and extract the corresponding depth value from the aligned depth image; back-project the center point pixel coordinates and depth value to three-dimensional space coordinates in the camera coordinate system, and use them as the gripping center point of the robotic arm to grip the lug. Step S800: Perform image moment analysis on the second depth pixel region to calculate the second central moment, thereby obtaining the principal axis deflection angle of the lug on the 2D image plane, which is used as the yaw angle for the robotic arm to grasp the lug; based on the depth gradient change of the second depth pixel region, calculate the normal vector of the fitting plane, which is used as the tilt angle of the lug relative to the camera. Step S900: Convert the pose matrix constructed from the grasping center point, yaw angle, and tilt angle in the camera coordinate system into the target pose matrix in the robot arm base coordinate system; parse the target pose and encapsulate it into grasping instructions that can be executed by the robot arm.

[0017] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a flexible ton bag lifting lug positioning method based on a neural network model as described above.

[0018] Compared with the prior art, the present invention has the following advantages: Firstly, this invention overcomes the challenge of visual camouflage in environments with weak textures and similar colors, significantly improving recognition robustness. The invention employs a depth-guided attention mechanism in its neural network model, utilizing height gradient information from the depth image as prior weights to perform weighted correction on the color feature map. Even when color features fail, the network can still sensitively detect minute height abrupt changes and geometric edges of the lugs relative to the bag surface, thus overcoming visual camouflage and achieving stable detection of flexible ton bag lugs under complex lighting and backgrounds of similar color.

[0019] Moreover, this invention overcomes the positioning failure problem caused by the non-rigid deformation of flexible objects, achieving high-precision grasping of flexible ton-bag lifting lugs. This invention abandons the traditional matching method based on rigid body models, employing a flexible positioning strategy combining 2D image moment features and depth mapping. By analyzing the 2D morphological features of the region of interest and combining them with depth space mapping, the dependence on the overall fixed shape of the object is decoupled. Regardless of the collapsed or twisted shape of the lifting lug, as long as it possesses a basic topological contour, the algorithm can accurately calculate its spatial grasping pose, significantly improving the grasping success rate.

[0020] Secondly, this invention addresses the computing power bottleneck on edge computing devices, enabling real-time, high-frequency operations on low-cost hardware. It employs a layered strategy of first performing 2D detection and locking, followed by local depth extraction, avoiding the blind processing of invalid background data across the entire scene. Furthermore, it introduces the Ghost module for lightweight reconstruction of the YOLOv8 backbone, significantly reducing the model's parameter count and computational load. This hardware-software co-optimization strategy allows the system to meet the high-frequency cycle requirements of industrial sites on low-power, low-cost edge industrial control computers, reducing enterprise hardware deployment costs.

[0021] Thirdly, this invention has good field adaptability and anti-interference ability. It can effectively filter out the noise from depth cameras caused by metal reflection or dust, which is common in industrial sites, and automatically remove residual bag background interference through the constraint of the 2D detection frame, ensuring the purity and accuracy of spatial coordinate calculation under harsh working conditions. Attached Figure Description

[0022] Figure 1 This is a flowchart of the flexible ton bag lifting lug positioning method of the present invention.

[0023] Figure 2 This is one of the network structure diagrams of the neural network model in this invention.

[0024] Figure 3 This is the second network structure diagram of the neural network model in this invention.

[0025] Figure 4 This is the third network structure diagram of the neural network model in this invention.

[0026] Figure 5 This is a top view of a device configuration based on the flexible ton bag lifting lug positioning method of the present invention.

[0027] Figure 6 The image shown is a physical representation of the detection results based on this invention. The blue box represents the second 2D bounding box and its confidence level (ear), while the green dot represents the capture center point of the second 2D bounding box and its coordinates. Detailed Implementation

[0028] Specific embodiments of the present invention will now be described with reference to the accompanying drawings. Many details are described below to provide a comprehensive understanding of the invention; however, those skilled in the art will be able to implement the invention without these details.

[0029] like Figures 1 to 4 As shown, a method for positioning flexible tonne bag lifting lugs based on a neural network model includes the following steps: Step S100: Simultaneously acquire color and depth images of the flexible ton bag using a camera; map the depth image onto the pixel coordinate system of the color image, so that the pixels in the color image correspond one-to-one with the depth values ​​in the depth image.

[0030] Specifically, the camera uses an industrial-grade RGB-D camera, whose hardware triggers the synchronous acquisition of color (RGB) and depth images of the flexible tonne bag, ensuring that the timestamps of the RGB frames and depth frames are consistent. Simultaneously, using the camera's factory-calibrated extrinsic parameter matrix, the depth image is mapped to the pixel coordinate system of the color image, ensuring that the pixels in the color image... Depth values ​​in the depth map One-to-one correspondence.

[0031] Step S200: Perform Limit Contrast Adaptive Histogram Equalization (CLAHE) on the color image to enhance the texture features of the lug edges.

[0032] Step S300: Input the depth image and the color image processed in step S200 into the neural network model, and output the first 2D bounding box of the hanger in the image pixel coordinate system. During the backbone network feature extraction stage, the neural network model extracts the aligned depth map gradient information through the depth-guided attention module to generate a spatial attention weight map, and simultaneously extracts the color feature map. The neural network model performs element-wise multiplication of the attention weight map and the color feature map, and outputs the first 2D bounding box of the hanger in the image pixel coordinate system. The first 2D bounding box represents the preliminary localization result of the hanger in the image pixel coordinate system.

[0033] like Figures 1 to 4 As shown, the neural network model is an improved YOLOv8 neural network model based on YOLOv8, and its network structure is as follows. Figures 2 to 4 As shown, the main improvements are as follows: Module Design: The original C2f module is replaced with the Ghost module in the backbone network of the neural network model. The Ghost module generates redundant feature maps through inexpensive linear operations, replacing the large number of parameter calculations in traditional convolutional layers. While maintaining the feature extraction capability essentially unchanged, the number of model parameters and floating-point operations are significantly reduced, thus ensuring that the system can meet the high-frequency cycle requirements of industrial production lines. In addition, a depth-guided attention module is designed in the feature extraction stage of the backbone network of the neural network model. It takes the aligned depth map gradient information as input and generates a spatial attention weight map.

[0034] Fusion logic: The spatial attention weight map and the color feature map are element-wise multiplied. In regions where the depth changes abruptly (i.e., where the lugs connect to the bag body, or where the lugs arch), the weight values ​​are higher; in flat regions (i.e., the bag body), the weight values ​​are lower.

[0035] Output: The network outputs the first 2D bounding box of the lug in the image pixel coordinate system. .in , These are the pixel coordinates of the preset reference vertex (such as the bottom right corner vertex) of the first 2D bounding box. The width of the first 2D bounding box. The height of the first 2D bounding box.

[0036] Furthermore, the formula for calculating the gradient magnitude is: ; in, This indicates that the aligned depth image is located at point... The depth value at that location; The formula for calculating the attention weight map is:

[0037]

[0038] in, Represents a depth feature map; σ It is the Sigmoid activation function, which normalizes the weights to (0, 1); Conv represents the convolution operation; This represents a feature map of a certain layer in the backbone network; This represents element-wise multiplication; (1+A) represents a residual connection concept that preserves the original features while enhancing key areas. It can also be written directly as A. A .

[0039] Step S400: Based on the pixel coordinates of the first 2D bounding box, for example, the first 2D bounding box... The first depth pixel region is extracted from the original depth image; using the camera intrinsic parameter matrix, the first depth pixel region is converted into 3D point cloud data. The extracted first depth pixel region is the region of interest (ROI) for subsequent steps, which can initially eliminate interference from irrelevant regions and reduce computational load.

[0040] Step S500: Denoise the 3D point cloud data, and calculate the normal vector and curvature value of each point in the denoised 3D point cloud data.

[0041] Specifically, the 3D point cloud data is input into a statistical outlier filter; the number of neighborhood points k and the standard deviation factor are set. The average distance of the point cloud is calculated by iterating through the points, and flying point noise that exceeds the statistical range is removed. Then, a KD-Tree index is built on the denoised 3D point cloud data, the covariance matrix of each point is calculated, and the eigenvalues ​​are solved to obtain the normal vector and curvature value of the point.

[0042] Step S600: First, cluster the denoised 3D point cloud data using both normal vector difference and spatial distance as dual measures.

[0043] Specifically, a normal vector difference threshold and a spatial distance threshold are set. A clustering algorithm is applied to the denoised 3D point cloud data, grouping two neighboring points into the same cluster only if both their normal vector difference and spatial distance are not greater than the normal vector difference threshold. The clustering algorithm includes, but is not limited to, the DBSCAN algorithm.

[0044] Then, after clustering, the point cloud cluster with the largest number of points that conforms to the characteristics of the hanging ear is retained as the target result, thereby eliminating the interference points caused by the planar bag body.

[0045] Specifically, the curvature changes described above that conform to the characteristics of a hanging lug include, but are not limited to: a significant spatial jump in curvature; and if a point cloud cluster contains a large number of neighboring points whose curvature differences exceed a set threshold (e.g., the curvature difference between neighboring points exceeds 0.1 and accounts for more than 30%), then the curvature change of the point cloud cluster is considered significant. In the hanging lug structure, non-planar areas such as the side walls and the inner walls of the hanging holes typically have large curvature changes, while the main body of the bag is a planar area with a curvature close to zero. Therefore, the curvature threshold can effectively distinguish between the two. It should be noted that the specific curvature threshold and determination method can be flexibly set according to the geometric characteristics of different hanging lugs.

[0046] Based on the retained point cloud clusters, the corresponding second depth pixel region is further extracted from the first depth pixel region in step S400; based on the second depth pixel region, the corresponding second 2D bounding box in the image pixel coordinate system is output, for example, the second 2D bounding box. ,Right now Figure 6 The blue box shown is the second 2D bounding box, which represents the precise positioning of the lug in the image pixel coordinate system.

[0047] To facilitate automatic grasping by a robotic arm, the present invention requires further determination of the lifting lug's orientation. Following steps S100 to S600, the invention further includes the following steps: Step S700: Extract the center point pixel coordinates of the second 2D bounding box from step S600, and extract the depth value corresponding to the center point from the aligned depth image; back-project the center point pixel coordinates and depth value to three-dimensional space coordinates in the camera coordinate system, which serve as the gripping center point for the robotic arm to grasp the lifting lug, i.e. Figure 6 The green dot within the blue box shown.

[0048] Based on the pinhole camera imaging model, pixel coordinates Spatial coordinates in the camera coordinate system The mapping relationship is as follows:

[0049] in, The depth value retrieved from the depth map , Camera intrinsic parameter matrix:

[0050] In the formula, , Focal length , The coordinates of the main point.

[0051] Specifically, in step S700, the center point pixel coordinates of the second 2D bounding box from step S600 are extracted. Center point pixel coordinates ; and extract the depth value corresponding to the center point from the aligned depth image. ; coordinates Back projection mapping to the corresponding 3D spatial coordinates in the camera coordinate system ,like Figure 6 The coordinates of the green dots shown.

[0052] Step S800: Perform image moment analysis on the second depth pixel region to calculate the second-order central moment, thereby obtaining the principal axis deflection angle of the lifting lug on the 2D image plane, which serves as the yaw angle for the robotic arm to grasp the lifting lug; based on the depth gradient change of the second depth pixel region, calculate the normal vector of the fitting plane, which serves as the tilt angle of the lifting lug relative to the camera. For example, if a three-dimensional spatial coordinate system is constructed with the camera optical axis as the Z-axis and the ZY plane as the horizontal plane, then the aforementioned yaw angle is the angle of rotation around the Y-axis, and the aforementioned tilt angle is the angle of rotation around the X-axis.

[0053] Step S900: Convert the pose matrix constructed from the grasping center point, yaw angle, and tilt angle in the camera coordinate system into the target pose matrix in the robot arm base coordinate system. Subsequently, resolve the target pose into position and attitude, and encapsulate it into grasping instructions that can be executed by the robot arm.

[0054] It should be noted that, since the camera in this embodiment is fixed to the scene, the pose matrix in the camera coordinate system is multiplied by the hand-eye calibration matrix to convert it into the target pose matrix in the robot arm base coordinate system. This conversion is existing technology and will not be described in detail here.

[0055] like Figures 1 to 5 As shown, in one specific embodiment, a conveyor belt 10 laterally transports flexible ton bags 20, causing one or more flexible ton bags 20 to remain at the end of the conveyor belt. At least one RGB-D depth camera is installed on the side of the end of the conveyor belt to photograph the flexible ton bags and locate their lifting lugs. A robotic arm 30 (including but not limited to the suspended sliding rail type robotic arm shown in the attached figure), connected to the process and positioned at the end of the conveyor belt, grasps the lifting lugs based on their positioning information and transfers the flexible ton bag 20 to the next workstation (such as a transport vehicle). Both the conveyor belt 10 and the robotic arm 30 are existing technologies, and their specific structures will not be described in detail here.

[0056] like Figures 1 to 6 As shown, preferably, two sets of RGB-D depth cameras are pre-configured to photograph the flexible ton bag from different directions. For example, one set of RGB-D depth cameras is set on the transverse side of the conveyor belt (e.g., Figure 5 (camera1 in the image), while another RGB-D depth camera (such as...) is set on the longitudinal side at the end of the conveyor belt. Figure 5(camera2 in the image). At this point, in step S300, the neural network model outputs the first 2D bounding box (e.g., camera2 in the image). Figure 6 (The blue box in the image), and also output the confidence score of the first 2D bounding box (e.g., ...). Figure 6 (the value of 'ear' in the text).

[0057] like Figures 1 to 5 As shown, steps S100 to S300 are first executed using one set of RGB-D depth cameras to obtain the first 2D bounding box and its confidence level. If the output confidence level is not lower than a preset threshold (e.g., 0.85, which can be set based on historical statistics or experiments), the subsequent steps are continued. If the output confidence level is lower than the preset threshold, steps S100 to S300 are re-executed using another set of RGB-D depth cameras. If the output confidence level still does not reach the preset threshold, it is judged as an abnormal situation and manual intervention is required.

[0058] It should be noted that the calculation and output of the confidence level mentioned above are existing technologies in this field. The specific implementation can be set by those skilled in the art according to the actual application scenario and detection accuracy requirements, and will not be elaborated here.

[0059] The present invention also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described flexible ton bag lifting lug positioning method based on an improved neural network model.

[0060] The above are merely specific embodiments of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial modifications made to the present invention using this concept shall be considered as infringing upon the protection scope of the present invention.

Claims

1. A method for positioning flexible tonne bag lifting lugs based on a neural network model, characterized in that, Includes the following steps: Step S100: Simultaneously acquire color and depth images of the flexible ton bag using a camera; Map the depth image to the pixel coordinate system of the color image so that the pixels in the color image correspond one-to-one with the depth values ​​in the depth image. Step S200: Perform contrast-limited adaptive histogram equalization on the color image to enhance the texture features of the lug edges; Step S300: Input the depth image and the color image processed in step S200 into the neural network model; In the backbone network feature extraction stage, the neural network model extracts the gradient information of the aligned depth map through the depth-guided attention module to generate a spatial attention weight map, and at the same time extracts the color feature map; The neural network model performs element-wise multiplication of the attention weight map and the color feature map, and outputs the first 2D bounding box of the lug in the image pixel coordinate system. Step S400: Extract the corresponding first depth pixel region from the original depth image based on the pixel coordinates of the first 2D bounding box; Using the camera intrinsic parameter matrix, the first depth pixel region is converted into three-dimensional point cloud data; Step S500: Denoise the 3D point cloud data, and calculate the normal vector and curvature value of each point in the denoised 3D point cloud data; Step S600: Cluster the denoised 3D point cloud data using both normal vector difference and spatial distance as dual metrics; After clustering, the point cloud cluster with the largest number of points that conforms to the characteristics of the hanging ear is retained as the target result, thereby eliminating the interference points caused by the planar bag body; based on the retained point cloud cluster, the corresponding second depth pixel region is further extracted from the first depth pixel region in step S400; based on the second depth pixel region, the corresponding second 2D bounding box is further selected from the inside of the first 2D bounding box.

2. The flexible ton bag lifting lug positioning method based on a neural network model according to claim 1, characterized in that, In step S300, the formula for calculating the gradient magnitude is: ； in, This indicates that the aligned depth image is located at point... The depth value at that location; The formula for calculating the attention weight map is: in, Represents a depth feature map; σ It is the Sigmoid activation function, which normalizes the weights to (0, 1); Conv represents the convolution operation; This represents a feature map of a certain layer in the backbone network; This represents element-wise multiplication.

3. A flexible ton bag lifting lug positioning method based on a neural network model according to claim 1 or 2, characterized in that, The neural network model is based on YOLOv8, and the original C2f module is replaced by the Ghost module in the backbone network.

4. The flexible ton bag lifting lug positioning method based on a neural network model according to claim 1, characterized in that, The specific steps S500 are as follows: The three-dimensional point cloud data is input into a statistical outlier filter; the number of neighborhood points and the standard deviation multiple are set, the average distance of the point cloud is calculated, and flying point noise that exceeds the statistical range is removed; A KD-Tree index is established for the denoised 3D point cloud data, the covariance matrix of each point is calculated, and then the eigenvalues ​​are solved to obtain the normal vector and curvature value of the point.

5. The flexible ton bag lifting lug positioning method based on a neural network model according to claim 1, characterized in that, Two RGB-D depth cameras are pre-configured to capture images of the flexible ton bag from different directions; in step S300, the neural network model also outputs the confidence score of the first 2D bounding box. Using one set of RGB-D depth cameras, execute steps S100 to S300. If the output confidence level is not lower than a preset threshold, continue to execute the subsequent steps. If the output confidence level is lower than the preset threshold, switch to another set of RGB-D depth cameras and re-execute steps S100 to S300. If the output confidence level still does not reach the preset threshold, it is judged as an abnormal situation and manual intervention is required.

6. The flexible ton bag lifting lug positioning method based on a neural network model according to claim 1, characterized in that, The curvature change conforms to the hanging ear characteristic specifically: the curvature exhibits a significant jump in space; if there are a large number of curvature differences between neighboring points within a point cloud cluster that exceed a set threshold, then the curvature change of the point cloud cluster is considered significant.

7. A flexible tonne bag lifting lug positioning method based on a neural network model according to any one of claims 1-6, characterized in that, It also includes the following steps: Step S700: Extract the center point pixel coordinates of the second 2D bounding box in step S600, and extract the corresponding depth value in the aligned depth image; back-project the center point pixel coordinates and depth value to three-dimensional space coordinates in the camera coordinate system, which are used as the gripping center point of the robotic arm to grip the lug. Step S800: Perform image moment analysis on the second depth pixel region, calculate the second-order central moment, and obtain the principal axis deflection angle of the lug on the 2D image plane, which is used as the yaw angle for the robotic arm to grasp the lug; based on the depth gradient change of the second depth pixel region, calculate the normal vector of the fitting plane, which is used as the tilt angle of the lug relative to the camera. Step S900: Convert the pose matrix constructed from the grasping center point, yaw angle and tilt angle in the camera coordinate system into the target pose matrix in the robot arm base coordinate system; The target pose is parsed and encapsulated into grasping instructions that can be executed by the robotic arm.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements a flexible ton bag lifting lug positioning method based on a neural network model as described in any one of claims 1 to 7.

Images