Intelligent identification and grabbing method and device for curling handle

Abstract

The application discloses an intelligent recognition and grabbing method and device for a curling ball handle, and belongs to the technical field of image and point cloud processing. The method comprises the following steps: a two-dimensional detection data set of a target handle is established to train a target detection network to determine a detection frame of the target handle; a depth map corresponding to the position of the detection frame is extracted, a binary depth map is obtained by filtering and thresholding the depth map by using a Gaussian filter, noise points in the binary depth map are removed, and then the binary depth map is converted into a point cloud map to obtain a three-dimensional point cloud of the target handle; the three-dimensional point cloud is down-sampled, and then three-dimensional cylinder fitting is performed to obtain the radius of the fitted cylinder and the central axis direction of the cylinder; the extraction and grabbing direction, the grabbing center position and the finger opening size of a grabbing device are determined according to the radius of the fitted cylinder and the central axis direction of the cylinder, and are sent to the grabbing device to complete grabbing. The method can realize accurate grabbing of the target handle by the robot, and has high real-time performance, good stability and accurate grabbing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image and point cloud processing, in particular to a method and device for intelligent recognition and grabbing of a curling ball handle based on fusion of depth and color image information. BACKGROUND

[0002] A curling ball is an important component of curling, and curling athletes need to throw the curling ball after grabbing it to gain an advantage in the final battle result with variable throwing strategies. In an intelligent curling robot, a mechanical arm needs to complete the grabbing and throwing of the handle of the curling ball. Intelligent recognition and grabbing of the curling ball require accurate extraction of the grabbing position of the handle.

[0003] There are few intelligent recognition and grabbing methods for the handle of the target, and the algorithm needs to be able to grab the handle of various shapes. The target containing the handle, such as the curling ball, has an irregular shape, and it is difficult to achieve the recognition of the posture of the handle and the extraction of the grabbing position on the handle in a short time by processing the color image.

[0004] In addition, grabbing is only one link in the completion of the task by the robot in the intelligent system of the robot, and for the intelligent curling robot system, the robot needs to complete the throwing of the curling ball. The first thing to be completed is the intelligent recognition and grabbing of the handle of the curling ball. Therefore, how to recognize the handle of the curling ball and extract the grabbing position plays an important role in the grabbing of the handle of the target.

[0005] Therefore, there is an urgent need for an intelligent recognition and grabbing method for the handle of the curling ball, which is high in real-time, good in stability, and accurate in grabbing. SUMMARY

[0006] The present application aims to at least partially solve one of the problems in the related art.

[0007] To this end, the first object of the present application is to propose an intelligent recognition and grabbing method for the handle of the curling ball, which can achieve accurate grabbing of the handle of the target by the robot and is high in real-time, good in stability, and accurate in grabbing.

[0008] The second object of the present application is to propose an intelligent recognition and grabbing device for the handle of the curling ball.

[0009] The third object of the present application is to propose a computer device.

[0010] The fourth object of the present application is to propose a non-transitory computer readable storage medium.

[0011] To achieve the above object, the application provides an intelligent recognition and grabbing method for a curling handle in an embodiment of the first aspect of the application, which comprises the following steps: step S1, establishing a two-dimensional detection data set of a target handle to train a target detection network to determine a detection frame of the target handle; step S2, extracting a depth map at a position corresponding to the detection frame, filtering and thresholding the depth map by using a Gaussian filter to obtain a binary depth map; step S3, removing noise points in the binary depth map and then converting the binary depth map into a point cloud map to obtain a three-dimensional point cloud of the target handle; step S4, performing down-sampling on the three-dimensional point cloud and then performing three-dimensional cylindrical fitting to obtain a radius of a fitted cylinder and a central axis direction of the cylinder; and step S5, determining a grabbing direction, a grabbing center position and a finger opening size of a grabbing device according to the radius of the fitted cylinder and the central axis direction of the cylinder, and sending the grabbing direction, the grabbing center position and the finger opening size to the grabbing device to complete grabbing.

[0012] The intelligent recognition and grabbing method for the curling handle according to the embodiment of the application uses color and depth information of a target to obtain a position and a direction of a target handle, extracts a grabbing position of the target handle, calculates an angle and a position at which a robot should move, and realizes accurate grabbing of the target handle by the robot, thereby overcoming the shortcomings of the prior intelligent recognition and grabbing method for the target handle.

[0013] In addition, the intelligent recognition and grabbing method for the curling handle according to the above embodiment of the application can further have the following additional technical features.

[0014] Further, in an embodiment of the application, the step S1 specifically comprises the following steps: step S101, collecting a color map of a target handle, labeling a position information of the target handle with a rectangular frame, and recording position information of each map to establish the two-dimensional detection data set; step S102, training a target detection network by using a back propagation algorithm and the two-dimensional detection data set, and updating parameters of the neural network; and step S103, selecting a detection result with the highest confidence in a current image as a detection frame of the target handle by using the trained neural network.

[0015] Further, in an embodiment of the application, the step S2 specifically comprises the following steps: step S201, extracting a depth value of a depth image corresponding to a pixel point in the target detection frame to form the depth map; step S202, performing hole filling on the depth map, and filtering the depth map after the hole filling by using a Gaussian operator to remove noise points; and step S203, selecting a threshold value according to a target characteristic to binarize the depth image after the noise points are removed, to obtain the binary depth map.

[0016] Further, in an embodiment of the application, the binary depth map is:

[0017]

[0018] img_proj = (img_guass-minVal) / (maxVal-minVal)*255

[0019] Wherein, img_threshold[i][j] is the binary depth map, img_proj[i][j] is the depth map mapped image, threshold is the threshold function, img_guass is the depth map after Gaussian filtering, minVal is the minimum value of depth, maxVal is the maximum value of depth.

[0020] Further, in an embodiment of the present application, the step S3 specifically comprises: step S301, removing the noise points in the binary image by erosion and dilation; step S302, converting the pixels with pixel value of 0 in the binary image after removing the noise points into the three-dimensional point cloud of the target handle through the camera parameter.

[0021] Further, in an embodiment of the present application, the step S4 specifically comprises: step S401, downsampling the three-dimensional point cloud; step S402, performing cylindrical three-dimensional fitting on the downsampled three-dimensional point cloud by using the RANSAC algorithm, to obtain the radius r of the fitted cylinder and the central axis direction cr of the cylinder.

[0022] Further, in an embodiment of the present application, the step S5 specifically comprises: step S501, keeping the normal vector of the end of the grabbing device and the background surface perpendicular, rotating the end joint of the grabbing device so that the finger opening and closing direction is perpendicular to the central axis direction of the cylinder, to determine the grabbing direction; step S502, taking the sum of the diameter of the cylinder and the preset allowance as the finger opening size; step S503, taking the center O of the downsampled three-dimensional point cloud as the grabbing center position, and rotating the end joint of the grabbing device so that the finger opening and closing direction is perpendicular to the central axis direction of the cylinder, to determine the grabbing direction. x y z

[0023]

[0024]

[0025]

[0026] Wherein, P is the point cloud after downsampling in step S401, p ix is the x-axis coordinate of the i-th point in the point cloud set, p iy is the y-axis coordinate of the i-th point in the point cloud set, p iz is the z-axis coordinate of the i-th point in the point cloud set, and n is the number of points in the point cloud set.

[0027] ​​​Step S504, the grabbing direction, the grabbing center position and the finger opening size are transmitted to the grabbing device to complete grabbing of the target handle by the grabbing completion robot arm.

[0028] To achieve the above object, the second aspect embodiment of the present application provides an intelligent recognition and grabbing device for curling handle, comprising: a determination detection frame module, which is used to establish a two-dimensional detection data set of a target handle to train a target detection network to determine a detection frame of the target handle; a binary module, which is used to extract a depth map at a position corresponding to the detection frame, and obtain a binary depth map by filtering and thresholding the depth map using a Gaussian filter; a three-dimensional point cloud module, which is used to remove noise points in the binary depth map and then convert the binary depth map into a point cloud map to obtain a three-dimensional point cloud of the target handle; a three-dimensional cylinder fitting module, which is used to down-sample the three-dimensional point cloud and then perform three-dimensional cylinder fitting to obtain a radius of the fitted cylinder and a central axis direction of the cylinder; and a grabbing module, which is used to determine a grabbing direction, a grabbing center position and a finger opening size of a grabbing device according to the radius of the fitted cylinder and the central axis direction of the cylinder, and transmit the grabbing direction, the grabbing center position and the finger opening size to the grabbing device to complete grabbing.

[0029] The third aspect embodiment of the present application provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the intelligent recognition and grabbing method for curling handle as described in the above embodiments.

[0030] The fourth aspect embodiment of the present application provides a non-temporary computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the intelligent recognition and grabbing method for curling handle as described in the above embodiments.

[0031] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0032] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0033] Figure 1 Flow chart of the intelligent recognition and grabbing method for curling handle according to one embodiment of the present application;

[0034] Figure 2 Specific execution schematic diagram of the intelligent recognition and grabbing method for curling handle according to one embodiment of the present application;

[0035] Figure 3 Target detection result schematic diagram of a target handle according to one embodiment of the present application;

[0036] Figure 4 Point cloud diagram for the extracted target handle according to an embodiment of the present application;

[0037] Figure 5 Diagram for the fitting result of a three-dimensional cylinder according to an embodiment of the present application;

[0038] Figure 6 Effect diagram for the target handle when being grabbed according to an embodiment of the present application;

[0039] Figure 7 Structural diagram of the intelligent recognition and grabbing device for the curling handle according to an embodiment of the present application. DETAILED DESCRIPTION

[0040] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0041] The intelligent recognition and grabbing method and device for the curling handle according to an embodiment of the present application are described below with reference to the accompanying drawings. First, the intelligent recognition and grabbing method for the curling handle according to an embodiment of the present application is described with reference to the accompanying drawings.

[0042] Figure 1 is a flowchart of the intelligent recognition and grabbing method for the curling handle according to an embodiment of the present application.

[0043] As shown in Figure 1 and 2 , the method comprises the following steps:

[0044] In step S1, a two-dimensional detection data set of the target handle is established by acquiring a video stream to train a target detection network to determine a detection frame of the target handle.

[0045] Further, in an embodiment of the present application, step S1 specifically comprises:

[0046] Step S101, acquiring a video stream, collecting color images of the target handle in the video stream, labeling the position information of the target handle with a rectangular frame, and recording the position information of each image to establish a two-dimensional detection data set;

[0047] Step S102, training the target detection network using the back propagation algorithm and the two-dimensional detection data set, and updating the parameters of the neural network;

[0048] Step S103, using the trained neural network to select the detection result with the highest confidence in the current image as the detection frame of the target handle.

[0049] Specifically, the camera is installed such that the optical axis of the camera is perpendicular to the target background, a video stream is acquired, a depth map and a color map are aligned, a color map is collected, a rectangular frame is labeled to indicate the position of the target handle, and the position information of the rectangular frame in each image is recorded to complete the establishment of a two-dimensional detection data set of the target handle, a target detection network is trained using a back propagation algorithm and the two-dimensional detection data set, and the parameters of the neural network are updated, and finally, the trained neural network is used for inference, and the detection result with the highest confidence in the current image of the camera is selected as the detection frame of the target handle.

[0050] In step S2, the depth map corresponding to the position of the detection frame is extracted, and the depth map is filtered and thresholded using a Gaussian filter to obtain a binary depth map.

[0051] Further, in an embodiment of the present application, step S2 specifically comprises:

[0052] Step S201, extracting the depth values of the depth image corresponding to the pixel points in the target detection frame to form a depth map;

[0053] Step S202, hole filling is performed on the depth map, and a Gaussian operator is used to filter the hole-filled depth map to remove noise points;

[0054] Step S203, according to the target characteristics, a threshold is selected to binarize the depth image after removing noise points to obtain a binary depth map.

[0055] Specifically, according to the target detection frame, the depth values of the depth image corresponding to the pixel points in the detection frame are extracted to form a depth map img_depth.

[0056] The depth map is hole-filled and filtered using a Gaussian operator to remove noise points in the depth map, wherein the filtered image is img_guass=guass(hole_filling(image_depth)), wherein image_depth is the depth map acquired by the camera, hole_filling is a hole filling algorithm, and guass is a Gaussian operator;

[0057] According to the target characteristics, a threshold is selected to binarize the image, specifically, the maximum value maxVal and the minimum value minVal of the depth map img_guass are obtained;

[0058] Secondly, the depth value is projected to [0, max], and the projection method is:

[0059] img_proj = (img_guass-minVal) / (maxVal-minVal)*255

[0060] The stable depth map is obtained by averaging the pixel values of the collected multiple images, and the binary depth map img_threshold = threshold(img_proj) is obtained, wherein threshold is a threshold function, and the specific process is as follows:

[0061]

[0062] Wherein, img_threshold[i][j] is a binary depth map, img_proj[i][j] is a depth map mapped image, threshold is a threshold function, img_guass is a depth map filtered by a Gaussian filter, minVal is a minimum depth value, and maxVal is a maximum depth value.

[0063] In step S3, the noise points in the binary depth map are removed, and then the binary depth map is converted into a point cloud map to obtain a three-dimensional point cloud of the target handle.

[0064] Further, in an embodiment of the present application, step S3 specifically includes:

[0065] Step S301, removing noise points in the binary image by using erosion and expansion;

[0066] Step S302, converting the pixels with a pixel value of 0 in the binary image after removing the noise points into a three-dimensional point cloud P = {p1, p2,...p n} of the target handle by using camera parameters.

[0067] In step S4, the three-dimensional point cloud is down-sampled, and then three-dimensional cylindrical fitting is performed to obtain the radius of the fitted cylinder and the central axis direction of the cylinder.

[0068] Further, in an embodiment of the present application, step S4 specifically includes:

[0069] Step S401, down-sampling the three-dimensional point cloud P to obtain a three-dimensional point cloud P' of the target handle, thereby reducing the number of subsequent processing;

[0070] Step S402, performing three-dimensional cylindrical fitting on the down-sampled three-dimensional point cloud P' by using a RANSAC algorithm to obtain the radius r of the fitted cylinder, the central axis direction cr of the cylinder, and a point M on the central axis.

[0071] In step S5, the grasping direction, the grasping center position and the finger opening size are determined according to the radius of the fitted cylinder and the direction of the central axis of the cylinder, and are sent to the grasping device to complete the grasping.

[0072] Further, in an embodiment of the present application, step S5 specifically comprises:

[0073] Step S501, keep the end of the grasping device perpendicular to the normal vector of the background surface, and rotate the end joint of the grasping device so that the finger opening direction is perpendicular to the central axis direction of the cylinder to determine the grasping direction;

[0074] Step S502, take the sum of the diameter of the cylinder and a preset allowance as the finger opening size;

[0075] Step S503, take the center O of the sampled three-dimensional point cloud as the grasping center position, x y z

[0076]

[0077]

[0078]

[0079] Wherein, P is the down-sampled point cloud in step S401, p ix is the x-axis coordinate of the i-th point in the point cloud set, p iy is the y-axis coordinate of the i-th point in the point cloud set, p iz is the z-axis coordinate of the i-th point in the point cloud set, and n is the number of points in the point cloud set.

[0080] Step S504, the grasping direction, the grasping center position and the finger opening size are transmitted to the grasping device to complete the grasping of the target handle by the mechanical arm.

[0081] The following will take the handle grasping of a land curling as an example to further illustrate the embodiments of the present application in combination with the drawings.

[0082] As shown in Figure 2 , first, the color image collected is subjected to target detection, and the target detection result of the target handle is as shown in Figure 3 , then the depth image is processed, and the point cloud of the target handle extracted by using the camera parameters is as shown in Figure 4 , RANSAC is used to fit the point cloud to a cylinder, and the fitting result is Figure 5 ​​​As shown, the point cloud after the cylinder fitting is shown as the white part in the figure, and the cylinder fitting result is the part other than white, and the parameters of the cylinder obtained are shown in Table 1. The effect of applying the method to the actual ice rink for intelligent recognition and grabbing of the curling handle is shown in Figure 6 .

[0083] Table 1 Parameters of the cylinder obtained by point cloud fitting

[0084]

[0085] In summary, according to the intelligent recognition and grabbing method of the curling ball handle proposed in the embodiment of the application, the color image and depth image information of the target are used to obtain the position and direction of the target handle, extract the grabbing position of the target handle, calculate the angle and position that the robot should move, and realize accurate grabbing of the target handle by the robot. Compared with the existing grabbing method, the method has higher real-time performance, better stability, and more accurate grabbing, and can be applied to various target handle grabbing application scenarios.

[0086] In order to realize the above-mentioned embodiment, the application further proposes an intelligent recognition and grabbing device for the curling ball handle according to the embodiment of the application, which is described below with reference to the accompanying drawings.

[0087] Figure 7 FIG. 1 is a structural schematic diagram of an intelligent recognition and grabbing device for a curling ball handle according to an embodiment of the application.

[0088] As shown in Figure 7 , the device 10 comprises a detection frame determination module 100, a binarization module 200, a three-dimensional point cloud module 300, a three-dimensional cylinder fitting module 400, and a grabbing module 500.

[0089] The detection frame determination module 100 is configured to establish a two-dimensional detection data set of the target handle to train a target detection network to determine a detection frame of the target handle. The binarization module 200 is configured to extract a depth image at a position corresponding to the detection frame, filter the depth image using a Gaussian filter, and threshold the depth image to obtain a binarized depth image. The three-dimensional point cloud module 300 is configured to remove noise points in the binarized depth image and then convert the binarized depth image into a point cloud image to obtain a three-dimensional point cloud of the target handle. The three-dimensional cylinder fitting module 400 is configured to downsample the three-dimensional point cloud, perform three-dimensional cylinder fitting on the handle point cloud using a random sampling consensus algorithm, and obtain a radius of the fitted cylinder and a central axis direction of the cylinder. The grabbing module 500 is configured to determine a grabbing direction, a grabbing center position, and a finger opening size of the grabbing device according to the radius of the fitted cylinder and the central axis direction of the cylinder, and send the grabbing direction, the grabbing center position, and the finger opening size to the grabbing device to complete grabbing.

[0090] It should be noted that the foregoing explanation and description of the intelligent recognition and grabbing method for the curling ball handle also apply to the device of this embodiment, which will not be described here again.

[0091] The intelligent recognition and grabbing device of the curling ball handle provided by the embodiment of the application uses color image and depth image information of a target to obtain the position and direction of the handle of the target, extracts the grabbing position of the handle of the target, calculates the angle and position at which the robot should move, and realizes accurate grabbing of the handle of the target by the robot, and has higher real-time performance, better stability and more accurate grabbing than the existing grabbing method.

[0092] In order to realize the above-mentioned embodiments, the application further provides a computer device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, and the processor realizes the intelligent recognition and grabbing method of the curling ball handle when executing the computer program.

[0093] In order to realize the above-mentioned embodiments, the application further provides a non-temporary computer readable storage medium, which stores a computer program, and the computer program realizes the intelligent recognition and grabbing method of the curling ball handle when being executed by a processor.

[0094] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0095] In addition, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one feature. In the description of the application, the meaning of "N" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0096] Any processes or methods described in the flowcharts or otherwise described herein can be understood as representing a module, segment, or portion of code that includes one or more executable instructions for implementing the specified logical function(s) or process(es). The scope of a preferred embodiment of this application includes embodiments that can carry out a function in a different order, or use different arrangements of steps, or combine certain steps, or a different sequence of steps, or use different steps, or use different functions without departing from the essence of the embodiments disclosed herein. The description of a process or method should be understood as representing executable instructions that can be employed to cause a system, device, or other implement to carry out the associated functions.

[0097] Logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be embodied in computer-readable media, in which a process or method can be executed, using an instruction execution system, apparatus, or device, such as a computer-based system, processor, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a computer- readable storage medium or a computer-readable communication medium. The computer-readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires (electrical connections), a portable computer diskette (a magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber (an optical device), and a portable compact disc read-only memory (CDROM). Note that the computer-readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

[0098] It should be understood that aspects of the application can be implemented in hardware, software, firmware, or combinations thereof. In the above embodiments, the steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. As such, in hardware implementations, any of the following technologies can be used, or a combination thereof, to implement the hardware: discrete logic circuitry having logic gates for implementing logic functions upon data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and so forth.

[0099] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment method can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0100] In addition, each functional unit in each embodiment of the present application can be integrated into one processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0101] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.

Claims

1. A method for intelligently recognizing and grasping curling puck handles, characterized in that, Includes the following steps: Step S1: Obtain the video stream to build a two-dimensional detection dataset of the target handle, and train the target detection network to determine the detection box of the target handle; Step S2: Extract the depth map corresponding to the detection box position, and use Gaussian filtering to filter and threshold the depth map to obtain a binarized depth map; Step S3: Remove noise from the binarized depth map and convert it into a point cloud map to obtain the three-dimensional point cloud of the target handle. Step S4: Downsample the three-dimensional point cloud and then perform three-dimensional cylinder fitting to obtain the radius and central axis direction of the fitted cylinder. Step S4 specifically includes: Step S401: Downsample the three-dimensional point cloud; Step S402: Use the RANSAC algorithm to perform cylindrical 3D fitting on the downsampled 3D point cloud to obtain the radius r and the central axis direction cr of the fitted cylinder. Step S5: Determine the extraction gripping direction, gripping center position, and gripping device finger opening size based on the radius of the fitted cylinder and the central axis direction of the cylinder, and send them to the gripping device to complete the gripping.

2. The intelligent recognition and grasping method for curling puck handles according to claim 1, characterized in that, Step S1 specifically includes: Step S101: Acquire video stream, capture color images of the target handle in the video stream, mark the position information of the target handle with a rectangle, and record the position information of each image to establish the two-dimensional detection dataset. Step S102: Train the target detection network using the backpropagation algorithm and the two-dimensional detection dataset, and update the parameters of the neural network; Step S103: Use the trained neural network to select the detection result with the highest confidence in the current image as the detection box of the target handle.

3. The intelligent recognition and grasping method for curling puck handles according to claim 1, characterized in that, Step S2 specifically includes: Step S201: Extract the depth values ​​of the depth images corresponding to the pixels within the target detection box to form the depth map; Step S202: Hole filling is performed on the depth map, and the depth map after hole filling is filtered using a Gaussian operator to remove noise. Step S203: Select a threshold based on the target characteristics to binarize the noise-removed depth image to obtain the binarized depth map.

4. The intelligent recognition and grasping method for curling puck handles according to claim 3, characterized in that, The binarized depth map is: in, For binarized depth maps, The image after depth map mapping. For threshold function, This is the depth map after Gaussian filtering. For the minimum depth, This represents the maximum depth.

5. The intelligent recognition and grasping method for curling puck handles according to claim 1, characterized in that, Step S3 specifically includes: Step S301: Remove noise from the binarized depth map using erosion and dilation. Step S302: Convert the pixels with a value of 0 in the binary image after removing noise points into a 3D point cloud of the target handle using camera parameters.

6. The intelligent recognition and grasping method for curling puck handles according to claim 1, characterized in that, Step S5 specifically includes: Step S501: Keep the end of the gripping device perpendicular to the normal vector of the background surface, and rotate the end joint of the gripping device so that the opening and closing direction of the fingers is perpendicular to the central axis of the cylinder, so as to determine the gripping direction; Step S502: The sum of the cylinder diameter and the preset margin is used as the finger opening size; Step S503, using the center of the downsampled 3D point cloud The center position of the grasp is... in, The point cloud after downsampling in step S401, For the point cloud set points Axis coordinates For the point cloud set points Axis coordinates For the point cloud set points Axis coordinates The number of points in the point cloud set; Step S504: The gripping direction, the gripping center position, and the finger opening size are transmitted to the gripping device to complete the gripping of the target handle by the robotic arm.

7. An intelligent recognition and gripping device for curling puck handles, characterized in that, include: The detection box determination module is used to build a two-dimensional detection dataset of the target handle in order to train the target detection network to determine the detection boxes of the target handle. The binarization module is used to extract the depth map of the corresponding position of the detection box, and to filter and threshold the depth map using Gaussian filtering to obtain a binarized depth map. The 3D point cloud module is used to remove noise from the binarized depth map and then convert it into a point cloud map to obtain the 3D point cloud of the target handle. A 3D cylinder fitting module is used to downsample the 3D point cloud and then perform 3D cylinder fitting to obtain the radius and central axis direction of the fitted cylinder. Specifically, it includes downsampling the 3D point cloud and using the RANSAC algorithm to perform 3D cylinder fitting on the downsampled 3D point cloud to obtain the radius r and central axis direction cr of the fitted cylinder. The grasping module is used to determine the grasping direction, grasping center position, and grasping device finger opening size based on the radius of the fitted cylinder and the central axis direction of the cylinder, and send them to the grasping device to complete the grasping.

8. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the intelligent identification and grasping method for curling puck handles as described in any one of claims 1-6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the intelligent recognition and grasping method for the curling puck handle as described in any one of claims 1-6.

Images