Mechanical arm control method, device, equipment and storage medium

Abstract

The application relates to the fields of artificial intelligence and digital medicine, and provides a mechanical arm control method, device, equipment and storage medium. The method detects left and right images, obtains first and second elliptical boundaries, selects first and second clustering ellipses and performs matching, obtains target left and right ellipses, identifies an initial distance between a desktop cup and an electronic device according to a left center point of the target left ellipse and a right center point of the target right ellipse, identifies a cup diameter of the desktop cup according to first and second left long axis vertices of the target left ellipse and first and second right long axis vertices of the target right ellipse, generates a distance relationship between a mechanical arm and the desktop cup based on a positional relationship between the electronic device and the mechanical arm and the initial distance, and controls the mechanical arm to operate, and the application is suitable for a dynamic scene. In addition, the application also relates to a blockchain technology, and the distance relationship and the cup diameter can be stored in the blockchain.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and digital medical technology, and in particular to a robotic arm control method, device, equipment and storage medium. Background Technology

[0002] With the development of artificial intelligence, solutions for controlling robotic arms have also emerged. In the scenario of a robotic arm pouring tea, because the teacups are quite similar, and there are certain order and procedural requirements for pouring tea, specific sized cups are needed, and the cups must be placed in specific positions, making it unsuitable for dynamic scenarios. Furthermore, robotic arm control solutions can also be applied to the field of digital healthcare; for example, the movement of the robotic arm can be controlled to place medical devices. Summary of the Invention

[0003] In view of the above, it is necessary to provide a robotic arm control method, device, equipment and storage medium that can solve the technical problem that robotic arms cannot adapt to dynamic scenarios.

[0004] On one hand, this invention proposes a robotic arm control method applied to an electronic device equipped with a binocular camera, the electronic device communicating with a robotic arm, and the robotic arm control method comprising:

[0005] Acquire the left and right images of the tabletop cup captured by the binocular camera;

[0006] Perform coarse ellipse detection on the left image to obtain multiple first ellipse boundaries, and perform coarse ellipse detection on the right image to obtain multiple second ellipse boundaries;

[0007] A first clustering ellipse is selected from the plurality of first elliptical boundaries, and a second clustering ellipse is selected from the plurality of second elliptical boundaries;

[0008] The first cluster ellipse is matched with the second cluster ellipse to obtain the target left ellipse of the left image and the target right ellipse of the right image;

[0009] The initial distance between the desktop cup and the electronic device is identified based on the left center point of the target left ellipse and the right center point of the target right ellipse, and the cup diameter of the desktop cup is identified based on the first left major axis vertex of the target left ellipse, the second left major axis vertex of the target left ellipse, the first right major axis vertex of the target right ellipse, and the second right major axis vertex of the target right ellipse.

[0010] Based on the positional relationship between the electronic device and the robotic arm, and the initial distance, the distance relationship between the robotic arm and the tabletop cup is generated;

[0011] Based on the distance relationship and the diameter of the cup, the robotic arm is controlled to operate.

[0012] According to a preferred embodiment of the present invention, the step of performing ellipse coarse detection on the left image to obtain a plurality of first ellipse boundaries includes:

[0013] The left image is subjected to Gaussian convolution to obtain a filtered image;

[0014] Based on a preset horizontal matrix and a preset vertical matrix, the filtered image is subjected to image gradient calculation to obtain a gradient image;

[0015] The gradient of each gradient pixel in the gradient image is compared with its corresponding neighboring pixels to generate an edge boundary image;

[0016] Obtain the gradient intensity of each edge pixel in the edge boundary image;

[0017] Edge pixels with gradient strength greater than or equal to a first preset gradient threshold are identified as target edge pixels, and edge pixels with gradient strength less than the first preset gradient threshold but greater than or equal to a second preset gradient threshold are identified as edge pixels to be identified, wherein the first preset gradient threshold is greater than the second preset gradient threshold.

[0018] The plurality of first elliptical boundaries are generated based on the target edge pixels and the edge pixels to be identified.

[0019] According to a preferred embodiment of the present invention, performing Gaussian convolution processing on the left image to obtain a filtered image includes:

[0020] Count the number of pixels for the length of the left image, and count the number of pixels for the height of the left image;

[0021] If the number of length pixels is not equal to the number of height pixels, then the left image is cropped to obtain a cropped image;

[0022] A convolution matrix is ​​generated based on the number of horizontal pixels in the cropped image;

[0023] Based on the convolution matrix, the cropping pixel matrix of each cropping pixel in the cropped image is convolved to obtain the brightness value of each cropping pixel;

[0024] The filtered image is generated based on multiple brightness values.

[0025] According to a preferred embodiment of the present invention, the step of calculating the image gradient of the filtered image based on a preset horizontal matrix and a preset vertical matrix to obtain a gradient image includes:

[0026] Based on the preset horizontal matrix, the filtering pixel matrix of each filtering pixel in the filtered image is convolved to obtain the horizontal gradient value of each filtering pixel.

[0027] The filter pixel matrix is ​​convolved based on the preset vertical matrix to obtain the vertical gradient value of each filter pixel.

[0028] The gradient image is generated based on the horizontal gradient value and the vertical gradient value.

[0029] According to a preferred embodiment of the present invention, generating the plurality of first elliptical boundaries based on the target edge pixels and the edge pixels to be identified includes:

[0030] Based on the pixel positions of the target edge pixels and the edge pixels to be identified in the edge boundary image, connect the target edge pixels and the edge pixels to be identified to obtain a connecting arc;

[0031] Identify the target arc corresponding to the edge pixel to be identified from the connecting arc, wherein the arc length of the target arc is equal to a preset length;

[0032] If the target arc includes the target edge pixels, then the edge pixels to be identified are determined as identified edge pixels;

[0033] Connect the target edge pixels and the identified edge pixels to obtain multiple concave arcs and multiple convex arcs;

[0034] Multiple arcs to be processed are generated based on the multiple concave arcs and the multiple convex arcs;

[0035] Ellipse fitting is performed on each arc to be processed to obtain the plurality of first elliptical boundaries.

[0036] According to a preferred embodiment of the present invention, matching the first clustering ellipse with the second clustering ellipse to obtain the target left ellipse of the left image and the target right ellipse of the right image includes:

[0037] Obtain the left intrinsic parameter matrix of the left camera and the right intrinsic parameter matrix of the right camera in the stereo camera;

[0038] A left coordinate system is constructed based on the left image, and a right coordinate system is constructed based on the right image;

[0039] Obtain the extrinsic matrix and translation vector of the left coordinate system and the right coordinate system;

[0040] Obtain the first cluster coordinate information of the first cluster center of the first clustering ellipse in the left coordinate system, and obtain the second cluster coordinate information of the second cluster center of the second clustering ellipse in the right coordinate system;

[0041] Error values ​​are generated based on the left intrinsic parameter matrix, the right intrinsic parameter matrix, the extrinsic parameter matrix, the translation vector, the first clustering coordinate information, and the second clustering coordinate information;

[0042] If the error value is less than the preset value, then the first cluster ellipse is determined as the target left ellipse, and the second cluster ellipse is determined as the target right ellipse.

[0043] According to a preferred embodiment of the present invention, identifying the diameter of the tabletop cup based on the first left major axis vertex of the target left ellipse, the second left major axis vertex of the target left ellipse, the first right major axis vertex of the target right ellipse, and the second right major axis vertex of the target right ellipse includes:

[0044] Based on the extrinsic parameter matrix, the translation vector, the left intrinsic parameter matrix, and the first left major axis vertex coordinate information in the left coordinate system, the first target coordinate information of the first left major axis vertex in the right coordinate system is generated;

[0045] Based on the extrinsic parameter matrix, the translation vector, the left intrinsic parameter matrix, and the second left major axis vertex coordinate information in the left coordinate system, the second target coordinate information of the second left major axis vertex in the right coordinate system is generated;

[0046] The first vertex disparity between the first left major axis vertex and the first right major axis vertex is generated based on the first right major axis coordinate information in the right coordinate system and the first target coordinate information;

[0047] The second vertex disparity between the second left major axis vertex and the second right major axis vertex is generated based on the second right major axis coordinate information of the second right coordinate system and the second target coordinate information;

[0048] The first rim distance between the desktop cup and the electronic device is generated based on the focal length of the binocular camera, the optical center distance between the left camera and the right camera, and the first vertex parallax; and the second rim distance between the desktop cup and the electronic device is generated based on the focal length, the optical center distance, and the second vertex parallax.

[0049] The diameter of the cup is obtained by calculating the difference between the distance between the first and second cup rims.

[0050] On the other hand, the present invention also proposes a robotic arm control device that operates on an electronic device equipped with a binocular camera. The electronic device communicates with the robotic arm, and the robotic arm control device includes:

[0051] The acquisition unit is used to acquire the left and right images of the tabletop cup captured by the binocular camera;

[0052] The detection unit is used to perform coarse ellipse detection on the left image to obtain multiple first ellipse boundaries, and to perform coarse ellipse detection on the right image to obtain multiple second ellipse boundaries.

[0053] A selection unit is used to select a first clustering ellipse from the plurality of first elliptical boundaries and to select a second clustering ellipse from the plurality of second elliptical boundaries;

[0054] A matching unit is used to match the first clustered ellipse with the second clustered ellipse to obtain the target left ellipse of the left image and the target right ellipse of the right image;

[0055] The identification unit is used to identify the initial distance between the desktop cup and the electronic device based on the left center point of the target left ellipse and the right center point of the target right ellipse, and to identify the cup diameter of the desktop cup based on the first left major axis vertex of the target left ellipse, the second left major axis vertex of the target left ellipse, the first right major axis vertex of the target right ellipse, and the second right major axis vertex of the target right ellipse.

[0056] A generation unit is used to generate a distance relationship between the robotic arm and the tabletop cup based on the positional relationship between the electronic device and the robotic arm and the initial distance;

[0057] The control unit is used to control the operation of the robotic arm based on the distance relationship and the diameter of the cup.

[0058] On the other hand, the present invention also proposes an electronic device, the electronic device comprising:

[0059] Memory, which stores computer-readable instructions; and

[0060] The processor executes computer-readable instructions stored in the memory to implement the robotic arm control method.

[0061] On the other hand, the present invention also proposes a computer-readable storage medium storing computer-readable instructions, which are executed by a processor in an electronic device to implement the robotic arm control method.

[0062] As can be seen from the above technical solutions, this application uses ellipse coarse detection, which, due to its low computational load, can detect the position of the table cup in real time. This allows for flexible adjustment and control of the robotic arm's operation. Furthermore, by selecting the first and second clustered ellipses, errors caused by coarse detection can be avoided. By matching the first and second clustered ellipses, and combining the characteristics of the binocular camera, the accuracy of filtering the target left and right ellipses is improved, thereby increasing the accuracy of distance relationship generation. Thus, this application, by introducing visual perception, can solve the problem of the robotic arm failing to accurately pour tea due to the movement of the table cup. By accurately identifying the distance relationship between the robotic arm and the table cup, this application can adapt to dynamic scenes without requiring the table cup to be placed in a specific position. In addition, this application can accurately identify the diameter of the table cup using the first left major axis vertex, the second left major axis vertex, the first right major axis vertex, and the second right major axis vertex. Therefore, this application can use table cups of any size, improving its practicality and further enhancing its adaptability to dynamic scenes. Attached Figure Description

[0063] Figure 1 This is an application scenario diagram of the robotic arm control method of the present invention.

[0064] Figure 2 This is a flowchart of a preferred embodiment of the robotic arm control method of the present invention.

[0065] Figure 3 This is a functional block diagram of a preferred embodiment of the robotic arm control device of the present invention.

[0066] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the robotic arm control method of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0068] like Figure 1 The figure shown is an application scenario diagram of the robotic arm control method of the present invention.

[0069] The robotic arm control method is applied to electronic device 1, which is equipped with a binocular camera 11. The electronic device 1 communicates with the robotic arm 2. The electronic device 1 can call a program to control the robotic arm 2 to perform actions such as operation and grasping. The binocular camera 11 can capture images of a tabletop cup 3. The electronic device 1, the robotic arm 2, and the tabletop cup 3 are all on the same plane.

[0070] This application mounts the binocular camera 11 onto the electronic device 1, rather than onto the robotic arm 2, enabling the binocular camera 11 to capture a fixed-point image of the tabletop cup 3. This improves the clarity of the left and right images obtained by the binocular camera from the tabletop cup, thereby ensuring that the robotic arm 2 can operate accurately.

[0071] like Figure 2 The diagram shown is a flowchart of a preferred embodiment of the robotic arm control method of the present invention. The order of the steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements.

[0072] The robotic arm control method described above can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0073] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0074] The robotic arm control method is applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored computer-readable instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0075] The electronic device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0076] The electronic devices may include network devices and / or user devices. The network devices include, but are not limited to, single network electronic devices, groups of multiple network electronic devices, or cloud computing-based systems consisting of a large number of hosts or network electronic devices.

[0077] The network in which the electronic device is located includes, but is not limited to: the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0078] 101. Obtain the left and right images of the tabletop cup captured by the binocular camera.

[0079] In at least one embodiment of the present invention, the binocular camera is mounted in the electronic device. The desktop cup and the electronic device are on the same plane.

[0080] The left image refers to the image generated by the left camera in the binocular camera controlled by the electronic device, and the right image refers to the image generated by the right camera in the binocular camera controlled by the electronic device. In a medical setting, the tabletop cup can be a medical device, so the left and right images can also be images obtained by the binocular camera capturing images of the medical device.

[0081] In at least one embodiment of the present invention, the electronic device obtains a cup image corresponding to the desktop cup from a database corresponding to the binocular camera, and identifies the left image and the right image based on the image tags of the cup image.

[0082] 102. Perform coarse ellipse detection on the left image to obtain multiple first ellipse boundaries, and perform coarse ellipse detection on the right image to obtain multiple second ellipse boundaries.

[0083] In at least one embodiment of the present invention, the plurality of first elliptical boundaries refer to elliptical boundaries identified from the left image. The plurality of second elliptical boundaries refer to elliptical boundaries identified from the right image.

[0084] In at least one embodiment of the present invention, the electronic device performs coarse ellipse detection on the left image to obtain a plurality of first ellipse boundaries, including:

[0085] The left image is subjected to Gaussian convolution to obtain a filtered image;

[0086] Based on a preset horizontal matrix and a preset vertical matrix, the filtered image is subjected to image gradient calculation to obtain a gradient image;

[0087] The gradient of each gradient pixel in the gradient image is compared with its corresponding neighboring pixels to generate an edge boundary image;

[0088] Obtain the gradient intensity of each edge pixel in the edge boundary image;

[0089] Edge pixels with gradient strength greater than or equal to a first preset gradient threshold are identified as target edge pixels, and edge pixels with gradient strength less than the first preset gradient threshold but greater than or equal to a second preset gradient threshold are identified as edge pixels to be identified, wherein the first preset gradient threshold is greater than the second preset gradient threshold.

[0090] The plurality of first elliptical boundaries are generated based on the target edge pixels and the edge pixels to be identified.

[0091] The filtered image refers to the image obtained after filtering out image noise from the left image.

[0092] The preset horizontal matrix and the preset vertical matrix can be set according to actual needs. The preset horizontal matrix and the preset vertical matrix can be set to 3*3 matrices.

[0093] The neighboring pixels refer to other gradient pixels whose pixel distance from each gradient pixel is less than a preset distance.

[0094] By performing Gaussian convolution on the left image, noise in the left image can be filtered out. Through the preset horizontal matrix and the preset vertical matrix, the brightness values ​​in the filtered image can be converted into gradient values. Then, by comparing the gradient of each gradient pixel with its corresponding neighboring pixels, the edge boundary image can be generated. By setting the first preset gradient threshold and the second preset gradient threshold, edge pixels with gradient strength less than the second preset gradient threshold can be removed, thereby improving the generation accuracy of the multiple first elliptical boundaries.

[0095] Specifically, the electronic device performs Gaussian convolution processing on the left image to obtain a filtered image, including:

[0096] Count the number of pixels for the length of the left image, and count the number of pixels for the height of the left image;

[0097] If the number of length pixels is not equal to the number of height pixels, then the left image is cropped to obtain a cropped image;

[0098] A convolution matrix is ​​generated based on the number of horizontal pixels in the cropped image;

[0099] Based on the convolution matrix, the cropping pixel matrix of each cropping pixel in the cropped image is convolved to obtain the brightness value of each cropping pixel;

[0100] The filtered image is generated based on multiple brightness values.

[0101] In this cropped image, the number of horizontal pixels and the number of vertical pixels are equal. When the number of length pixels is less than the number of height pixels, then both the number of horizontal pixels and the number of vertical pixels are equal to the number of length pixels. When the number of length pixels is greater than the number of height pixels, then both the number of horizontal pixels and the number of vertical pixels are equal to the number of height pixels.

[0102] The formula for generating each element of the convolution matrix is:

[0103]

[0104] 1≤i,j≤(2k+1);

[0105] Among them, A ij This represents the matrix element in the i-th row and j-th column of the convolution matrix.

[0106] The cropped pixel matrix refers to a matrix centered on the pixel value of the cropped pixel. The matrix form of the cropped pixel matrix can be 3*3, etc.

[0107] By using the specified number of horizontal pixels, a convolution matrix can be reasonably generated. Then, by performing convolution processing on each cropped pixel using the convolution matrix, noise in the left image can be effectively filtered out, improving the accuracy of the filtered image.

[0108] Specifically, the electronic device performs image gradient calculation on the filtered image based on a preset horizontal matrix and a preset vertical matrix to obtain a gradient image including:

[0109] Based on the preset horizontal matrix, the filtering pixel matrix of each filtering pixel in the filtered image is convolved to obtain the horizontal gradient value of each filtering pixel.

[0110] The filter pixel matrix is ​​convolved based on the preset vertical matrix to obtain the vertical gradient value of each filter pixel.

[0111] The gradient image is generated based on the horizontal gradient value and the vertical gradient value.

[0112] The filtered pixel matrix refers to a matrix centered on the brightness value of the filtered pixel. The matrix form of the filtered pixel matrix can be 3*3, etc.

[0113] The gradient intensity value of each pixel in the gradient image is the square root of the sum of the squares of the horizontal gradient value and the vertical gradient value.

[0114] By performing convolution processing on the filtered pixel matrix using the preset horizontal matrix and the preset vertical matrix respectively, gradient values ​​of the image can be generated from different dimensions. Furthermore, by combining the gradient values ​​from different dimensions, the gradient image can be generated more accurately.

[0115] Specifically, the electronic device performs gradient comparison between each gradient pixel in the gradient image and its corresponding neighboring pixels to generate an edge boundary image, including:

[0116] Obtain the gradient intensity of each gradient pixel in the gradient image;

[0117] Based on the pixel position of each gradient pixel in the gradient image, the neighboring pixels of each gradient pixel are identified;

[0118] The gradient intensity of each gradient pixel is compared with the gradient intensity of its neighboring pixels.

[0119] The edge boundary image is generated based on the gradient pixels whose gradient intensity is greater than that of their neighboring pixels.

[0120] The neighboring pixels refer to other gradient pixels that are adjacent to each gradient pixel.

[0121] By comparing the gradient intensity of each gradient pixel with the gradient intensity of its neighboring pixels, edge pixels can be accurately filtered out, thereby improving the accuracy of the generated edge boundary image.

[0122] Specifically, the electronic device generates the plurality of first elliptical boundaries based on the target edge pixels and the edge pixels to be identified, including:

[0123] Based on the pixel positions of the target edge pixels and the edge pixels to be identified in the edge boundary image, connect the target edge pixels and the edge pixels to be identified to obtain a connecting arc;

[0124] Identify the target arc corresponding to the edge pixel to be identified from the connecting arc, wherein the arc length of the target arc is equal to a preset length;

[0125] If the target arc includes the target edge pixels, then the edge pixels to be identified are determined as identified edge pixels;

[0126] Connect the target edge pixels and the identified edge pixels to obtain multiple concave arcs and multiple convex arcs;

[0127] Multiple arcs to be processed are generated based on the multiple concave arcs and the multiple convex arcs;

[0128] Ellipse fitting is performed on each arc to be processed to obtain the plurality of first elliptical boundaries.

[0129] By detecting whether the target arc includes the target edge pixels, it is possible to accurately detect whether the edge pixels to be identified belong to noise or edge pixels, thereby improving the generation accuracy of the multiple concave arcs and the multiple convex arcs. Furthermore, by performing ellipse fitting on each arc to be processed, the generation accuracy of the multiple first elliptical boundaries is improved.

[0130] In other embodiments, the electronic device generates multiple second elliptical boundaries in a manner similar to that of the electronic device generating multiple first elliptical boundaries, and this application will not elaborate further on this.

[0131] 103. Select a first clustering ellipse from the plurality of first elliptical boundaries, and select a second clustering ellipse from the plurality of second elliptical boundaries.

[0132] In at least one embodiment of the present invention, the first clustered ellipse refers to the elliptical image corresponding to the tabletop cup captured by the left camera, and the second clustered ellipse refers to the elliptical image corresponding to the tabletop cup captured by the right camera.

[0133] In at least one embodiment of the present invention, the electronic device selects a first clustering ellipse from the plurality of first elliptical boundaries, including:

[0134] Count the number of pixels that overlap with the corresponding connecting arc of each first ellipse boundary;

[0135] Calculate the total number of pixels in each first ellipse boundary;

[0136] Calculate the ratio of the number of pixels to the total number of pixels;

[0137] The boundary of the first ellipse with the largest ratio is determined as the first cluster ellipse.

[0138] The overlapping pixel refers to the pixel whose boundary of the first ellipse coincides with the corresponding connecting arc.

[0139] The first cluster ellipse can be accurately selected by using the ratio of the stated ratio.

[0140] In other embodiments, the electronic device selects the second clustering ellipse in a similar manner to the electronic device selects the first clustering ellipse, and this application will not elaborate further on this.

[0141] 104. Match the first cluster ellipse with the second cluster ellipse to obtain the target left ellipse of the left image and the target right ellipse of the right image.

[0142] In at least one embodiment of the present invention, the target left ellipse refers to the first cluster ellipse in the left image that matches the second cluster ellipse, and the target right ellipse refers to the second cluster ellipse in the right image that matches the first cluster ellipse.

[0143] In at least one embodiment of the present invention, the electronic device matches the first clustered ellipse with the second clustered ellipse to obtain the target left ellipse of the left image and the target right ellipse of the right image, including:

[0144] Obtain the left intrinsic parameter matrix of the left camera and the right intrinsic parameter matrix of the right camera in the stereo camera;

[0145] A left coordinate system is constructed based on the left image, and a right coordinate system is constructed based on the right image;

[0146] Obtain the extrinsic matrix and translation vector of the left coordinate system and the right coordinate system;

[0147] Obtain the first cluster coordinate information of the first cluster center of the first clustering ellipse in the left coordinate system, and obtain the second cluster coordinate information of the second cluster center of the second clustering ellipse in the right coordinate system;

[0148] Error values ​​are generated based on the left intrinsic parameter matrix, the right intrinsic parameter matrix, the extrinsic parameter matrix, the translation vector, the first clustering coordinate information, and the second clustering coordinate information;

[0149] If the error value is less than the preset value, then the first cluster ellipse is determined as the target left ellipse, and the second cluster ellipse is determined as the target right ellipse.

[0150] The formula for generating the error value is as follows:

[0151]

[0152] Where y represents the error value, K1 represents the left intrinsic parameter matrix, K2 represents the right intrinsic parameter matrix, r represents the extrinsic parameter matrix, t represents the translation vector, (x1, y1) represents the first clustering coordinate information, and (x2, y2) represents the second clustering coordinate information.

[0153] The preset value is a positive real number. The specific value of the preset value can be set according to actual needs. In the ideal case, the preset value can be set to 0.

[0154] The error value can accurately measure the matching relationship between the first cluster ellipse and the second cluster ellipse, thereby accurately generating the target left ellipse and the target right ellipse.

[0155] 105. The initial distance between the desktop cup and the electronic device is identified based on the left center point of the target left ellipse and the right center point of the target right ellipse, and the cup diameter of the desktop cup is identified based on the first left major axis vertex of the target left ellipse, the second left major axis vertex of the target left ellipse, the first right major axis vertex of the target right ellipse, and the second right major axis vertex of the target right ellipse.

[0156] In at least one embodiment of the present invention, the initial distance refers to the distance between the desktop cup and the electronic device.

[0157] In at least one embodiment of the present invention, the electronic device identifies the diameter of the desktop cup based on the first left major axis vertex of the target left ellipse, the second left major axis vertex of the target left ellipse, the first right major axis vertex of the target right ellipse, and the second right major axis vertex of the target right ellipse, including:

[0158] Based on the extrinsic parameter matrix, the translation vector, the left intrinsic parameter matrix, and the first left major axis vertex coordinate information in the left coordinate system, the first target coordinate information of the first left major axis vertex in the right coordinate system is generated;

[0159] Based on the extrinsic parameter matrix, the translation vector, the left intrinsic parameter matrix, and the second left major axis vertex coordinate information in the left coordinate system, the second target coordinate information of the second left major axis vertex in the right coordinate system is generated;

[0160] The first vertex disparity between the first left major axis vertex and the first right major axis vertex is generated based on the first right major axis coordinate information in the right coordinate system and the first target coordinate information;

[0161] The second vertex disparity between the second left major axis vertex and the second right major axis vertex is generated based on the second right major axis coordinate information of the second right coordinate system and the second target coordinate information;

[0162] The first rim distance between the desktop cup and the electronic device is generated based on the focal length of the binocular camera, the optical center distance between the left camera and the right camera, and the first vertex parallax; and the second rim distance between the desktop cup and the electronic device is generated based on the focal length, the optical center distance, and the second vertex parallax.

[0163] The diameter of the cup is obtained by calculating the difference between the distance between the first and second cup rims.

[0164] The method for generating the first target coordinate information is as follows:

[0165]

[0166] Where f represents the first target coordinate information, and (a1, b1) represents the first left major axis coordinate information.

[0167] The first vertex disparity refers to the difference between the first target coordinate information and the first right major axis coordinate information.

[0168] The first cup edge distance is generated by calculating the product of the focal length and the optical center distance, and calculating the proportion of the product in the first vertex parallax.

[0169] Through the above implementation method, the distance between the first and second cup rims can be accurately calculated, thereby improving the accuracy of the cup diameter.

[0170] In other embodiments, the electronic device identifies the initial distance in a similar way to the electronic device identify the first cup rim distance and the second cup rim distance, which will not be described in detail here.

[0171] 106. Based on the positional relationship between the electronic device and the robotic arm and the initial distance, generate the distance relationship between the robotic arm and the tabletop cup.

[0172] In at least one embodiment of the present invention, the positional relationship includes the specific position and positional length of the electronic device relative to the robotic arm.

[0173] The distance relationship refers to the distance between the tabletop cup and the robotic arm.

[0174] In at least one embodiment of the present invention, when the specific position is that the electronic device and the robotic arm are respectively located on both sides of the tabletop cup, the electronic device calculates the difference between the position length and the initial distance to obtain the distance relationship.

[0175] In other embodiments, when the specific position is such that the robotic arm and the tabletop cup are on either side of the electronic device, the electronic device calculates the sum of the position length and the initial distance to obtain the distance relationship.

[0176] In other embodiments, when the specific position is when the electronic device and the desktop cup are on either side of the robotic arm, the electronic device calculates the difference between the initial distance and the position length to obtain the distance relationship.

[0177] Through the above implementation method, the distance relationship can be accurately quantified by combining the specific position of the electronic device relative to the robotic arm.

[0178] 107. Based on the distance relationship and the diameter of the cup, control the robotic arm to operate.

[0179] It should be emphasized that, to further ensure the privacy and security of the distance relationship and the cup diameter, the distance relationship and the cup diameter can also be stored in a blockchain node.

[0180] In at least one embodiment of the present invention, the electronic device generates a control program based on the distance relationship and the diameter of the cup, and the electronic device calls the control program to control the robotic arm to perform operations such as pouring tea and grasping the cup on the table.

[0181] This application can accurately control the robotic arm to grasp the cup on the table by identifying the diameter of the cup, and can accurately control the robotic arm to perform operations such as pouring tea from the cup on the table by identifying the distance relationship.

[0182] As can be seen from the above technical solutions, this application uses ellipse coarse detection, which, due to its low computational load, can detect the position of the table cup in real time. This allows for flexible adjustment and control of the robotic arm's operation. Furthermore, by selecting the first and second clustered ellipses, errors caused by coarse detection can be avoided. By matching the first and second clustered ellipses, and combining the characteristics of the binocular camera, the accuracy of filtering the target left and right ellipses is improved, thereby increasing the accuracy of distance relationship generation. Thus, this application, by introducing visual perception, can solve the problem of the robotic arm failing to accurately pour tea due to the movement of the table cup. By accurately identifying the distance relationship between the robotic arm and the table cup, this application can adapt to dynamic scenes without requiring the table cup to be placed in a specific position. In addition, this application can accurately identify the diameter of the table cup using the first left major axis vertex, the second left major axis vertex, the first right major axis vertex, and the second right major axis vertex. Therefore, this application can use table cups of any size, improving its practicality and further enhancing its adaptability to dynamic scenes.

[0183] like Figure 3The diagram shown is a functional block diagram of a preferred embodiment of the robotic arm control device of the present invention. The robotic arm control device 11 includes an acquisition unit 110, a detection unit 111, a selection unit 112, a matching unit 113, an identification unit 114, a generation unit 115, and a control unit 116. The module / unit referred to in this invention is a series of computer-readable instruction segments that can be acquired by the processor 13 and perform a fixed function, stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0184] The robotic arm control device 11 operates on an electronic device, which is equipped with a binocular camera and communicates with the robotic arm.

[0185] The acquisition unit 110 is used to acquire the left and right images of the tabletop cup captured by the binocular camera;

[0186] The detection unit 111 is used to perform coarse ellipse detection on the left image to obtain multiple first ellipse boundaries, and to perform coarse ellipse detection on the right image to obtain multiple second ellipse boundaries.

[0187] The selection unit 112 is used to select a first clustering ellipse from the plurality of first elliptical boundaries and to select a second clustering ellipse from the plurality of second elliptical boundaries;

[0188] The matching unit 113 is used to match the first clustering ellipse with the second clustering ellipse to obtain the target left ellipse of the left image and the target right ellipse of the right image.

[0189] The identification unit 114 is used to identify the initial distance between the desktop cup and the electronic device based on the left center point of the target left ellipse and the right center point of the target right ellipse, and to identify the cup diameter of the desktop cup based on the first left major axis vertex of the target left ellipse, the second left major axis vertex of the target left ellipse, the first right major axis vertex of the target right ellipse, and the second right major axis vertex of the target right ellipse.

[0190] The generation unit 115 is used to generate the distance relationship between the robotic arm and the tabletop cup based on the positional relationship between the electronic device and the robotic arm and the initial distance;

[0191] Control unit 116 is used to control the operation of the robotic arm based on the distance relationship and the diameter of the cup.

[0192] In at least one embodiment of the present invention, the detection unit 111 is further configured to perform Gaussian convolution processing on the left image to obtain a filtered image;

[0193] Based on a preset horizontal matrix and a preset vertical matrix, the filtered image is subjected to image gradient calculation to obtain a gradient image;

[0194] The gradient of each gradient pixel in the gradient image is compared with its corresponding neighboring pixels to generate an edge boundary image;

[0195] Obtain the gradient intensity of each edge pixel in the edge boundary image;

[0196] Edge pixels with gradient strength greater than or equal to a first preset gradient threshold are identified as target edge pixels, and edge pixels with gradient strength less than the first preset gradient threshold but greater than or equal to a second preset gradient threshold are identified as edge pixels to be identified, wherein the first preset gradient threshold is greater than the second preset gradient threshold.

[0197] The plurality of first elliptical boundaries are generated based on the target edge pixels and the edge pixels to be identified.

[0198] In at least one embodiment of the present invention, the detection unit 111 is further configured to count the number of length pixels of the left image and the number of height pixels of the left image;

[0199] If the number of length pixels is not equal to the number of height pixels, then the left image is cropped to obtain a cropped image;

[0200] A convolution matrix is ​​generated based on the number of horizontal pixels in the cropped image;

[0201] Based on the convolution matrix, the cropping pixel matrix of each cropping pixel in the cropped image is convolved to obtain the brightness value of each cropping pixel;

[0202] The filtered image is generated based on multiple brightness values.

[0203] In at least one embodiment of the present invention, the detection unit 111 is further configured to perform convolution processing on the filter pixel matrix of each filter pixel in the filtered image based on the preset horizontal matrix to obtain the horizontal gradient value of each filter pixel.

[0204] The filter pixel matrix is ​​convolved based on the preset vertical matrix to obtain the vertical gradient value of each filter pixel.

[0205] The gradient image is generated based on the horizontal gradient value and the vertical gradient value.

[0206] In at least one embodiment of the present invention, the detection unit 111 is further configured to connect the target edge pixel and the edge pixel to be identified in the edge boundary image according to the pixel positions of the target edge pixel and the edge pixel to be identified in the edge boundary image to obtain a connecting arc;

[0207] Identify the target arc corresponding to the edge pixel to be identified from the connecting arc, wherein the arc length of the target arc is equal to a preset length;

[0208] If the target arc includes the target edge pixels, then the edge pixels to be identified are determined as identified edge pixels;

[0209] Connect the target edge pixels and the identified edge pixels to obtain multiple concave arcs and multiple convex arcs;

[0210] Multiple arcs to be processed are generated based on the multiple concave arcs and the multiple convex arcs;

[0211] Ellipse fitting is performed on each arc to be processed to obtain the plurality of first elliptical boundaries.

[0212] In at least one embodiment of the present invention, the matching unit 113 is further configured to obtain the left intrinsic parameter matrix of the left camera in the binocular camera and the right intrinsic parameter matrix of the right camera in the binocular camera;

[0213] A left coordinate system is constructed based on the left image, and a right coordinate system is constructed based on the right image;

[0214] Obtain the extrinsic matrix and translation vector of the left coordinate system and the right coordinate system;

[0215] Obtain the first cluster coordinate information of the first cluster center of the first clustering ellipse in the left coordinate system, and obtain the second cluster coordinate information of the second cluster center of the second clustering ellipse in the right coordinate system;

[0216] Error values ​​are generated based on the left intrinsic parameter matrix, the right intrinsic parameter matrix, the extrinsic parameter matrix, the translation vector, the first clustering coordinate information, and the second clustering coordinate information;

[0217] If the error value is less than the preset value, then the first cluster ellipse is determined as the target left ellipse, and the second cluster ellipse is determined as the target right ellipse.

[0218] In at least one embodiment of the present invention, the identification unit 114 is further configured to generate first target coordinate information of the first left major axis vertex in the right coordinate system based on the extrinsic parameter matrix, the translation vector, the left intrinsic parameter matrix, and the first left major axis coordinate information of the first left major axis vertex in the left coordinate system.

[0219] Based on the extrinsic parameter matrix, the translation vector, the left intrinsic parameter matrix, and the second left major axis vertex coordinate information in the left coordinate system, the second target coordinate information of the second left major axis vertex in the right coordinate system is generated;

[0220] The first vertex disparity between the first left major axis vertex and the first right major axis vertex is generated based on the first right major axis coordinate information in the right coordinate system and the first target coordinate information;

[0221] The second vertex disparity between the second left major axis vertex and the second right major axis vertex is generated based on the second right major axis coordinate information of the second right coordinate system and the second target coordinate information;

[0222] The first rim distance between the desktop cup and the electronic device is generated based on the focal length of the binocular camera, the optical center distance between the left camera and the right camera, and the first vertex parallax; and the second rim distance between the desktop cup and the electronic device is generated based on the focal length, the optical center distance, and the second vertex parallax.

[0223] The diameter of the cup is obtained by calculating the difference between the distance between the first and second cup rims.

[0224] As can be seen from the above technical solutions, this application uses ellipse coarse detection, which, due to its low computational load, can detect the position of the table cup in real time. This allows for flexible adjustment and control of the robotic arm's operation. Furthermore, by selecting the first and second clustered ellipses, errors caused by coarse detection can be avoided. By matching the first and second clustered ellipses, and combining the characteristics of the binocular camera, the accuracy of filtering the target left and right ellipses is improved, thereby increasing the accuracy of distance relationship generation. Thus, this application, by introducing visual perception, can solve the problem of the robotic arm failing to accurately pour tea due to the movement of the table cup. By accurately identifying the distance relationship between the robotic arm and the table cup, this application can adapt to dynamic scenes without requiring the table cup to be placed in a specific position. In addition, this application can accurately identify the diameter of the table cup using the first left major axis vertex, the second left major axis vertex, the first right major axis vertex, and the second right major axis vertex. Therefore, this application can use table cups of any size, improving its practicality and further enhancing its adaptability to dynamic scenes.

[0225] like Figure 4 The diagram shown is a schematic diagram of the electronic device for implementing the robotic arm control method of the present invention.

[0226] In one embodiment of the present invention, the electronic device 1 includes, but is not limited to, a memory 12, a processor 13, and computer-readable instructions, such as a robotic arm control program, stored in the memory 12 and executable on the processor 13.

[0227] Those skilled in the art will understand that the schematic diagram is merely an example of electronic device 1 and does not constitute a limitation on electronic device 1. It may include more or fewer components than shown in the diagram, or combine certain components, or different components. For example, electronic device 1 may also include input / output devices, network access devices, buses, etc.

[0228] The processor 13 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 13 is the computing core and control center of the electronic device 1, connecting various parts of the electronic device 1 through various interfaces and lines, and executing the operating system of the electronic device 1, as well as various installed application programs and program code.

[0229] For example, the computer-readable instructions can be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules / units can be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer-readable instructions in the electronic device 1. For example, the computer-readable instructions can be divided into an acquisition unit 110, a detection unit 111, a selection unit 112, a matching unit 113, an identification unit 114, a generation unit 115, and a control unit 116.

[0230] The memory 12 can be used to store the computer-readable instructions and / or modules. The processor 13 implements various functions of the electronic device 1 by running or executing the computer-readable instructions and / or modules stored in the memory 12 and calling the data stored in the memory 12. The memory 12 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. The memory 12 may include non-volatile and volatile memory, such as: hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other storage devices.

[0231] The memory 12 can be the external memory and / or internal memory of the electronic device 1. Furthermore, the memory 12 can be a physical memory, such as a memory module, a TF card (Trans-flash Card), etc.

[0232] If the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when executed by a processor, the computer-readable instructions can implement the steps of the various method embodiments described above.

[0233] The computer-readable instructions include computer-readable instruction code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer-readable instruction code, recording medium, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), and random access memory (RAM).

[0234] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed robotic arm control, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0235] Combination Figure 2 The memory 12 in the electronic device 1 stores computer-readable instructions to implement a robotic arm control method, and the processor 13 can execute the computer-readable instructions to achieve the following:

[0236] Acquire the left and right images of a tabletop cup captured by a binocular camera;

[0237] Perform coarse ellipse detection on the left image to obtain multiple first ellipse boundaries, and perform coarse ellipse detection on the right image to obtain multiple second ellipse boundaries;

[0238] A first clustering ellipse is selected from the plurality of first elliptical boundaries, and a second clustering ellipse is selected from the plurality of second elliptical boundaries;

[0239] The first cluster ellipse is matched with the second cluster ellipse to obtain the target left ellipse of the left image and the target right ellipse of the right image;

[0240] The initial distance between the desktop cup and the electronic device is identified based on the left center point of the target left ellipse and the right center point of the target right ellipse, and the cup diameter of the desktop cup is identified based on the first left major axis vertex of the target left ellipse, the second left major axis vertex of the target left ellipse, the first right major axis vertex of the target right ellipse, and the second right major axis vertex of the target right ellipse.

[0241] Based on the positional relationship between the electronic device and the robotic arm, and the initial distance, the distance relationship between the robotic arm and the tabletop cup is generated;

[0242] Based on the distance relationship and the diameter of the cup, the robotic arm is controlled to operate.

[0243] Specifically, the specific implementation method of the processor 13 for the above-mentioned computer-readable instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0244] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0245] The computer-readable storage medium stores computer-readable instructions, which, when executed by the processor 13, are used to perform the following steps:

[0246] Acquire the left and right images of a tabletop cup captured by a binocular camera;

[0247] Perform coarse ellipse detection on the left image to obtain multiple first ellipse boundaries, and perform coarse ellipse detection on the right image to obtain multiple second ellipse boundaries;

[0248] A first clustering ellipse is selected from the plurality of first elliptical boundaries, and a second clustering ellipse is selected from the plurality of second elliptical boundaries;

[0249] The first cluster ellipse is matched with the second cluster ellipse to obtain the target left ellipse of the left image and the target right ellipse of the right image;

[0250] The initial distance between the desktop cup and the electronic device is identified based on the left center point of the target left ellipse and the right center point of the target right ellipse, and the cup diameter of the desktop cup is identified based on the first left major axis vertex of the target left ellipse, the second left major axis vertex of the target left ellipse, the first right major axis vertex of the target right ellipse, and the second right major axis vertex of the target right ellipse.

[0251] Based on the positional relationship between the electronic device and the robotic arm, and the initial distance, the distance relationship between the robotic arm and the tabletop cup is generated;

[0252] Based on the distance relationship and the diameter of the cup, the robotic arm is controlled to operate.

[0253] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0254] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0255] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0256] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices described may also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0257] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A robotic arm control method, applied in an electronic device, wherein the electronic device is equipped with a binocular camera, and the electronic device communicates with the robotic arm, characterized in that, The robotic arm control method includes: Acquire the left and right images of the tabletop cup captured by the binocular camera; Perform coarse ellipse detection on the left image to obtain multiple first ellipse boundaries, and perform coarse ellipse detection on the right image to obtain multiple second ellipse boundaries; A first clustering ellipse is selected from the plurality of first elliptical boundaries, and a second clustering ellipse is selected from the plurality of second elliptical boundaries; The first cluster ellipse is matched with the second cluster ellipse to obtain the target left ellipse of the left image and the target right ellipse of the right image; The initial distance between the desktop cup and the electronic device is identified based on the left center point of the target left ellipse and the right center point of the target right ellipse. The diameter of the desktop cup is then identified based on the first left major axis vertex, the second left major axis vertex, the first right major axis vertex, and the second right major axis vertex of the target right ellipse. This includes: obtaining the left intrinsic parameter matrix of the left camera and the right intrinsic parameter matrix of the right camera in the binocular camera; constructing a left coordinate system based on the left image and a right coordinate system based on the right image; obtaining the extrinsic parameter matrices and translation vectors of the left and right coordinate systems; generating the first target coordinate information of the first left major axis vertex in the right coordinate system based on the extrinsic parameter matrix, the translation vector, the left intrinsic parameter matrix, and the first left major axis coordinate information of the first left major axis vertex in the left coordinate system; and determining the cup diameter based on the extrinsic parameter matrix, the translation vector, the left intrinsic parameter matrix, and the first left major axis coordinate information of the first left major axis vertex in the right coordinate system. The second left major axis vertex is used to generate the second target coordinate information in the right coordinate system based on the second left major axis coordinate information in the left coordinate system. The first vertex disparity between the first left major axis vertex and the first right major axis vertex is generated based on the first right major axis coordinate information in the right coordinate system and the first target coordinate information. The second vertex disparity between the second left major axis vertex and the second right major axis vertex is generated based on the second right major axis coordinate information in the right coordinate system and the second target coordinate information. The first rim distance between the desktop cup and the electronic device is generated based on the focal length of the binocular camera, the optical center distance between the left and right cameras, and the first vertex disparity. The second rim distance between the desktop cup and the electronic device is generated based on the focal length, the optical center distance, and the second vertex disparity. The difference between the first rim distance and the second rim distance is calculated to obtain the cup diameter. Based on the positional relationship between the electronic device and the robotic arm, and the initial distance, the distance relationship between the robotic arm and the tabletop cup is generated; Based on the distance relationship and the diameter of the cup, the robotic arm is controlled to operate.

2. The robotic arm control method as described in claim 1, characterized in that, The step of performing coarse ellipse detection on the left image to obtain multiple first ellipse boundaries includes: The left image is subjected to Gaussian convolution to obtain a filtered image; Based on a preset horizontal matrix and a preset vertical matrix, the filtered image is subjected to image gradient calculation to obtain a gradient image; The gradient of each gradient pixel in the gradient image is compared with its corresponding neighboring pixels to generate an edge boundary image; Obtain the gradient intensity of each edge pixel in the edge boundary image; Edge pixels with gradient strength greater than or equal to a first preset gradient threshold are identified as target edge pixels, and edge pixels with gradient strength less than the first preset gradient threshold but greater than or equal to a second preset gradient threshold are identified as edge pixels to be identified, wherein the first preset gradient threshold is greater than the second preset gradient threshold. The plurality of first elliptical boundaries are generated based on the target edge pixels and the edge pixels to be identified.

3. The robotic arm control method as described in claim 2, characterized in that, The step of performing Gaussian convolution processing on the left image to obtain the filtered image includes: Count the number of pixels for the length of the left image, and count the number of pixels for the height of the left image; If the number of length pixels is not equal to the number of height pixels, then the left image is cropped to obtain a cropped image; A convolution matrix is ​​generated based on the number of horizontal pixels in the cropped image; Based on the convolution matrix, the cropping pixel matrix of each cropping pixel in the cropped image is convolved to obtain the brightness value of each cropping pixel; The filtered image is generated based on multiple brightness values.

4. The robotic arm control method as described in claim 2, characterized in that, The step of calculating the image gradient of the filtered image based on a preset horizontal matrix and a preset vertical matrix to obtain the gradient image includes: Based on the preset horizontal matrix, the filtering pixel matrix of each filtering pixel in the filtered image is convolved to obtain the horizontal gradient value of each filtering pixel. The filter pixel matrix is ​​convolved based on the preset vertical matrix to obtain the vertical gradient value of each filter pixel. The gradient image is generated based on the horizontal gradient value and the vertical gradient value.

5. The robotic arm control method as described in claim 2, characterized in that, The step of generating the plurality of first elliptical boundaries based on the target edge pixels and the edge pixels to be identified includes: Based on the pixel positions of the target edge pixels and the edge pixels to be identified in the edge boundary image, connect the target edge pixels and the edge pixels to be identified to obtain a connecting arc; Identify the target arc corresponding to the edge pixel to be identified from the connecting arc, wherein the arc length of the target arc is equal to a preset length; If the target arc includes the target edge pixels, then the edge pixels to be identified are determined as identified edge pixels; Connect the target edge pixels and the identified edge pixels to obtain multiple concave arcs and multiple convex arcs; Multiple arcs to be processed are generated based on the multiple concave arcs and the multiple convex arcs; Ellipse fitting is performed on each arc to be processed to obtain the plurality of first elliptical boundaries.

6. The robotic arm control method as described in claim 1, characterized in that, The step of matching the first clustered ellipse with the second clustered ellipse to obtain the target left ellipse of the left image and the target right ellipse of the right image includes: Obtain the first cluster coordinate information of the first cluster center of the first clustering ellipse in the left coordinate system, and obtain the second cluster coordinate information of the second cluster center of the second clustering ellipse in the right coordinate system; Error values ​​are generated based on the left intrinsic parameter matrix, the right intrinsic parameter matrix, the extrinsic parameter matrix, the translation vector, the first clustering coordinate information, and the second clustering coordinate information; If the error value is less than the preset value, then the first cluster ellipse is determined as the target left ellipse, and the second cluster ellipse is determined as the target right ellipse.

7. A robotic arm control device, operating within an electronic device, wherein the electronic device is equipped with a binocular camera, and the electronic device communicates with the robotic arm, characterized in that, The robotic arm control device includes: The acquisition unit is used to acquire the left and right images of the tabletop cup captured by the binocular camera; The detection unit is used to perform coarse ellipse detection on the left image to obtain multiple first ellipse boundaries, and to perform coarse ellipse detection on the right image to obtain multiple second ellipse boundaries. A selection unit is used to select a first clustering ellipse from the plurality of first elliptical boundaries and to select a second clustering ellipse from the plurality of second elliptical boundaries; A matching unit is used to match the first clustered ellipse with the second clustered ellipse to obtain the target left ellipse of the left image and the target right ellipse of the right image; The identification unit is configured to identify the initial distance between the desktop cup and the electronic device based on the left center point of the target left ellipse and the right center point of the target right ellipse, and to identify the cup diameter of the desktop cup based on the first left major axis vertex, the second left major axis vertex of the target left ellipse, the first right major axis vertex of the target right ellipse, and the second right major axis vertex of the target right ellipse. The unit includes: acquiring the left intrinsic parameter matrix of the left camera and the right intrinsic parameter matrix of the right camera in the binocular camera; constructing a left coordinate system based on the left image and a right coordinate system based on the right image; acquiring the extrinsic parameter matrices and translation vectors of the left and right coordinate systems; generating the first target coordinate information of the first left major axis vertex in the right coordinate system based on the extrinsic parameter matrix, the translation vector, the left intrinsic parameter matrix, and the first left major axis coordinate information of the first left major axis vertex in the left coordinate system; and acquiring the extrinsic parameter matrix, the translation vector, and the left intrinsic parameter matrix. The matrix and the second left major axis vertex in the left coordinate system are used to generate the second target coordinate information of the second left major axis vertex in the right coordinate system; the first vertex disparity between the first left major axis vertex and the first right major axis vertex is generated based on the first right major axis vertex in the right coordinate system and the first target coordinate information; the second vertex disparity between the second left major axis vertex and the second right major axis vertex is generated based on the second right major axis vertex in the right coordinate system and the second target coordinate information; the first rim distance between the desktop cup and the electronic device is generated based on the focal length of the binocular camera, the optical center distance between the left camera and the right camera, and the first vertex disparity; and the second rim distance between the desktop cup and the electronic device is generated based on the focal length, the optical center distance, and the second vertex disparity; the difference between the first rim distance and the second rim distance is calculated to obtain the cup diameter; A generation unit is used to generate a distance relationship between the robotic arm and the tabletop cup based on the positional relationship between the electronic device and the robotic arm and the initial distance; The control unit is used to control the operation of the robotic arm based on the distance relationship and the diameter of the cup.

8. An electronic device, characterized in that, The electronic device includes: Memory, which stores computer-readable instructions; and The processor executes computer-readable instructions stored in the memory to implement the robotic arm control method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which are executed by a processor in an electronic device to implement the robotic arm control method as described in any one of claims 1 to 6.

Images