Workpiece moving method, device and equipment based on image recognition and storage medium

Abstract

The invention provides a workpiece moving method and device based on image recognition, equipment and a storage medium, and the method comprises the steps: obtaining a first image and first physical coordinate data of a first region, and recognizing a first center physical coordinate and a first rotation angle of a workpiece center point in the first region based on the first image; first parameter data of a mechanical arm are obtained, a coordinate compensation value of the workpiece is calculated based on the first parameter data, the first rotation angle and the first center physical coordinate, the first center physical coordinate is modified based on the coordinate compensation value, and a grabbing coordinate is obtained; and the workpiece is grabbed on the basis of the grabbing coordinates and moved to the position above the second area, the flange plate in the mechanical arm is rotated on the basis of the first parameter data, and the workpiece is placed in the second area on the basis of the rotated angle.

Description

Technical Field

[0001] The present disclosure relates to the field of computer control, and particularly to a workpiece moving method, device, equipment and storage medium based on image recognition. Background Art

[0002] In an industrial scenario, since the end of the robotic arm needs to replace various tools according to the actual application scenario, it often involves the problem of rotating around the center point of these tools. By default, the robotic arm is in the world coordinate system, and the rotation center point is directly below the flange rather than the center point of the tool. At this time, a conversion relationship is required to convert the rotation from the flange center point to the tool center point. Conventional solutions perform the conversion of the tool coordinate system through the four-point or six-point calibration carried by the robotic arm itself, or deduce the conversion matrix between the world coordinate system and the tool coordinate system through the design parameters of the robotic arm.

[0003] All such solutions need to ensure that the tool center rotates around the same point, and then take four points with different poses. It is easy to have deviations through visual observation. After being converted to the tool coordinate system, the problem of quadrants will be involved. In the application scenarios that require rotation, the problem of quadrant inversion will occur, and the quadrant of the tool coordinate system will be strongly related to the rotation posture, making the coordinates inconvenient to use. Summary of the Invention

[0004] The present disclosure provides a workpiece moving method, device, equipment and storage medium based on image recognition to at least solve the above technical problems existing in the prior art.

[0005] According to a first aspect of the present disclosure, there is provided a workpiece moving method based on image recognition, the method including:

[0006] Obtain a first image and first physical coordinate data of a first region, and recognize a first central physical coordinate and a first rotation angle of a workpiece center point in the first region based on the first image;

[0007] Obtain first parameter data of the robotic arm, calculate a coordinate compensation value of the workpiece based on the first parameter data, the first rotation angle and the first central physical coordinate, and modify the first central physical coordinate based on the coordinate compensation value to obtain a grasping coordinate;

[0008] Grasp the workpiece based on the grasping coordinate, move it above a second region, rotate the flange in the robotic arm based on the first parameter data, and place the workpiece in the second region based on the rotated angle.

[0009] In an implementable embodiment, the method further includes:

[0010] Obtain a second image and second physical coordinate data of a third region, and recognize a second central physical coordinate of a workpiece center point in the third region based on the second image;

[0011] The robotic arm grasps the workpiece based on the second central physical coordinates and moves the workpiece to the first area for placement.

[0012] In an implementable manner, identifying the second central physical coordinates of the workpiece center point in the second area based on the second image includes:

[0013] Identifying the first shadow area in the second image based on the gray information of the second image, and determining the workpiece center point in the third area based on the first shadow area;

[0014] Determining the first pixel coordinates of the workpiece center point in the third area based on the pixel information of the second image;

[0015] Converting the first pixel coordinates into the second central physical coordinates based on the second physical coordinate data and the first coordinate conversion algorithm.

[0016] In an implementable manner, identifying the first central physical coordinates and the first rotation angle of the workpiece center point in the first area based on the first image includes:

[0017] Identifying the second shadow area in the first image based on the gray information of the first image, and determining the workpiece center point in the first area based on the second shadow area;

[0018] Determining the second pixel coordinates of the workpiece center point in the first area based on the pixel information of the first image;

[0019] Converting the second pixel coordinates into the first central physical coordinates based on the first physical coordinate data and the first coordinate conversion algorithm;

[0020] Determining all the pixel coordinates of the workpiece in the first area based on the second shadow area and the pixel information of the first image;

[0021] Converting all the pixel coordinates into the workpiece physical coordinates based on the first physical coordinate data and the first coordinate conversion algorithm, and determining the first rotation angle of the workpiece based on the workpiece physical coordinates.

[0022] In an implementable manner, calculating the coordinate compensation value of the workpiece based on the first parameter data, the first rotation angle and the first central physical coordinates includes:

[0023] The first parameter data includes the horizontal distance from the center point of the robotic arm flange to the robotic arm grasping point and the second angle data of the robotic arm;

[0024] Calculating the abscissa compensation value of the first central coordinate based on the sine function, the abscissa of the first central physical coordinates, the horizontal distance, the second angle data and the first rotation angle;

[0025] Calculate a vertical coordinate compensation value of the first center coordinate based on the cosine function, the vertical coordinate of the first center physical coordinate, the horizontal distance, the second angle data, and the first rotation angle;

[0026] A coordinate compensation value is determined based on the abscissa compensation value and the ordinate compensation value.

[0027] In one embodiment, the method further comprises:

[0028] Obtaining specification parameters of the robotic arm and position information of the first area, the second area, and the third area, and calculating a preset trajectory for the robotic arm to move between the first area, the second area, and the third area based on the specification parameters and the position information;

[0029] Obtaining environmental information of the first area, the second area, and the third area, and determining whether interference will occur when the robotic arm moves based on the environmental information and a preset trajectory;

[0030] When no interference occurs, generating a coordinate modification instruction, and modifying the first center physical coordinate based on the coordinate compensation value in response to the coordinate modification instruction;

[0031] When interference occurs, the first center coordinate is used as the grabbing coordinate.

[0032] According to a second aspect of the present disclosure, there is provided a workpiece moving device based on image recognition, the device comprising:

[0033] a first image recognition unit, configured to acquire a first image and first physical coordinate data of a first area, and recognize a first central physical coordinate and a first rotation angle of a center point of a workpiece in the first area based on the first image;

[0034] a coordinate compensation calculation unit, configured to obtain first parameter data of the robotic arm, calculate a coordinate compensation value of the workpiece based on the first parameter data, the first rotation angle, and the first center physical coordinate, and modify the first center physical coordinate based on the coordinate compensation value to obtain a grasping coordinate;

[0035] The robot control unit is used to grab the workpiece based on the grab coordinates, move to the top of the second area, rotate the flange in the robot based on the first parameter data, and place the workpiece in the second area based on the rotated angle.

[0036] In one embodiment, the device further comprises:

[0037] a second image recognition unit configured to acquire a second image and second physical coordinate data of the third area, and to recognize a second center physical coordinate of a center point of the workpiece in the third area based on the second image; the robotic arm grasps the workpiece based on the second center physical coordinate and moves the workpiece to the first area for placement;

[0038] The second image recognition unit is further configured to: identify a first shadow area in the second image based on grayscale information of the second image, determine a center point of the workpiece in the third area based on the first shadow area; determine a first pixel coordinate of the center point of the workpiece in the third area based on pixel information of the second image; and convert the first pixel coordinate into a second center physical coordinate based on the second physical coordinate data and the first coordinate conversion algorithm;

[0039] The first image recognition unit is further configured to: identify a second shadow area in the first image based on grayscale information of the first image, determine a center point of the workpiece in the first area based on the second shadow area; determine a second pixel coordinate of the center point of the workpiece in the first area based on pixel information of the first image; convert the second pixel coordinate into a first center physical coordinate based on the first physical coordinate data and a first coordinate conversion algorithm; determine all pixel coordinates of the workpiece in the first area based on the second shadow area and the pixel information of the first image; convert all pixel coordinates into physical coordinates of the workpiece based on the first physical coordinate data and the first coordinate conversion algorithm, and determine a first rotation angle of the workpiece based on the physical coordinates of the workpiece;

[0040] The coordinate compensation calculation unit is further configured to: wherein the first parameter data includes a horizontal distance from a center point of the robot arm flange to a grasping point of the robot arm and second angle data of the robot arm; calculate a horizontal coordinate compensation value of the first center coordinate based on a sine function, a horizontal coordinate of the first center physical coordinate, a horizontal distance, the second angle data, and a first rotation angle; calculate a vertical coordinate compensation value of the first center coordinate based on a cosine function, a vertical coordinate of the first center physical coordinate, a horizontal distance, the second angle data, and the first rotation angle; and determine a coordinate compensation value based on the horizontal coordinate compensation value and the vertical coordinate compensation value;

[0041] The interference measurement unit is used to obtain the specification parameters of the robot arm, the position information of the first area, the second area and the third area, and calculate the preset trajectory of the robot arm moving between the first area, the second area and the third area based on the specification parameters and the position information; obtain the environmental information of the first area, the second area and the third area, and judge whether interference will occur when the robot arm moves based on the environmental information and the preset trajectory; when no interference occurs, generate a coordinate modification instruction, and in response to the coordinate modification instruction, modify the first center physical coordinate based on the coordinate compensation value; when interference occurs, use the first center coordinate as the grasping coordinate.

[0042] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0043] at least one processor; and

[0044] a memory communicatively connected to the at least one processor; wherein,

[0045] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0046] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.

[0047] The workpiece movement method, device, equipment and storage medium based on image recognition disclosed in the present invention, ...

[0048] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings, in which several embodiments of the present disclosure are shown by way of example and not limitation, wherein:

[0050] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0051] Figure 1 A schematic diagram of the implementation process of a workpiece moving method based on image recognition according to an embodiment of the present disclosure is shown;

[0052] Figure 2 A schematic diagram of the implementation process of another workpiece moving method based on image recognition according to an embodiment of the present disclosure is shown;

[0053] Figure 3 A schematic diagram of a workpiece moving device based on image recognition according to an embodiment of the present disclosure is shown;

[0054] Figure 4 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0055] To make the purposes, features, and advantages of the present disclosure more apparent and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative work shall fall within the scope of protection of the present disclosure.

[0056] Figure 1The figure shows a schematic implementation flow diagram of a workpiece movement method based on image recognition according to an embodiment of the present disclosure. As Figure 1 shown, the implementation flow of a workpiece movement method based on image recognition according to an embodiment of the present disclosure includes the following steps:

[0057] Step 101: Obtain the first image and the first physical coordinate data of the first area, and identify the first central physical coordinate and the first rotation angle of the workpiece center point in the first area based on the first image.

[0058] In the embodiment of the present disclosure, the first area is the correction area of the workpiece, the first image is obtained by the camera taking a picture of the first area, which contains the relative position information between the workpiece and the first area, and the first physical coordinate data is the physical coordinate data mapped by the first area in the robotic arm control system. Among them, the coverage range of the first physical coordinate data includes the entire first area and also covers the physical coordinate data corresponding to the positions in the first image.

[0059] In the embodiment of the present disclosure, through the gray information in the first picture and the coordinates of the pixel points in the picture, the pixel coordinates of the workpiece in the first image are converted into the corresponding positions in the above-mentioned first physical coordinates, and the first rotation angle of the workpiece with the workpiece center point as the axis is determined through the relationship between the position of the workpiece and the horizontal coordinate system. Specifically: Identify the second shadow area in the first image based on the gray information of the first image, where the second shadow area is the shadow area of the workpiece in the first area in the first image, and determine the workpiece center point in the first area based on the second shadow area; Determine the second pixel coordinates of the workpiece center point in the first area based on the pixel information of the first image; Convert the second pixel coordinates into the first central physical coordinates based on the first physical coordinate data and the first coordinate conversion algorithm, where the first coordinate conversion algorithm can be the nine-point calibration conversion algorithm; Determine all the pixel coordinates of the workpiece in the first area based on the second shadow area and the pixel information of the first image; Convert all the pixel coordinates into the workpiece physical coordinates based on the first physical coordinate data and the first coordinate conversion algorithm, and determine the first rotation angle of the workpiece based on the workpiece physical coordinates.

[0060] In the embodiments of the present disclosure, before the workpiece enters the correction area, the robotic arm can also move the workpiece from the disordered area to the correction area. Specifically: obtain the second image and the second physical coordinate data of the third area, where the third area is the disordered area of the workpiece, the second image is the image information of the third area obtained by the camera photographing, which contains the relative relationship between multiple randomly placed workpieces in the third area and the position of the third area, and the second physical coordinate data is the physical coordinate data mapped by the third area in the robotic arm control system. Based on the second image, identify the second central physical coordinate of the workpiece center point in the third area. Specifically: identify the first shadow area in the second image based on the gray information of the second image, where the first shadow area is the shadow area of the stacked workpieces in the third area, and determine the workpiece center point in the third area based on the first shadow area. At this time, the workpiece center point is the center point of multiple stacked workpieces, not the center point position of a single workpiece; determine the first pixel coordinate of the workpiece center point in the third area based on the pixel information of the second image; and convert the first pixel coordinate into the second central physical coordinate based on the second physical coordinate data and the first coordinate conversion algorithm. The robotic arm grabs the workpiece based on the second central physical coordinate and moves the workpiece to the first area for placement.

[0061] Step 102: Obtain the first parameter data of the robotic arm, calculate the coordinate compensation value of the workpiece based on the first parameter data, the first rotation angle, and the first central physical coordinate, and modify the first central physical coordinate based on the coordinate compensation value to obtain the grasping coordinate.

[0062] In the embodiments of the present disclosure, the first parameter data includes the horizontal distance from the center point of the robotic arm flange to the grasping point of the robotic arm and the second angle data of the robotic arm. The second angle data is the rotation angle of the distance from the center point of the flange to the initial position of the robotic arm grasping tool in the vertical direction (i.e., the longitudinal coordinate axis), and this angle is a fixed angle and does not change with the change of the robotic arm grasping action; among them, calculating the coordinate compensation value of the workpiece based on the first parameter data, the first rotation angle, and the first central physical coordinate is specifically: calculate the abscissa compensation value of the first central coordinate based on the sine function, the abscissa x of the first central physical coordinate, the horizontal distance r, the second angle data θ, and the first rotation angle ω: r·(sin(θ + ω) - sinθ), calculate the ordinate compensation value of the first central coordinate based on the cosine function, the ordinate y of the first central physical coordinate, the horizontal distance r, the second angle data θ, and the first rotation angle ω: r·(cosθ - cos(θ + ω)), and determine the coordinate compensation value based on the abscissa compensation value and the ordinate compensation value. And replace the first central physical coordinate with it as the grasping coordinate.

[0063] Step 103: Grasp the workpiece based on the grasping coordinate, move it above the second area, rotate the flange in the robotic arm based on the first parameter data, and place the workpiece in the second area based on the rotated angle.

[0064] In an embodiment of the present disclosure, the second region is an installation region, and the first parameter data further includes the specification parameters of the robotic arm, such as: the arm length of the robotic arm, the rotation angles and interference angles of each axis, and the movement range of the robotic arm. Obtain the specification parameters of the robotic arm, and the position information of the first region, the second region, and the third region. Among them, the position information of the first region, the second region, and the third region is the coordinate position information mapped by the first region, the second region, and the third region in the robotic arm control system. Based on the specification parameters and the position information, calculate the preset trajectory for the robotic arm to move between the first region, the second region, and the third region. This trajectory can be obtained through computer simulation; obtain the environmental information of the first region, the second region, and the third region. The environmental information is whether there are other devices or equipment near the first region, the second region, and the third region. If so, use their corresponding physical coordinates as the environmental information. If not, there is no environmental information. Based on the environmental information and the preset trajectory, determine whether interference will occur when the robotic arm moves; when no interference occurs, generate a coordinate modification instruction. In response to the coordinate modification instruction, modify the first central physical coordinate to the grasping coordinate based on the coordinate compensation value; when interference occurs, use the first central coordinate as the grasping coordinate. After that, grasp the workpiece based on the grasping coordinate, move above the second region, rotate the flange in the robotic arm based on the first parameter data, and place the workpiece in the second region based on the rotated angle.

[0065] Figure 2 The schematic diagram of the implementation process of another workpiece movement method based on image recognition in an embodiment of the present disclosure is shown, as Figure 2 shown, the implementation process of another workpiece movement method based on image recognition in an embodiment of the present disclosure includes the following steps:

[0066] Step 201, the robotic arm grabs the circuit board from the disordered region and moves the circuit board to the correction region.

[0067] In an embodiment of the present disclosure, the robotic arm first moves to the disordered region, where multiple circuit boards are stacked. By taking a photo, obtain the center point of the circuit board stack in the disordered region. At this time, it is only necessary to take a rough photo of the circuit board to find the center point, and convert the center point coordinates in the image into actual physical coordinates. After the robotic arm moves to the physical coordinates of the center, it descends to grab the circuit board. Because the overlapping angles and center points of multiple circuit boards may be misfound, it is not necessary to pay attention to the rotation angle between the robotic arm and the circuit board, and move the grabbed circuit board to the correction region through the preset position coordinates of the correction region.

[0068] In the embodiments of the present disclosure, the image recognition method for converting the center point coordinates in the image into actual physical coordinates is specifically as follows: identifying the shadow area of the stacked circuit board workpieces in the photographed image based on the gray information of the photographed image of the disordered area, determining the center point of the workpiece in the disordered area based on the shadow area of the stacked circuit board workpieces; determining the pixel coordinates of the center point of the workpiece in the disordered area based on the pixel information of the photographed image; and converting the pixel coordinates into the physical coordinates of the center of the stacked circuit board workpieces based on the physical coordinate data mapped in the robotic arm control system for the disordered area and the nine-point calibration coordinate conversion algorithm.

[0069] Step 202: Place the circuit board in the correction area for image recognition and positioning.

[0070] In the embodiments of the present disclosure, the circuit board is moved to the correction area and the robotic arm rises again. Since there is only one circuit board in the correction area at this time, the physical coordinates of the accurate center point of the circuit board can be obtained through the image recognition method in the above step 201. On this basis, the tilt angle of the workpiece is determined by the horizontal rotation angle of the physical coordinates of the circuit board workpiece around the center point in the correction area, and then the coordinates of the workpiece in the image under the physical coordinates are corrected, and then the recognition and positioning are carried out. Among them, the specific correction method is as follows: in the Cartesian coordinate system, the robotic arm coordinates are described by six parameters X, Y, Z, U, V, and W, where U represents the angle of rotation around the X axis, V represents the angle of rotation around the Y axis, and W represents the angle of rotation around the Z axis.

[0071] In the embodiment of the present disclosure, in most automation application scenarios, all operations are planar. At this time, among the six parameters of the robot arm, U rotating around X and V rotating around Y will not change. The height Z will only take effect when the lowering operation is completed after positioning. So the parameters that need to be paid attention to are XY on the plane and the angle W rotating around Z. Suppose that we move from point A1 to (Δx, Δy) without rotation. The point we move (Δx, Δy) in the world coordinate system is exactly the same as the point we reach by moving (Δx, Δy) in the tool coordinate system. It can be seen that after the conversion of the coordinate system, if no rotation is involved, (Δx, Δy) is just a relative translation. Since the robot arm rotates around the center of the flange, the center point of the grasped circuit board has offset in the X and Y directions, so this offset needs to be compensated back. By taking the circuit board as the center point, it is mapped to the center point of the flange (because it is actually the center point of the moving flange) and an offset occurs, so it is necessary to calculate the Δx and Δy between the two flanges. Since the distance from the center of the flange to the center of the tool remains constant, the center of the flange is understood as the deviation distance between the two points on the circle when it is rotated around the center of the tool. Assuming that the original center of the flange is A1 (X1, Y1), it is rotated around the Y axis by angle α (i.e. the W of the current position of the robot arm), and the center of the circuit board A2 (X2, Y2) is rotated by angle w1 based on A1, then according to the circle formula, we have:

[0072]

[0073] Therefore, by adding Δx and Δy to the original coordinates, the tool can be rotated in the world coordinate system, that is, the coordinate positions of the center point x and y of the circuit board can be changed to compensate for the tilt angle rotation of w to achieve coordinate conversion. Specifically, according to the above conversion relationship, when the coordinates of the center point of the circuit board are identified as A(x, y, z, u, v, w), based on the distance r from the center point of the flange to the center point of the circuit board (this is a fixed parameter) and the default tool angle θ, the coordinates on the X and Y axes are calculated as follows:

[0074] x1=xr(sin(θ+ω)-sinθ)

[0075] y1=yr(cosθ-cos(θ+ω))

[0076] In step 203 , the robot arm grabs the circuit board from the correction area according to the positioning result and places it in the labeling area.

[0077] In an embodiment of the present disclosure, the robotic arm obtains a new physical coordinate point B(x1, y1, z, u, v, w) according to the positioning result, moves the robotic arm to the coordinates of point B, descends to grasp the circuit board from the correction area, and moves to the placement labeling area according to the preset coordinate range of the labeling area. At this time, the robotic arm can descend in the correct posture. Because the center point is grasped according to the tilt angle of the board when grasping, when the robotic arm returns to the correct posture, the circuit board will also be corrected, and the circuit board is placed to complete the subsequent labeling action.

[0078] In an embodiment of the present disclosure, during the process of the robotic arm grasping and moving the circuit board, it is also necessary to distinguish the actions of the rotation grasping tool that need to be concerned, that is, when there is physical interference during the process of grasping and moving the circuit board, coordinate A is used as the grasping coordinate instead of coordinate B. The method for determining whether there is physical interference is specifically as follows: The specification parameters of the robotic arm include: the arm length of the robotic arm, the rotation angles of each axis and the interference angle, and the movement range of the robotic arm. Obtain the coordinate position information mapped by the specification parameters of the robotic arm, the disordered area, the correction area, and the labeling area in the robotic arm control system, and calculate the preset trajectory of the robotic arm moving between the disordered area, the correction area, and the labeling area based on the specification parameters and the position information. This trajectory can be obtained through computer simulation;

[0079] Obtain the environmental information of the disordered area, the correction area, and the labeling area, where the environmental information is whether there are other devices and equipment in the vicinity of the disordered area, the correction area, and the labeling area. If so, use their corresponding physical coordinates as the environmental information. If not, there is no environmental information. Based on the environmental information and the preset trajectory, determine whether there will be interference when the robotic arm moves.

[0080] Figure 3 The figure shows a schematic diagram of a workpiece moving device based on image recognition according to an embodiment of the present disclosure, as Figure 3 shown, a workpiece moving device based on image recognition according to an embodiment of the present disclosure includes:

[0081] A first image recognition unit 301, configured to obtain a first image and first physical coordinate data of a first area, and recognize a first central physical coordinate and a first rotation angle of a workpiece center point in the first area based on the first image.

[0082] The first image recognition unit 301 is also used to identify the second shadow area in the first image based on the grayscale information of the first image, and determine the center point of the workpiece in the first area based on the second shadow area; determine the second pixel coordinates of the center point of the workpiece in the first area based on the pixel information of the first image; convert the second pixel coordinates into the first center physical coordinates based on the first physical coordinate data and the first coordinate conversion algorithm; determine all pixel coordinates of the workpiece in the first area based on the second shadow area and the pixel information of the first image; convert all pixel coordinates into physical coordinates of the workpiece based on the first physical coordinate data and the first coordinate conversion algorithm, and determine a first rotation angle of the workpiece based on the physical coordinates of the workpiece.

[0083] The coordinate compensation calculation unit 302 is used to obtain the first parameter data of the robot arm, calculate the coordinate compensation value of the workpiece based on the first parameter data, the first rotation angle and the first center physical coordinate, and modify the first center physical coordinate based on the coordinate compensation value to obtain the grasping coordinate.

[0084] The coordinate compensation calculation unit 302 is also used to: the first parameter data includes the horizontal distance from the center point of the robot flange to the robot grasping point and the second angle data of the robot; calculate the horizontal coordinate compensation value of the first center coordinate based on the sine function, the horizontal coordinate of the first center physical coordinate, the horizontal distance, the second angle data and the first rotation angle; calculate the vertical coordinate compensation value of the first center coordinate based on the cosine function, the vertical coordinate of the first center physical coordinate, the horizontal distance, the second angle data and the first rotation angle; determine the coordinate compensation value based on the horizontal coordinate compensation value and the vertical coordinate compensation value.

[0085] The robot control unit 303 is configured to grasp the workpiece based on the grasping coordinates, move to above the second area, rotate the flange in the robot based on the first parameter data, and place the workpiece in the second area based on the rotated angle.

[0086] The second image recognition unit 304 is used to obtain the second image and second physical coordinate data of the third area, and recognize the second center physical coordinates of the center point of the workpiece in the third area based on the second image; the robotic arm grabs the workpiece based on the second center physical coordinates and moves the workpiece to the first area for placement.

[0087] The second image recognition unit 304 is also used to identify a first shadow area in the second image based on the grayscale information of the second image, and determine the center point of the workpiece in the third area based on the first shadow area; determine the first pixel coordinates of the center point of the workpiece in the third area based on the pixel information of the second image; and convert the first pixel coordinates into second center physical coordinates based on the second physical coordinate data and the first coordinate conversion algorithm.

[0088] The interference measurement unit 305 is configured to obtain the specification parameters of the robotic arm, and the position information of the first area, the second area, and the third area, calculate a preset trajectory for the robotic arm to move between the first area, the second area, and the third area based on the specification parameters and the position information; obtain the environmental information of the first area, the second area, and the third area, and determine whether interference will occur when the robotic arm moves based on the environmental information and the preset trajectory; when no interference occurs, generate a coordinate modification instruction, and in response to the coordinate modification instruction, modify the first central physical coordinate based on the coordinate compensation value; when interference occurs, use the first central coordinate as the grasping coordinate.

[0089] In an exemplary embodiment, the first image recognition unit 301, the coordinate compensation calculation unit 302, the robotic arm control unit 303, the second image recognition unit 304, the interference measurement unit 305, etc. can be implemented by one or more central processing units (CPUs, Central Processing Unit), graphics processing units (GPUs, Graphics Processing Unit), application specific integrated circuits (ASICs, Application Specific Integrated Circuit), DSPs, programmable logic devices (PLDs, Programmable Logic Device), complex programmable logic devices (CPLDs, Complex Programmable LogicDevice), field programmable gate arrays (FPGAs, Field-Programmable Gate Array), general purpose processors, controllers, microcontroller units (MCUs, Micro Controller Unit), microprocessors (Microprocessor), or other electronic components.

[0090] Regarding the device in the above embodiments, the specific manners in which each module and unit perform operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0091] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0092] Figure 4A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0093] like Figure 4 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0094] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0095] The computing unit 801 can be a variety of general and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as a workpiece movement method based on image recognition. For example, in some embodiments, a workpiece movement method based on image recognition can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the workpiece movement method based on image recognition described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute a workpiece movement method based on image recognition in any other appropriate manner (for example, by means of firmware).

[0096] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0097] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0098] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0099] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic, speech, or tactile input).

[0100] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0101] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is generated by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0102] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitations are imposed herein.

[0103] In addition, the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise specifically defined.

[0104] As described above, this is only the specific implementation manner of this disclosure, but the protection scope of this disclosure is not limited thereto. Any person skilled in the art within the technical scope disclosed in this disclosure can easily think of changes or substitutions, which should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be subject to the protection scope of the claims.

Claims

1. A workpiece moving method based on image recognition, characterized in that, The method comprises: Acquire a first image and first physical coordinate data of a first area, and identify a first central physical coordinate and a first rotation angle of a center point of a workpiece in the first area based on the first image; Acquiring first parameter data of the robotic arm, calculating a coordinate compensation value of the workpiece based on the first parameter data, the first rotation angle, and the first center physical coordinate, and modifying the first center physical coordinate based on the coordinate compensation value to obtain a grasping coordinate; The workpiece is grasped based on the grasping coordinates, moved to above the second area, the flange in the robot arm is rotated based on the first parameter data, and the workpiece is placed in the second area based on the rotated angle.

2. The workpiece moving method based on image recognition according to claim 1, wherein The method further comprises: Acquire a second image and second physical coordinate data of a third area, and identify a second center physical coordinate of a center point of the workpiece in the third area based on the second image; The robot arm grabs the workpiece based on the second center physical coordinates and moves the workpiece to the first area for placement.

3. The workpiece moving method based on image recognition according to claim 2, characterized in that: The identifying, based on the second image, a second central physical coordinate of the center point of the workpiece in the third area includes: identifying a first shadow area in the second image based on grayscale information of the second image, and determining a center point of the workpiece in the third region based on the first shadow area; determining a first pixel coordinate of a center point of the workpiece in the third area based on pixel information of the second image; The first pixel coordinates are converted into second center physical coordinates based on the second physical coordinate data and a first coordinate conversion algorithm.

4. A workpiece moving method based on image recognition according to claim 1, characterized in that, The identifying, based on the first image, a first central physical coordinate and a first rotation angle of a center point of the workpiece in the first area includes: identifying a second shadow area in the first image based on grayscale information of the first image, and determining a center point of the workpiece in the first area based on the second shadow area; determining a second pixel coordinate of a center point of the workpiece in the first area based on pixel information of the first image; converting the second pixel coordinates into first center physical coordinates based on the first physical coordinate data and a first coordinate conversion algorithm; determining all pixel coordinates of the workpiece within the first area based on the second shadow area and pixel information of the first image; The coordinates of all pixels are converted into physical coordinates of a workpiece based on the first physical coordinate data and a first coordinate conversion algorithm, and a first rotation angle of the workpiece is determined based on the physical coordinates of the workpiece.

5. A workpiece moving method based on image recognition according to claim 1, characterized in that The calculating the coordinate compensation value of the workpiece based on the first parameter data, the first rotation angle and the first center physical coordinate includes: The first parameter data includes the horizontal distance from the center point of the robot arm flange to the gripping point of the robot arm and the second angle data of the robot arm; Calculating a horizontal coordinate compensation value of the first center coordinate based on a sine function, the horizontal coordinate of the first center physical coordinate, the horizontal distance, the second angle data, and the first rotation angle; Calculate the vertical coordinate compensation value of the first central coordinate based on the cosine function, the vertical coordinate of the first central physical coordinate, the horizontal distance, the second angle data, and the first rotation angle; Determine the coordinate compensation value based on the horizontal coordinate compensation value and the vertical coordinate compensation value.

6. The workpiece moving method based on image recognition according to claim 2, wherein The method further includes: Obtain the specification parameters of the robotic arm, the position information of the first region, the second region, and the third region, and calculate a preset trajectory for the robotic arm to move between the first region, the second region, and the third region based on the specification parameters and the position information; Obtain the environmental information of the first region, the second region, and the third region, and determine whether interference will occur when the robotic arm moves based on the environmental information and the preset trajectory; When no interference occurs, generate a coordinate modification instruction, and in response to the coordinate modification instruction, modify the first central physical coordinate based on the coordinate compensation value; When interference occurs, use the first central coordinate as the grasping coordinate.

7. A workpiece moving device based on image recognition, characterized in that, The device includes: A first image recognition unit, configured to obtain a first image and first physical coordinate data of a first region, and recognize a first central physical coordinate and a first rotation angle of a workpiece center point in the first region based on the first image; A coordinate compensation calculation unit, configured to obtain first parameter data of the robotic arm, calculate a coordinate compensation value of the workpiece based on the first parameter data, the first rotation angle, and the first central physical coordinate, and modify the first central physical coordinate based on the coordinate compensation value to obtain a grasping coordinate; A robotic arm control unit, configured to grasp the workpiece based on the grasping coordinate, move above the second region, rotate a flange in the robotic arm based on the first parameter data, and place the workpiece in the second region based on the rotated angle.

8. The workpiece moving device based on image recognition according to claim 7, characterized in that, The device further includes: A second image recognition unit, configured to obtain a second image and second physical coordinate data of a third region, and recognize a second central physical coordinate of a workpiece center point in the third region based on the second image; the robotic arm grasps the workpiece based on the second central physical coordinate and moves the workpiece to the first region for placement; The second image recognition unit is further configured to recognize a first shadow area in the second image based on the gray-scale information of the second image, determine the workpiece center point in the third region based on the first shadow area; determine a first pixel coordinate of the workpiece center point in the third region based on the pixel information of the second image; and convert the first pixel coordinate into a second central physical coordinate based on the second physical coordinate data and a first coordinate conversion algorithm; The first image recognition unit is further configured to recognize a second shadow area in the first image based on the gray information of the first image, determine a workpiece center point in the first area based on the second shadow area; determine a second pixel coordinate of the workpiece center point in the first area based on the pixel information of the first image; convert the second pixel coordinate into a first center physical coordinate based on the first physical coordinate data and the first coordinate conversion algorithm; determine all pixel coordinates of the workpiece in the first area based on the second shadow area and the pixel information of the first image; convert all the pixel coordinates into workpiece physical coordinates based on the first physical coordinate data and the first coordinate conversion algorithm, and determine a first rotation angle of the workpiece based on the workpiece physical coordinates. The coordinate compensation calculation unit is further configured to the first parameter data includes the horizontal distance from the center point of the robot flange to the robot gripping point and the second angle data of the robot; calculate an abscissa compensation value of the first center coordinate based on the sine function, the abscissa of the first center physical coordinate, the horizontal distance, the second angle data and the first rotation angle; calculate an ordinate compensation value of the first center coordinate based on the cosine function, the ordinate of the first center physical coordinate, the horizontal distance, the second angle data and the first rotation angle; determine the coordinate compensation value based on the abscissa compensation value and the ordinate compensation value. The interference measurement unit is configured to obtain the specification parameters of the robot, and the position information of the first area, the second area and the third area, calculate a preset trajectory for the robot to move between the first area, the second area and the third area based on the specification parameters and the position information; obtain the environmental information of the first area, the second area and the third area, and determine whether interference will occur when the robot moves based on the environmental information and the preset trajectory; when no interference occurs, generate a coordinate modification instruction, and in response to the coordinate modification instruction, modify the first center physical coordinate based on the coordinate compensation value; when interference occurs, use the first center coordinate as the gripping coordinate.

9. An electronic device, characterized in that, Comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1-6.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause a computer to execute the method according to any one of claims 1-6.

Images