Robotic arm sorting control method, medium, equipment and device

Abstract

The present invention discloses a robotic arm sorting control method, medium, equipment and device, which includes: obtaining a first target object image through a camera installed on the robotic arm; performing edge detection to obtain position information of the target object, and obtaining a region image corresponding to the target object based on the position information, and inputting the region image into a pre-trained label recognition model to perform label recognition; when label recognition fails, controlling the robotic arm to move based on the position information corresponding to the target object; when the distance between the camera and the target object is less than or equal to a preset distance threshold, obtaining a second target object image, performing label recognition on the second target object image to obtain label information corresponding to the target object, and sorting and classifying the target objects based on the label information; when object information recognition fails, the robotic arm can be effectively controlled to perform secondary recognition of the object information, thereby improving the success rate of object sorting.

Description

Technical Field

[0001] The present application relates to the field of robotic arm control technology, and in particular to a robotic arm sorting control method, medium, equipment and device. Background Art

[0002] With the development of society, vision-based automatic grasping and sorting by robotic arms have been widely used in all walks of life.

[0003] In related technologies, when controlling a robotic arm, most of the time, a fixed camera is used to take a picture of the object to be grasped, and the camera is controlled to grasp the object based on the recognition result of the image obtained from the picture. However, due to factors such as camera accuracy and on-site environment, image recognition failure often occurs during the automatic sorting of objects by the robotic arm, which in turn causes the robotic arm to fail to sort the target object. Summary of the Invention

[0004] The present invention aims to at least partially address one of the technical problems in the related art. To this end, one object of the present invention is to provide a robotic arm sorting control method that effectively controls the robotic arm when object information recognition fails, allowing for a secondary recognition of the object information and improving the object sorting success rate.

[0005] In a first aspect, an embodiment of the present invention proposes a robotic arm sorting control method, comprising: acquiring a first target object image through a camera installed on the robotic arm; performing edge detection on the first target object image to obtain position information of the target object, and cutting the first target object image according to the position information to obtain a region image corresponding to the target object, and inputting the region image into a pre-trained label recognition model to perform label recognition on the region image through the label recognition model; when the label recognition fails, controlling the robotic arm to move according to the position information corresponding to the target object so that the distance between the camera and the target object is less than or equal to a preset distance threshold; when the distance between the camera and the target object is less than or equal to the preset distance threshold, acquiring a second target object image, and performing label recognition on the second target object image to obtain label information corresponding to the target object, and sorting and classifying the target objects according to the label information.

[0006] According to an embodiment of the present invention, a robotic arm sorting control method first acquires a first target object image using a camera mounted on the robotic arm. Next, edge detection is performed on the first target object image to obtain position information of the target object, and the first target object image is segmented based on the position information to obtain a region image corresponding to the target object. The region image is input into a pre-trained label recognition model to perform label recognition on the region image using the label recognition model. Next, when label recognition fails, the robotic arm is controlled to move based on the position information corresponding to the target object so that the distance between the camera and the target object is less than or equal to a preset distance threshold. Next, when the distance between the camera and the target object is less than or equal to the preset distance threshold, a second target object image is acquired, and label recognition is performed on the second target object image to obtain label information corresponding to the target object. The target objects are then sorted and classified based on the label information. This allows the robotic arm to be effectively controlled to perform secondary recognition of the object information when object information recognition fails, thereby improving the object sorting success rate.

[0007] In some embodiments, before controlling the robotic arm to move according to the position information corresponding to the target item, the control method further includes: obtaining the posture information of the robotic arm, and judging whether the robotic arm can complete the sorting and classification of the target item based on the posture information of the robotic arm and the position information of the target item; when the robotic arm cannot complete the sorting and classification of the target item, the robotic arm sends the area image corresponding to the target item to the cloud server, so that the cloud server controls the next robotic arm to prioritize sorting and classification of the target item.

[0008] In some embodiments, the cloud server controls the next robotic arm to prioritize sorting and categorizing the target item, including: the cloud server sends a prioritized sorting instruction corresponding to the target item to the next robotic arm, wherein the prioritized sorting instruction includes a region image corresponding to the target item; the next robotic arm obtains a third target item image and compares the region image with the third target item image to determine whether the third target item image contains a target item for which the previous robotic arm failed to successfully identify the label and sort; and if so, prioritizes sorting and categorizing the target item for which the previous robotic arm failed to successfully identify the label and sort.

[0009] In some embodiments, edge detection is performed on the first target object image to obtain location information of the target object, including: segmenting and filtering the first target object image to obtain a preprocessed image; calculating the gradient of the preprocessed image to obtain a gradient magnitude image, and performing non-maximum suppression on the gradient magnitude image to obtain a set of candidate edge points; and screening the candidate edge points in the set of candidate edge points using a threshold method to obtain a set of valid edge points.

[0010] In some embodiments, segmenting the first target item image includes: dividing the first target item image to obtain multiple neighborhoods; calculating the local standard deviation and local mean corresponding to each neighborhood, and calculating the local threshold corresponding to the neighborhood based on the local standard deviation and the local mean, and performing segmentation processing based on the local threshold to obtain a local segmentation result; splicing the local segmentation results to obtain a final segmented image.

[0011] In some embodiments, the first target item image is filtered according to the following formula:

[0012]

[0013] Among them, g(x,y) represents the image after filtering, represents the inverse discrete Fourier transform, F(u,v) represents the discrete Fourier transform of the first target object image, and H(u,v) represents a filter function.

[0014] To achieve the above-mentioned object, the second aspect of the present invention proposes a computer-readable storage medium on which a robot arm sorting control program is stored. When the robot arm sorting control program is executed by a processor, the robot arm sorting control method described above is implemented.

[0015] According to an embodiment of the present invention, a computer-readable storage medium stores a robotic arm sorting control program, so that when a processor executes the robotic arm sorting control program, the processor implements the robotic arm sorting control method described above. This allows the robotic arm to be effectively controlled when item information recognition fails, so as to perform a secondary recognition of item information and improve the success rate of item sorting.

[0016] To achieve the above-mentioned purpose, the third aspect of the present invention proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the robotic arm sorting control method as described above is implemented.

[0017] According to the computer device of an embodiment of the present invention, the robot arm sorting control program is stored in a memory, so that when the processor executes the robot arm sorting control program, the robot arm sorting control method as described above is implemented. This allows the robot arm to be effectively controlled when object information recognition fails, so as to perform a secondary recognition of the object information and improve the success rate of object sorting.

[0018] To achieve the above objectives, a fourth aspect of the present invention provides a robotic arm sorting control device, comprising: an acquisition module for acquiring a first target object image using a camera mounted on the robotic arm; an edge detection module for performing edge detection on the first target object image to obtain position information of the target object; a label recognition module for segmenting the first target object image based on the position information to obtain a region image corresponding to the target object, and inputting the region image into a pre-trained label recognition model to perform label recognition on the region image using the label recognition model; a control module for controlling the robotic arm to move based on the position information corresponding to the target object when label recognition fails, so that the distance between the camera and the target object is less than or equal to a preset distance threshold; the control module is further configured to control the acquisition module to acquire a second target object image when the distance between the camera and the target object is less than or equal to the preset distance threshold, and to control the label recognition module to perform label recognition on the second target object image to obtain label information corresponding to the target object, and to sort and classify the target objects based on the label information.

[0019] According to an embodiment of the present invention, a robotic arm sorting control device is provided with an acquisition module for acquiring a first target object image using a camera mounted on the robotic arm; an edge detection module for performing edge detection on the first target object image to obtain position information of the target object; a label recognition module for segmenting the first target object image based on the position information to obtain a region image corresponding to the target object, and inputting the region image into a pre-trained label recognition model to perform label recognition on the region image using the label recognition model; a control module for controlling the robotic arm to move based on the position information corresponding to the target object when label recognition fails, so that the distance between the camera and the target object is less than or equal to a preset distance threshold; the control module is further configured to control the acquisition module to acquire a second target object image when the distance between the camera and the target object is less than or equal to the preset distance threshold, and to control the label recognition module to perform label recognition on the second target object image to obtain label information corresponding to the target object, and to sort and classify the target objects based on the label information. In this way, when object information recognition fails, the robotic arm is effectively controlled to perform secondary object information recognition, thereby improving the object sorting success rate.

[0020] In some embodiments, the control module is also used to obtain the posture information of the robotic arm, and determine whether the robotic arm can complete the sorting and classification of the target item based on the posture information of the robotic arm and the location information of the target item; and when the robotic arm cannot complete the sorting and classification of the target item, the robotic arm sends the area image corresponding to the target item to the cloud server, so that the cloud server controls the next robotic arm to prioritize sorting and classification of the target item.

[0021] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of a robotic arm sorting control method according to an embodiment of the present invention;

[0023] Figure 2 is a block diagram of a computer device according to an embodiment of the present invention;

[0024] Figure 3 4 is a block diagram of a robotic arm sorting control device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0026] The following describes a robot arm sorting control method according to an embodiment of the present invention with reference to the accompanying drawings.

[0027] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a robot arm sorting control method according to an embodiment of the present invention. Figure 1 As shown, the robot arm sorting control method includes the following steps:

[0028] S101, acquiring an image of a first target object through a camera installed on a robotic arm.

[0029] That is, the camera is fixedly mounted on the fixed section of the robotic arm, so that the image of the target object corresponding to the object to be sorted is obtained through the camera fixedly mounted on the robotic arm.

[0030] S102: performing edge detection on the first target object image to obtain position information of the target object, cutting the first target object image according to the position information to obtain a region image corresponding to the target object, and inputting the region image into a pre-trained label recognition model to perform label recognition on the region image through the label recognition model.

[0031] In some embodiments, edge detection is performed on the first target object image to obtain position information of the target object, including: segmenting and filtering the first target object image to obtain a preprocessed image; calculating the gradient of the preprocessed image to obtain a gradient magnitude image, and performing non-maximum suppression on the gradient magnitude image to obtain a set of candidate edge points; and screening the candidate edge points in the set of candidate edge points using a threshold method to obtain a set of valid edge points.

[0032] It's understandable that gradient calculation can enhance image edges, but it also increases image noise. Therefore, before performing gradient calculation, the first target object image is segmented and filtered to obtain a preprocessed image. This removes image noise before gradient calculation to prevent image noise from affecting the accuracy of subsequent calculations. By performing non-maximum suppression on the gradient magnitude image, the corresponding local maxima are obtained, resulting in a set of candidate edge points. Then, using the thresholding method, the corresponding set of valid edge points is obtained. Specifically, dual thresholding and connectivity analysis can be used to obtain the valid edge point set.

[0033] In some embodiments, segmentation processing of the first target object image includes: dividing the first target object image to obtain multiple neighborhoods; calculating the local standard deviation and local mean corresponding to each neighborhood, and calculating the local threshold corresponding to the neighborhood based on the local standard deviation and local mean, and performing segmentation processing based on the local threshold to obtain a local segmentation result; splicing the local segmentation results to obtain a final segmented image.

[0034] It is understandable that if a preset fixed threshold is used for segmentation, noise may cause the grayscale values of some pixels to increase or decrease. Therefore, the above method of dividing the first target object image into multiple neighborhoods and calculating the local thresholds corresponding to the neighborhoods can effectively ensure the effectiveness of local segmentation, thereby improving the overall image segmentation effect.

[0035] In some embodiments, the first target item image is filtered according to the following formula:

[0036]

[0037] Among them, g(x,y) represents the image after filtering, represents the inverse discrete Fourier transform, F(u,v) represents the discrete Fourier transform of the first target object image, and H(u,v) represents the filter function.

[0038] That is, a discrete Fourier transform is performed on the first target object image, and then its inverse transform is calculated to complete the filtering process of the first target object image.

[0039] S103: When tag recognition fails, the robot arm is controlled to move according to the position information corresponding to the target object so that the distance between the camera and the target object is less than or equal to a preset distance threshold.

[0040] That is to say, if the tag recognition is successful, the target item is sorted and classified directly according to the tag information obtained by recognition; if the tag recognition fails, the robotic arm is controlled to move so that the distance between the camera installed on the robotic arm and the target item is less than or equal to the preset distance threshold; so that the target item is photographed and recognized a second time to avoid the inability to sort the item due to recognition failure.

[0041] There are many ways to set the tag information.

[0042] As an example, the tag information may be a QR code information, in which the item attributes, owner information, etc. corresponding to the target item are stored.

[0043] As another example, the label information is text information, which includes information such as the sender, recipient, mobile phone number, and address of the target item.

[0044] In some embodiments, before controlling the robotic arm to move according to the position information corresponding to the target object, it also includes: obtaining the posture information of the robotic arm, and judging whether the robotic arm can complete the sorting and classification of the target object based on the posture information of the robotic arm and the position information of the target object; when the robotic arm cannot complete the sorting and classification of the target object, the robotic arm sends the area image corresponding to the target object to the cloud server, so that the cloud server controls the next robotic arm to prioritize sorting and classification of the target object.

[0045] It can be understood that after the installation is completed, each robotic arm is provided with a corresponding working area (corresponding to a part of the conveyor belt); the items to be sorted are moved by the loading of the conveyor belt. When the label recognition fails, it takes a certain amount of time for the robotic arm to move to the position corresponding to the target item in the current posture at the current moment; if the position corresponding to the target item is close to the edge of the working area of the current robotic arm, the current robotic arm may not be able to complete the sorting of the target item even if it moves immediately. Therefore, before controlling the robotic arm to move, the current situation is first judged to determine whether the robotic arm can complete the sorting of the target item at the current moment. If it cannot be completed at the current moment, the robotic arm will send the area image corresponding to the target item to the cloud server, so that the cloud server can control the next robotic arm to prioritize the sorting of the target item based on the received area image.

[0046] As an example, first, the expected posture of the robot arm is determined according to the position information of the target object; then, trajectory planning is performed according to the expected posture of the robot arm and the current posture information of the robot arm to determine the motion trajectory of the robot arm to grasp the target object in the current posture; then, calculation is performed based on the trajectory planning result to obtain a first estimated time required for the robot arm to grasp the target object, and then, the sum of the first estimated time and the second estimated time required for the robot arm to perform a second shooting and identification of the target object is used as the final estimated time; then, the remaining time from the current time when the target object leaves the working area of the robot arm is calculated according to the position information of the target object, the working area position information of the robot arm and the movement speed of the conveyor belt; then, it is determined whether the final estimated time is greater than the remaining time; if so, it means that the robot arm cannot currently complete the grasping of the target object; if not, it means that the robot arm can currently complete the grasping of the target object.

[0047] In some embodiments, the cloud server controls the next robotic arm to prioritize sorting and classifying the target items, including: the cloud server sends a priority sorting instruction corresponding to the target item to the next robotic arm, wherein the priority sorting instruction includes an area image corresponding to the target item; the next robotic arm obtains a third target item image, and compares the area image with the third target item image to determine whether the third target item image contains the target item that the previous robotic arm failed to successfully identify and sort, and when the judgment result is yes, prioritizes sorting and classifying the target item that the previous robotic arm failed to successfully identify and sort.

[0048] That is, after the robotic arm receives the priority sorting instruction sent by the cloud server, each time the robotic arm obtains the third target object image through the camera installed on it, it will compare the area image in the priority sorting instruction with the third target object image to determine whether the third target object image contains the target object that the previous robotic arm failed to successfully identify and sort; if so, the target object will be prioritized for sorting and classification; if not, the sorting and classification will be carried out according to normal steps.

[0049] S104, when the distance between the camera and the target object is less than or equal to a preset distance threshold, obtaining a second target object image, performing label recognition on the second target object image to obtain label information corresponding to the target object, and sorting and classifying the target object according to the label information.

[0050] In summary, according to an embodiment of the present invention, a robotic arm sorting control method first acquires a first target object image using a camera mounted on the robotic arm. Next, edge detection is performed on the first target object image to obtain position information of the target object, and the first target object image is segmented based on the position information to obtain a region image corresponding to the target object. The region image is then input into a pre-trained label recognition model to perform label recognition on the region image using the label recognition model. Then, if label recognition fails, the robotic arm is controlled to move based on the position information corresponding to the target object so that the distance between the camera and the target object is less than or equal to a preset distance threshold. Next, if the distance between the camera and the target object is less than or equal to the preset distance threshold, a second target object image is acquired, and label recognition is performed on the second target object image to obtain label information corresponding to the target object. The target objects are then sorted and classified based on the label information. This allows the robotic arm to be effectively controlled to perform secondary recognition of the object information when object information recognition fails, thereby improving the object sorting success rate.

[0051] To implement the above embodiment, an embodiment of the present invention provides a computer-readable storage medium on which a robotic arm sorting control program is stored. When the robotic arm sorting control program is executed by a processor, the robotic arm sorting control method described above is implemented.

[0052] According to an embodiment of the present invention, a computer-readable storage medium stores a robotic arm sorting control program, so that when a processor executes the robotic arm sorting control program, the processor implements the robotic arm sorting control method described above. This allows the robotic arm to be effectively controlled when item information recognition fails, so as to perform a secondary recognition of item information and improve the success rate of item sorting.

[0053] In order to implement the above embodiment, the embodiment of the present invention provides a computer device, such as Figure 2 As shown, the computer device 200 includes a memory 201 and a processor 202, and a computer program stored in the memory 201 and executable on the processor 202. When the processor 202 executes the program, the robot arm sorting control method described above is implemented.

[0054] In summary, according to the computer device of an embodiment of the present invention, the robot arm sorting control program is stored in the memory, so that when the processor executes the robot arm sorting control program, the robot arm sorting control method as described above is implemented, thereby achieving effective control of the robot arm when the object information recognition fails, so as to perform a secondary recognition of the object information and improve the success rate of object sorting.

[0055] In order to implement the above embodiment, the embodiment of the present invention proposes a robot arm sorting control device, such as Figure 3 As shown, the robotic arm sorting control device includes: an acquisition module 10, an edge detection module 20, a label recognition module 30 and a control module 40.

[0056] The acquisition module 10 is configured to acquire an image of a first target object through a camera mounted on the robotic arm;

[0057] an edge detection module 20 for performing edge detection on the first target object image to obtain position information of the target object;

[0058] The label recognition module 30 is configured to segment the first target object image according to the position information to obtain a region image corresponding to the target object, and input the region image into a pre-trained label recognition model to perform label recognition on the region image through the label recognition model;

[0059] The control module 40 is used to control the movement of the robotic arm according to the position information corresponding to the target object when the tag recognition fails, so that the distance between the camera and the target object is less than or equal to a preset distance threshold;

[0060] The control module 40 is also used to control the acquisition module to acquire a second target object image when the distance between the camera and the target object is less than or equal to a preset distance threshold, and to control the label recognition module to perform label recognition on the second target object image to obtain label information corresponding to the target object, and to sort and classify the target object according to the label information.

[0061] In some embodiments, the control module 40 is also used to obtain the posture information of the robotic arm, and determine whether the robotic arm can complete the sorting and classification of the target item based on the posture information of the robotic arm and the location information of the target item. When the robotic arm cannot complete the sorting and classification of the target item, the robotic arm sends the area image corresponding to the target item to the cloud server so that the cloud server controls the next robotic arm to prioritize sorting and classification of the target item.

[0062] In some embodiments, the device also includes a preprocessing module, and the edge detection module 20 is used to perform segmentation processing and filtering processing on the first target object image to obtain a preprocessed image; calculate the gradient of the preprocessed image to obtain a gradient amplitude image, and perform non-maximum suppression on the gradient amplitude image to obtain a set of candidate edge points; and screen the candidate edge points in the candidate edge point set by a threshold method to obtain a set of valid edge points.

[0063] In some embodiments, the edge detection module 20 is also used to divide the first target object image to obtain multiple neighborhoods; calculate the local standard deviation and local mean corresponding to each neighborhood, and calculate the local threshold corresponding to the neighborhood based on the local standard deviation and local mean, and perform segmentation processing based on the local threshold to obtain a local segmentation result; splice the local segmentation results to obtain a final segmented image.

[0064] In some embodiments, the first target item image is filtered according to the following formula:

[0065]

[0066] Among them, g(x,y) represents the image after filtering, represents the inverse discrete Fourier transform, F(u,v) represents the discrete Fourier transform of the first target object image, and H(u,v) represents the filter function.

[0067] In summary, according to the robotic arm sorting control device of the embodiment of the present invention, an acquisition module is provided for acquiring a first target object image through a camera installed on the robotic arm; an edge detection module is provided for performing edge detection on the first target object image to obtain position information of the target object; a label recognition module is provided for cutting the first target object image according to the position information to obtain a region image corresponding to the target object, and inputting the region image into a pre-trained label recognition model to perform label recognition on the region image through the label recognition model; a control module is provided for, when the label recognition fails, performing label recognition on the region image according to the target object. The control module is further configured to control the robot arm to move according to the corresponding position information so that the distance between the camera and the target object is less than or equal to a preset distance threshold; the control module is further configured to control the acquisition module to acquire a second target object image when the distance between the camera and the target object is less than or equal to the preset distance threshold, and control the label recognition module to perform label recognition on the second target object image to obtain label information corresponding to the target object, and sort and classify the target objects according to the label information; thereby achieving effective control of the robot arm to perform secondary recognition of the object information when the object information recognition fails, thereby improving the success rate of object sorting.

[0068] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or in conjunction with such instruction execution system, apparatus, or device. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion having one or more wires (electronic device), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0069] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0070] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0071] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0072] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0073] In the present invention, unless otherwise specified or limited, the terms "installed," "connected," "connect," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection; direct connection, or indirect connection through an intermediate medium; internal communication between two components, or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0074] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediary. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0075] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A robotic arm sorting control method, characterized in that: The following steps are involved: Acquire an image of a first target object through a camera installed on the robotic arm; performing edge detection on the first target object image to obtain position information of the target object, segmenting the first target object image according to the position information to obtain a region image corresponding to the target object, and inputting the region image into a pre-trained label recognition model to perform label recognition on the region image through the label recognition model; When the tag recognition fails, controlling the robot arm to move according to the position information corresponding to the target object so that the distance between the camera and the target object is less than or equal to a preset distance threshold; When the distance between the camera and the target object is less than or equal to a preset distance threshold, a second target object image is obtained, and label recognition is performed on the second target object image to obtain label information corresponding to the target object, and the target object is sorted and classified according to the label information.

2. The robot arm sorting control method according to claim 1, characterized in that: Before controlling the robot arm to move according to the position information corresponding to the target object, the method further includes: Acquiring the posture information of the robotic arm, and judging whether the robotic arm can complete the sorting and classification of the target object based on the posture information of the robotic arm and the location information of the target object; When the items to be sorted are moved by the loading on the conveyor belt, and it takes a certain amount of time for the robotic arm to move to the position corresponding to the target item in the current posture at the current moment, the robotic arm cannot complete the sorting of the target item even if it moves immediately, then it is determined that the robotic arm cannot complete the sorting and classification of the target item. The robotic arm sends the area image corresponding to the target item to the cloud server so that the cloud server can control the next robotic arm to prioritize the sorting and classification of the target item.

3. The robot arm sorting control method according to claim 2, characterized in that: The cloud server controls the next robotic arm to prioritize and sort the target items, including: The cloud server sends a priority sorting instruction corresponding to the target item to the next robotic arm, wherein the priority sorting instruction includes an area image corresponding to the target item; The next robotic arm obtains a third target object image and compares the area image with the third target object image to determine whether the third target object image contains target objects that the previous robotic arm failed to successfully identify with a label and did not complete sorting, and if the judgment result is yes, the target objects that the previous robotic arm failed to successfully identify with a label and did not complete sorting are prioritized for sorting and classification.

4. The robot arm sorting control method according to claim 1, characterized in that: Performing edge detection on the first target object image to obtain location information of the target object includes: performing segmentation processing and filtering processing on the first target object image to obtain a preprocessed image; Calculating the gradient of the preprocessed image to obtain a gradient magnitude image, and performing non-maximum suppression on the gradient magnitude image to obtain a set of candidate edge points; The candidate edge points in the candidate edge point set are screened by a threshold method to obtain a valid edge point set.

5. The robot arm sorting control method according to claim 4, characterized in that: Segmenting the first target object image includes: Dividing the first target object image to obtain a plurality of neighborhoods; Calculating a local standard deviation and a local mean corresponding to each neighborhood, calculating a local threshold corresponding to the neighborhood based on the local standard deviation and the local mean, and performing segmentation processing based on the local threshold to obtain a local segmentation result; The local segmentation results are spliced to obtain a final segmented image.

6. The robot arm sorting control method according to claim 4, characterized in that: The first target object image is filtered according to the following formula: Among them, g(x,y) represents the image after filtering, represents the inverse discrete Fourier transform, F(u,v) represents the discrete Fourier transform of the first target object image, and H(u,v) represents a filter function.

7. A computer-readable storage medium, characterized in that A robotic arm sorting control program is stored thereon, and when the robotic arm sorting control program is executed by the processor, the robotic arm sorting control method according to any one of claims 1 to 6 is implemented.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the robot arm sorting control method according to any one of claims 1 to 6 is implemented.

9. A robotic arm sorting control device, characterized in that: include: an acquisition module, configured to acquire an image of a first target object through a camera mounted on the robotic arm; an edge detection module, configured to perform edge detection on the first target object image to obtain location information of the target object; a label recognition module, configured to segment the first target object image according to the position information to obtain a region image corresponding to the target object, and input the region image into a pre-trained label recognition model to perform label recognition on the region image through the label recognition model; a control module, configured to control the movement of the robotic arm according to the position information corresponding to the target object when the tag recognition fails, so that the distance between the camera and the target object is less than or equal to a preset distance threshold; The control module is further configured to control the acquisition module to acquire a second target object image when the distance between the camera and the target object is less than or equal to a preset distance threshold, and to control the label recognition module to perform label recognition on the second target object image to obtain label information corresponding to the target object, and to sort and classify the target object according to the label information.

10. The robot arm sorting control device according to claim 9, characterized in that: The control module is further configured to obtain the posture information of the robotic arm, and determine whether the robotic arm is capable of completing the sorting and classification of the target item based on the posture information of the robotic arm and the position information of the target item. Furthermore, when the item to be sorted is moved by the loading of a conveyor belt, and it takes a certain amount of time for the robotic arm to move to the position corresponding to the target item in its current posture, and the robotic arm cannot complete the sorting of the target item even if it moves immediately, it is determined that the robotic arm is unable to complete the sorting and classification of the target item, and the robotic arm sends an image of the area corresponding to the target item to a cloud server so that the cloud server controls the next robotic arm to prioritize sorting and classification of the target item.

Images