Medicine bag detection and grasping point positioning method based on contour convexity

Through the contour convexity detection and grab point positioning method, the problem of grab point positioning when the robot grabs small packaged medicine bags is solved, the efficiency and accuracy of the medicine bags are improved, the cost is reduced, and it is suitable for use in embedded systems.

CN113888570BActive Publication Date: 2024-12-03CHONGQING UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111081882.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-15
Publication Date
2024-12-03
Estimated Expiration
2041-09-15

AI Technical Summary

Technical Problem

In automated pharmacies, when a robot grabs a small package of medicine bags, it is difficult to accurately determine the grab point and position of the medicine bag, resulting in low gripping efficiency and high cost.

Method used

Using the medicine bag detection and grabbing point positioning method based on convexity of the convexity of the convexity of the convexity of the convexity of the convexity of the vertices, the image is collected by the camera for grayscale, filtering, and binarization, tracking the convexity of the convexity of the vertices, and fitting the convexity of the vertices, and judging the convexity of the vertices in the adjacent areas. Divide the medicine bag between the concave vertices, calculate the side length and area, and locate the most suitable medicine bag and its grasping point.

Benefits of technology

It effectively reduces the difficulty of splitting stacked medicine bags, realizes accurate grasping point positioning of medicine bags, improves grasping efficiency, and reduces labor costs. Compared with deep learning algorithms, it has low hardware requirements and short processing time, which is suitable for Embedded system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113888570B_ABST
    Figure CN113888570B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for detecting and locating grabbing points of medicine bags based on contour concavity, including the following steps: I) collecting images of medicine bags stacked on a sorting table by a camera; II) sequentially graying, median filtering, binarization, closing and opening the images; III) contour tracking of the binary image; IV) fitting the contour to obtain a polygonal contour; V) judging the concavity of the vertices of the polygonal contour; VI) medicine bag segmentation, where a medicine bag is segmented between every two concave vertices; VII) locating grabbing points of stacked medicine bags. Aiming at the actual situation that the images of stacked medicine bags are convex polygons after fitting, the present invention creatively utilizes the characteristic that the vertices of the junction of two adjacent medicine bags are concave vertices to segment the medicine bags, thereby reducing the segmentation difficulty of the stacked medicine bags, and on this basis, realizes the positioning of the final grabbing points of the medicine bags, solving the problem of selecting and locating the most suitable medicine bags for grabbing the stacked medicine bags at present.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of automated pharmacies, and in particular to a method for detecting medicine bags and locating grasping points when a mechanical arm grasps small-package medicine bags. Background Art

[0002] In recent years, the market share of small-packaged Chinese medicine has continued to increase, but most pharmacies still use traditional manual methods to dispense it. At a time when medical resources are relatively scarce, the daily prescription volume of tertiary hospitals and large Chinese medicine hospitals is 500-1500 prescriptions. Manual prescription dispensing is labor-intensive, inefficient, and has high labor costs.

[0003] Using a robot to replace manual medicine picking can improve medicine picking efficiency and reduce labor costs. However, how to find the most suitable medicine package among the medicine packages scattered and stacked on the sorting table and determine the location of the picking point is the key to the robot's ability to successfully pick up small-packaged Chinese medicine bags. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a medicine bag detection and grasping point positioning method based on contour convexity, so as to solve the technical problem of how to determine the most suitable medicine bag for grasping and the grasping point position when a robot arm grasps small-packaged Chinese medicine bags on a sorting table.

[0005] The present invention provides a method for detecting a medicine bag and locating a grasping point based on contour concavity and convexity, comprising the following steps:

[0006] 1) collecting images of medicine bags stacked on the sorting table through a camera arranged above the sorting table;

[0007] II) graying, median filtering, binarization, closing and opening the image captured by the camera in sequence;

[0008] III) performing contour tracking on the binary image processed in step II);

[0009] IV) fitting the outer contour to obtain a polygonal contour;

[0010] V) determining the concavity and convexity of the vertices of the polygonal contour;

[0011] VI)) Medicine bag segmentation: a medicine bag is segmented between two adjacent concave vertices of the polygonal contour;

[0012] VII) Position the grasping point of the stacked medicine bags: First, calculate the sum of the side lengths of each medicine bag after segmentation in the polygon contour obtained by fitting. Consider the medicine bag with the largest sum of side lengths as having the smallest overlapping area with other medicine bags, that is, the currently most suitable medicine bag for grasping. Then, obtain the length and width of the medicine bag from the contour fitting diagram of this medicine bag and restore the medicine bag to a rectangle. Next, find the midpoint of the longest side of the medicine bag in the fitting image, and then translate this midpoint along the direction perpendicular to the longest side into the medicine bag. The translation distance is two-fifths of the width of the medicine bag, and the translated point is the final grasping point.

[0013] Further, in the step III), the contour tracking includes determining the boundary points of the contour and finding the next contour point;

[0014] The determination of the boundary points of the contour includes the following steps:

[0015] a) In the binary image, the pixels with gray values of 0 and 1 are respectively called 0-pixel points and 1-pixel points. The pixel point in the i-th row and j-th column is denoted as (i, j), and if f[i][j] represents the gray value of this point, then:

[0016]

[0017] b) Scan the binary image row by row. When encountering a 1-pixel during the scanning process, count the number of 1-pixel points in its 8-connected direction and 4-connected direction, and denote them as num_8 and num_4 respectively. Then:

[0018]

[0019] c) Judge the boundary point situation:

[0020] (1) If 1 < num_8[i][j] < 4, then this point is a boundary point, and let f[i][j] = 1;

[0021] (2) If 3 < num_8[i][j] < 8, continue to judge the value of num_4[i][j]. If num_4[i][j] ≠ 4, then this point is a boundary point, and let f[i][j] = 1;

[0022] (3) In other situations except for situation (1) and situation (2), this point is not a boundary point, and let f[i][j] = 0;

[0023] d) Remove the interference points among the image boundary points obtained in step c): First, define the following variables

[0024]

[0025]

[0026]

[0027]

[0028]

[0029]

[0030] Then follow the steps below to eliminate interference points:

[0031] Step 1: When the pixel point num_a[i][j]=4 is encountered during the line-by-line scanning process, execute Step 2, otherwise execute Step 6;

[0032] Step 2: If num_b[i][j] = num_down[i][j], the two points in the previous row of this point are pseudo boundary points, so let f[i][j] = 0, f[i][j+1] = 0;

[0033] Step 3: If num_b[i][j] = num_up[i][j], the next two points in the row are pseudo boundary points, so let f[i+1][j] = 0, f[i+1][j+1] = 0;

[0034] Step 4: If num_b[i][j] = num_left[i][j], then the two points in the row to the right of the point are pseudo boundary points, let f[i+1][j] = 0, f[i+1][j+1] = 0;

[0035] Step 5: If num_b[i][j] = num_right[i][j], then the two points in the row to the left of the point are pseudo boundary points, let f[i][j] = 0, f[i][j+1] = 0;

[0036] Step 6: The processing of this point is completed, and the processing of the next point continues;

[0037] Described searching for next contour point comprises the following steps:

[0038] Step 1: Scan the image in a line-by-line scanning mode, record the first contour point found as p0, and search for the next boundary point in the 8-neighborhood of the contour point according to the set scanning direction dir; when f[i][j] = 1, record the scanning direction, set pre_dir = dir, f[i][j] = 0, and record the position of the point as p_next;

[0039] Step 2: At the point p_next, scan its 8-neighborhood in the same scanning direction dir. The weight wei is the number of 1-pixel points in the 8-neighborhood of a point. When a point satisfies f[i][j]=1 and wei[i][j]=2, then the point is the next selected point, otherwise continue; when dir=pre_dir and f[i][j]=1, then the point is the next selected point, otherwise select the first point with f[i][j]=1, record the position p_now and direction dir of the point, and set f[i][j]=0;

[0040] Step 3: If p_now = p0, the contour scan is completed, go to Step 1, continue scanning another area, and record the first boundary point found as p1;

[0041] Step 4: If p_now≠p0, set p_next=p_now, pre_dir=dir and continue to execute step 2;

[0042] Step 5: If no 1-pixel point is found in the binary image, the contour tracing ends.

[0043] Further, the step IV) of fitting the profile comprises the following steps:

[0044] Step 1: Get the point with the largest curvature on the contour curve. This point divides the curve into two segments. Both segments have corresponding point sequences. Process each segment separately.

[0045] Step 2: Remember L i For the points on the first curve, subscript i = 0, 1, 2, ..., n-1, input the coordinates of the n points on the first curve;

[0046] Step 3: Let A = L0, B = L n-1 , connect AB to generate stack S, and push point B into the stack;

[0047] Step 4: Find the point between AB that is farthest from the line connecting the two points, record it as C, and the distance from this point to the line connecting AB is d;

[0048] Step 5: Determine the relationship between d and the threshold. If it is greater than the threshold, then B = C, and add C to the feature point sequence into the stack, reconnect AB, and return to Step 4; otherwise, execute Step 6;

[0049] Step 6: Determine whether B is equal to the top element S of the stack n-1 If they are not equal, then A=C, B=S n-1 , reconnect AB and return to Step 4; otherwise, execute Step 7;

[0050] Step7: Determine whether the coordinates of B are equal to L n-1 If they are equal, then A = B and B = S n-2 , reconnect AB, S performs a stack pop operation once, and returns to Step4;

[0051] Step8: If the stack is empty, it means that the processing of the first curve is completed, and the above operations are repeated for the second curve.

[0052] Furthermore, the step V) of determining the concavity and convexity of the polygon contour vertices includes the following steps:

[0053] ① Determine the direction of the polygon:

[0054] First, sequentially denote the n vertices of the polygon contour as P1, P2,..., P n , where the subscripts 1 to n are the sequence numbers of the vertices, and the leftmost extreme point is denoted as P L , the rightmost extreme point is denoted as P R , the uppermost extreme point is denoted as P U , the lowermost extreme point is denoted as P D , if there are multiple extreme points in a certain direction, then the first point that appears in the vertex sequence is used as the extreme point in this direction, where the subscripts L, R, U, and D are the sequence numbers of the corresponding extreme points; it is defined that when walking along the vertex sequence on the polygon contour, the bounded area enclosed by the polygon contour is always on the left hand side, then the direction of the polygon is called the counterclockwise direction; it is defined that when walking along the vertex sequence on the polygon contour, the bounded area enclosed by the polygon contour is always on the right hand side, then the direction of the polygon is called the clockwise direction;

[0055] Then, judge the direction of the polygon according to the magnitude order of the sequence numbers of the four extreme points. The judgment method is as follows:

[0056] The first case, when there are no overlapping points among P L , P R , P U and P D , the judgment rule is: if L < U < R or R < D < L, then the direction of the polygon is clockwise; if L < D < R or R < U < L, then the direction of the polygon is counterclockwise;

[0057] The second case, when there are overlapping points among P L , P R , P U and P DWhen only two points overlap, the judgment rule is as follows: if D < L < R or L < R < D or L < U < R, the direction of the polygon is clockwise; if L > R > U or U > L > R or L > D > R, the direction of the polygon is also clockwise; if U < L < R or L < R < U or L < D < R, the direction of the polygon is counterclockwise; if L > R > D or D > L > R or R < U < L, the direction of the polygon is also counterclockwise;

[0058] In the third case, when P L , P R , P U and P D have only two pairs of overlapping points, one pole of the polygon is removed, and the vertex sequence numbers after that pole are decreased by 1. Then, judge the direction of the polygon according to the case where only two points in P L , P R , P U and P D overlap;

[0059] ② Define the interior angle formed by two adjacent sides intersecting at a vertex as less than 180°, then call this vertex a convex vertex; define the interior angle formed by two adjacent sides intersecting at a vertex as greater than 180°, then call this vertex a concave vertex; the interior angle mentioned refers to the angle within the bounded region enclosed by the polygon contour;

[0060] For a polygon with n vertices, the method to judge the convexity and concavity of vertex P i is as follows: First, form a triangle with vertex P i and its previous vertex P i-1 , and its next vertex P i+1 . Then, judge the direction of this triangle. The method to judge the triangle direction is as follows:

[0061] When this triangle can be regarded as a polygon with only two poles overlapping, the method to judge the triangle direction is the same as the second case of judging the polygon direction;

[0062] When this triangle can be regarded as a polygon with two pairs of overlapping poles, first connect the two pairs of overlapping poles with a straight line, and regard the midpoint of the straight line as P L or P R . Then, judge the direction of this triangle according to the second case of judging the polygon direction;

[0063] If the triangle direction is consistent with the polygon direction, then vertex P i is a convex vertex, otherwise it is a concave vertex.

[0064] Advantages of the present invention:

[0065] 1. The present invention is a method for detecting and locating grasping points based on the contour convexity. Aiming at the actual situation that the stacked medicine bags are convex polygons after image fitting, the method creatively utilizes the characteristic that the vertices at the junction of two adjacent medicine bags are concave vertices to segment the medicine bags, thereby reducing the difficulty of segmenting the stacked medicine bags, and on this basis realizes the positioning of the final grasping point of the medicine bags, solving the problem of selecting and locating the most suitable medicine bags for grasping at present in the stacked medicine bags.

[0066] 2. The present invention is based on the method for detecting and locating grab points based on the concavity of contours. The concavity of each point of the approximate polygon after fitting is judged in turn. There is a medicine bag between two adjacent concave points, and the medicine bag with the largest area is the current grab bag to be detected. Compared with the target detection algorithm of deep learning, the detection algorithm adopted by the present invention has the following main advantages while ensuring the detection accuracy: 1) There is no need to collect a large amount of data and perform annotation and training; 2) The performance requirements for hardware facilities are relatively low, which is convenient for implantation into embedded systems; 3) The processing time is relatively short and the real-time performance is good; 4) The rectangular features of the medicine bag are highly restored, which is convenient for the subsequent grab point positioning processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is the current process diagram of grabbing medicine bags for testing;

[0068] Figure 2 is the connected scene of pixels;

[0069] Figure 3 Get renderings for contour points;

[0070] Figure 4 The eight scanning directions of pixels;

[0071] Figure 5 This is the contour tracking effect diagram;

[0072] Figure 6 Extract the effect diagram for the contour;

[0073] Figure 7 Flowchart for the improved Douglas Peuker-algorithm;

[0074] Figure 8 This is the contour fitting effect diagram;

[0075] Fig. 9 This is the effect diagram of stacked medicine bags segmentation. DETAILED DESCRIPTION

[0076] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0077] The present embodiment provides a method for detecting medicine bags and locating gripping points based on contour concavity and convexity, which includes the following steps:

[0078] I) The image of the medicine bags stacked on the sorting table is collected by a camera arranged above the sorting table.

[0079] II) Image preprocessing: The images captured by the camera are subjected to grayscale conversion, median filtering, binarization, closing operation and opening operation in sequence.

[0080] Grayscale refers to the abbreviation of the color value of each pixel in a black and white image. The value range is generally 0 to 255, where 0 is black and 255 is white. In the color image captured by the camera, each pixel contains three components: red (R), green (G), and blue (B). RGB images reflect the color characteristics of the target object rather than its morphological distribution, and the amount of information that needs to be processed is three times that of grayscale images. Therefore, in order to perform efficient upper-level operations such as segmentation, detection, and recognition on the image later, it is necessary to convert the color image from a black and white image, that is, to complete the grayscale processing.

[0081] In this embodiment, the weighted average method is used to grayscale the image, and the calculation formula is as follows:

[0082] Gray(i,j)=0.229*R(i,j)+0.578*G(i,j)+0.114*B(i,j)

[0083] Where Gray(i,j) is the gray value of each pixel after processing, R(i,j), G(i,j), and B(i,j) correspond to the values ​​of the red, green, and blue channels in the original color, respectively. The coefficients of the three are determined by the human eye's perception of each type of light. This method can better preserve the morphological characteristics.

[0084] Usually, the image is mixed with noise to a certain extent due to the influence of the shooting environment (such as light, dust, etc.) or the manufacturing process of the camera itself is defective, or the digital signal of the picture is disturbed during the encoding, transmission and decoding process. Therefore, it is necessary to eliminate the noise as much as possible without destroying the original characteristics of the image, that is, to complete the filtering process.

[0085] In this embodiment, median filtering is used to eliminate noise. Median filtering is nonlinear. Its principle is to obtain the grayscale values ​​of surrounding pixels, then sort these grayscale values, and finally fill the pixel value of the center point with the sorted median value. Median filtering not only has a good noise suppression effect, but also maintains the edge information of the image well.

[0086] Binarization is to set the grayscale value of the pixel on the image to 0 or 255. After the grayscale image is binarized, the number of pixels is suddenly reduced to 2, making the processing simpler.

[0087] In order to achieve an ideal effect in subsequent upper-layer operations, morphological operations need to be performed on the image. In morphological operations, a structuring element is first set. This element has obvious processing effects on some information and not on others. Therefore, operations on specific information can be achieved, thereby obtaining an ideal output. There are four basic morphological operations: dilation, erosion, opening operation, and closing operation. In this embodiment, the closing operation is used to smooth the contour of the bag and fill the internal contour; the opening operation is used to remove the interference caused by impurities such as dust on the sorting table.

[0088] III) Perform contour tracking on the binary image processed in step II). The contour tracking includes determining the boundary points of the contour and finding the next contour point;

[0089] The determination of the boundary points of the contour includes the following steps:

[0090] a) The pixels with gray values of 0 and 1 in the binary image are respectively called 0-pixel points and 1-pixel points. The pixel point in the i-th row and j-th column is denoted as (i, j). If f[i][j] represents the gray value of this point, then there is:

[0091]

[0092] b) Scan the binary image row by row. During the scanning process, when a 1-pixel is encountered, count the number of 1-pixel points in its 8-connected direction and 4-connected direction, and denote them as num_8 and num_4 respectively. Then there is:

[0093]

[0094] c) Judge the situation of boundary points:

[0095] (1) If 1 < num_8[i][j] < 4, then this point is a boundary point, and let f[i][j] = 1;

[0096] (2) If 3 < num_8[i][j] < 8, continue to judge the value of num_4[i][j]. If num_4[i][j] ≠ 4, then this point is a boundary point, and let f[i][j] = 1;

[0097] (3) In other situations except for situation (1) and situation (2), this point is not a boundary point, and let f[i][j] = 0;

[0098] d) Remove the interference points among the image boundary points obtained in step c): First, define the following variables

[0099]

[0100]

[0101]

[0102]

[0103]

[0104]

[0105] Then follow the steps below to eliminate interference points:

[0106] Step 1: When the pixel point num_a[i][j]=4 is encountered during the line-by-line scanning process, execute Step 2, otherwise execute Step 6;

[0107] Step 2: If num_b[i][j] = num_down[i][j], the two points in the previous row of this point are pseudo boundary points, so let f[i][j] = 0, f[i][j+1] = 0;

[0108] Step 3: If num_b[i][j] = num_up[i][j], the next two points in the row are pseudo boundary points, so let f[i+1][j] = 0, f[i+1][j+1] = 0;

[0109] Step 4: If num_b[i][j] = num_left[i][j], then the two points in the row to the right of the point are pseudo boundary points, let f[i+1][j] = 0, f[i+1][j+1] = 0;

[0110] Step 5: If num_b[i][j] = num_right[i][j], then the two points in the row to the left of the point are pseudo boundary points, let f[i][j] = 0, f[i][j+1] = 0;

[0111] Step 6: The processing of this point is completed and the next point will be processed.

[0112] The points finally retained are boundary points. The processing rule of the traditional 8-connected boundary scanning method is: if the number of 1 pixel points in the 8-connected domain of a point is between 2 and 7, it is regarded as a boundary point; otherwise, it is regarded as a non-boundary point, so more boundary points are retained; and this embodiment obtains fewer contour points after eliminating interference points.

[0113] Described searching for next contour point comprises the following steps:

[0114] Step 1: Scan the image in a line-by-line scanning mode, record the first contour point found as p0, and search for the next boundary point in the 8-neighborhood of the contour point according to the set scanning direction dir. The scanning direction in this embodiment is as follows: Figure 4As shown in the figure, the next boundary point is found in its 8-neighborhood in the order of 0 to 7; when f[i][j]=1, the scanning direction is recorded, pre_dir=dir, f[i][j]=0, and the position of the point is recorded as p_next;

[0115] Step 2: At the point p_next, scan its 8-neighborhood in the same scanning direction dir. The weight wei is the number of 1-pixel points in the 8-neighborhood of a point. When a point satisfies f[i][j]=1 and wei[i][j]=2, then the point is the next selected point, otherwise continue; when dir=pre_dir and f[i][j]=1, then the point is the next selected point, otherwise select the first point with f[i][j]=1, record the position p_now and direction dir of the point, and set f[i][j]=0;

[0116] Step 3: If p_now = p0, the contour scan is completed, go to Step 1, continue scanning another area, and record the first boundary point found as p1;

[0117] Step 4: If p_now≠p0, set p_next=p_now, pre_dir=dir and continue to execute step 2;

[0118] Step 5: If no 1-pixel point is found in the binary image, the contour tracing ends.

[0119] The contour tracking rule used in this embodiment combines the scanning direction dir of the previous point and the weight wei of the next point. The rule of scanning direction dir in this embodiment is as follows: Figure 4 As shown in the figure, the weight wei is the number of 1 pixel points in the 8-neighborhood of a certain point. Figure 5 For the target boundary tracking result, the contour points start from 0 and are recorded in the order of 0, 1, ..., 27. The above algorithm can be used to obtain the contour of the medicine bag image. In the case of stacked medicine bags, in order to reduce the probability of the tip falling on the overlap between the medicine bags, the method of grabbing from the outside to the inside is adopted, so the inner contour can be discarded. The final contour extraction effect of the medicine bag image is as follows Figure 6 shown.

[0120] IV) fitting the contour to obtain a polygonal contour, comprising the following steps:

[0121] Step 1: Get the point with the largest curvature on the contour curve. This point divides the curve into two segments. Both segments have corresponding point sequences. Process each segment separately.

[0122] Step 2: Remember L i For the points on the first curve, subscript i = 0, 1, 2, ..., n-1, input the coordinates of the n points on the first curve;

[0123] Step 3: Let A = L0, B = L n-1 , connect AB to generate stack S, and push point B into the stack;

[0124] Step 4: Find the point between AB that is farthest from the line connecting the two points, record it as C, and the distance from this point to the line connecting AB is d;

[0125] Step 5: Determine the relationship between d and the threshold. If it is greater than the threshold, then B = C, and add C to the feature point sequence into the stack, reconnect AB, and return to Step 4; otherwise, execute Step 6;

[0126] Step 6: Determine whether B is equal to the top element S of the stack n-1 If they are not equal, then A=C, B=S n-1 , reconnect AB and return to Step 4; otherwise, execute Step 7;

[0127] Step 7: Determine whether the coordinates of B are consistent with L n-1 Equal, if equal, then A=B, B=S n-2 , reconnect AB, S performs a pop action, and returns to Step 4;

[0128] Step 8: If the stack is empty, it means the first segment of the curve has been processed. Repeat the above operation for the second segment.

[0129] In image processing, the Douglas-Peucker (DP) algorithm is generally used to fit contours. The algorithm is rotationally and translationally invariant. The principle is to approximate a curve with a series of points, and then remove some of the points by setting a threshold. The final result is composed of the undeleted points. However, the traditional DP algorithm takes a long time to implement recursively, and if the recursive level is too deep, it may cause a stack overflow.

[0130] In order to retain the poles, the curve is divided into two sections for processing. However, the traditional DP algorithm does not record the node with the largest distance in the middle, resulting in a deep recursive nesting level and slow program operation. In this embodiment, the segmented processing of the DP algorithm is realized with the help of a stack data structure. The algorithm takes one end of the curve as the starting point, and takes the first point and the last point as the processing range of the improved DP algorithm. Next, the relationship between the distance of the farthest point and the threshold is compared to simplify the line elements. The use of the improved DP algorithm can achieve rapid fitting of the contour and obtain the polygonal approximate contour of the target object. However, in the fitting process, if the threshold is too large, key information will be lost, resulting in inaccurate positioning of the grasping point; if the threshold is too small, it will lead to an increase in the information of subsequent processing. In this embodiment, an iterative threshold method is used, that is, an original threshold is first set, and the distance between the fitting points is calculated. If there are points with a smaller distance, the threshold is increased and the fitting continues.

[0131] V) Determine the concavity and convexity of the polygon contour vertices, including the following steps:

[0132] ① Determine the direction of the polygon:

[0133] First, sequentially denote the n vertices of the polygon contour as P1, P2,..., P n , where the subscripts 1 to n are the sequence numbers of the vertices, and the leftmost extreme point is denoted as P L , the rightmost extreme point is denoted as P R , the uppermost extreme point is denoted as P U , the lowermost extreme point is denoted as P D , if there are multiple extreme points in a certain direction, then the first point that appears in the vertex sequence is used as the extreme point in this direction, where the subscripts L, R, U, and D are the sequence numbers of the corresponding extreme points; it is defined that when walking along the vertex sequence on the polygon contour, if the bounded area enclosed by the polygon contour is always on the left hand side, then the direction of the polygon is called the counterclockwise direction; it is defined that when walking along the vertex sequence on the polygon contour, if the bounded area enclosed by the polygon contour is always on the right hand side, then the direction of the polygon is called the clockwise direction;

[0134] Then, determine the direction of the polygon based on the magnitude order of the sequence numbers of the four extreme points. The determination method is as follows:

[0135] The first case, when there are no overlapping points among P L , P R , P U and P D , the determination rule is: if L < U < R or R < D < L, then the direction of the polygon is clockwise; if L < D < R or R < U < L, then the direction of the polygon is counterclockwise;

[0136] The second case, when there are only two overlapping points among P L , P R , P U and P D , the determination rule is: if D < L < R or L < R < D or L < U < R, then the direction of the polygon is clockwise; if L > R > U or U > L > R or L > D > R, then the direction of the polygon is also clockwise; if U < L < R or L < R < U or L < D < R, then the direction of the polygon is counterclockwise, if L > R > D or D > L > R or R < U < L, then the direction of the polygon is also counterclockwise;

[0137] The third case, when there are among P L , P R , P U and P DIf only two pairs of points overlap, remove one of the extreme points of the polygon and reduce the sequence number of the vertices after the extreme point by 1, and then repeat the process according to P. L , P R , P U and P D Determine the direction of the polygon when only two points overlap;

[0138] ② If the internal angle formed by two adjacent sides intersecting a vertex is less than 180°, the vertex is called a convex vertex; if the internal angle formed by two adjacent sides intersecting a vertex is greater than 180°, the vertex is called a concave vertex; the internal angle refers to the angle within the bounded area surrounded by the polygon outline;

[0139] For a polygon with n vertices, determine the vertex P i The method to find the convexity is: first, the vertex P i With its previous vertex P i-1 , the next vertex P i+1 Form a triangle, and then determine the direction of the triangle. The method to determine the direction of the triangle is:

[0140] When the triangle can be regarded as a polygon with only two vertices overlapping, the method for determining the direction of the triangle is the same as the second case for determining the direction of the polygon;

[0141] When the triangle can be regarded as a polygon with two pairs of overlapping poles, first connect the two pairs of overlapping poles with a straight line and regard the midpoint of the straight line as P L or P R Then, the direction of the triangle is determined according to the second case of determining the direction of the polygon;

[0142] If the triangle orientation is consistent with the polygon orientation, then vertex P i is a convex vertex, and vice versa is a concave vertex;

[0143] VI) Drug bag segmentation: Starting from vertex P1, the concavity and convexity of each vertex of the polygonal contour are judged in turn, and a drug bag is segmented between two adjacent concave vertices;

[0144] VII) Locating the grasping point of stacked medicine bags: First, calculate and fit the sum of the side lengths of each medicine bag after segmentation in the polygonal contour, and regard the medicine bag with the largest sum of side lengths as the one with the smallest overlapping area with other medicine bags, that is, the medicine bag that is currently most suitable for grasping; then obtain the length and width of the medicine bag from the contour fitting image of the medicine bag, and restore the medicine bag to a rectangle; then find the midpoint of the longest side of the medicine bag in the fitting image, and then translate the midpoint into the medicine bag in a direction perpendicular to the longest side. The translation distance is two-fifths of the width of the medicine bag, and the point after translation is the final grasping point.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A method for detecting medicine bags and locating grasping points based on contour concavity and convexity, comprising the following steps: I) Collect images of medicine bags stacked on a sorting table through a camera arranged above the sorting table; II) Successively perform grayscale conversion, median filtering, binarization processing, closing operation, and opening operation on the images collected by the camera; III) Perform contour tracking on the binary image processed in step II); IV) Fit the outer contour to obtain a polygon contour; It is characterized in that: V) Judge the concavity and convexity of the vertices of the polygon contour; VI) Medicine bag segmentation: Segment out a medicine bag between two adjacent concave vertices of the polygon contour; VII) Locate the grasping points of stacked medicine bags: First calculate the sum of the side lengths of each medicine bag after segmentation in the polygon contour obtained by fitting. The medicine bag with the largest sum of side lengths is regarded as having the smallest overlapping area with other medicine bags, that is, the currently most suitable medicine bag to be grasped. Then obtain the length and width of the medicine bag from the contour fitting diagram of this medicine bag, and restore the medicine bag to a rectangle. Then find the midpoint of the longest side of the medicine bag in the fitting image, and then translate this midpoint along the direction perpendicular to the longest side into the medicine bag by a distance of two-fifths of the width of the medicine bag. The translated point is the final grasping point.

2. The method for detecting medicine bags and locating grasping points based on contour concavity and convexity according to claim 1, characterized in that: The contour tracking in step III) includes determining the boundary points of the contour and finding the next contour point; The determination of the boundary points of the contour includes the following steps: a) The pixels with grayscale values of 0 and 1 in the binary image are respectively called 0-pixel points and 1-pixel points. The pixel point at the i-th row and j-th column is denoted as (i, j), and if f[i][j] represents the grayscale value of this point, then: b) Scan the binary image row by row. When encountering a 1-pixel during the scanning process, count the number of 1-pixel points in its 8-connected direction and 4-connected direction, and denote them as num_8 and num_4 respectively. Then: c) Judge the boundary point situation: (1) If 1 < num_8[i][j] < 4, then this point is a boundary point, and let f[i][j] = 1; (2) If 3 < num_8[i][j] < 8, continue to judge the value of num_4[i][j]. If num_4[i][j] ≠ 4, then this point is a boundary point, and let f[i][j] = 1; (3) In other situations except (1) and (2), this point is not a boundary point, and let f[i][j] = 0; d) Remove the interference points among the image boundary points obtained in step c): First define the following variables Then exclude interference points according to the following steps: Step1: When encountering a pixel point with num_a[i][j] = 4 during the row-by-row scanning process, execute Step2, otherwise execute Step6; Step2: If num_b[i][j] = num_down[i][j], then the two points in the upper row of this point are pseudo-boundary points, and let f[i][j] = 0, f[i][j + 1] = 0; Step3: If num_b[i][j] = num_up[i][j], then the two points in the lower row of this point are pseudo-boundary points, and let f[i + 1][j] = 0, f[i + 1][j + 1] = 0; Step 4: If num_b[i][j] = num_left[i][j], then the two points in the row to the right of the point are pseudo boundary points, let f[i+1][j] = 0, f[i+1][j+1] = 0; Step 5: If num_b[i][j] = num_right[i][j], then the two points in the row to the left of the point are pseudo boundary points, let f[i][j] = 0, f[i][j+1] = 0; Step 6: The processing of this point is completed, and the processing of the next point continues; Described searching for next contour point comprises the following steps: Step 1: Scan the image in a line-by-line scanning mode, record the first contour point found as p0, and search for the next boundary point in the 8-neighborhood of the contour point according to the set scanning direction dir; when f[i][j] = 1, record the scanning direction, set pre_dir = dir, f[i][j] = 0, and record the position of the point as p_next; Step 2: At the point p_next, scan its 8-neighborhood in the same scanning direction dir. The weight wei is the number of 1-pixel points in the 8-neighborhood of a point. When a point satisfies f[i][j]=1 and wei[i][j]=2, then the point is the next selected point, otherwise continue; when dir=pre_dir and f[i][j]=1, then the point is the next selected point, otherwise select the first point with f[i][j]=1, record the position p_now and direction dir of the point, and set f[i][j]=0; Step 3: If p_now = p0, the contour scan is completed, go to Step 1, continue scanning another area, and record the first boundary point found as p1; Step 4: If p_now≠p0, set p_next=p_now, pre_dir=dir and continue to execute step 2; Step 5: If no 1-pixel point is found in the binary image, the contour tracing ends.

3. The method for detecting medicine bags and locating grasping points based on contour concavity and convexity according to claim 1, characterized in that: The step IV) of fitting the profile comprises the following steps: Step 1: Get the point with the largest curvature on the contour curve. This point divides the curve into two segments. Both segments have corresponding point sequences. Process each segment separately. Step 2: Remember L i For the points on the first curve, subscript i = 0, 1, 2, ..., n-1, input the coordinates of the n points on the first curve; Step 3: Let A = L0, B = L n-1 , connect AB to generate stack S, and push point B into the stack; Step 4: Find the point between AB that is farthest from the line connecting the two points, record it as C, and the distance from this point to the line connecting AB is d; Step 5: Determine the relationship between d and the threshold. If it is greater than the threshold, then B = C, and add C to the feature point sequence into the stack, reconnect AB, and return to Step 4; otherwise, execute Step 6; Step 6: Determine whether B is equal to the top element S of the stack n-1 Equal, if not equal, then A=C, B=S n-1 , reconnect AB and return to Step 4; otherwise, execute Step 7; Step 7: Determine whether the coordinates of B are consistent with L n-1 Equal, if equal, then A=B, B=S n-2 , reconnect AB, S performs a pop action, and returns to Step 4; Step 8: If the stack is empty, it means the first segment of the curve has been processed. Repeat the above operation for the second segment.

4. The method for detecting medicine bags and locating grasping points based on contour concavity and convexity according to claim 1, characterized in that: The step V) of determining the concavity and convexity of the polygonal contour vertices comprises the following steps: ① Determine the direction of the polygon: First, the n vertices of the polygon outline are recorded as P1, P2, ..., P n , subscripts 1 to n are the sequence numbers of the vertices, and the leftmost vertex is denoted by P L , the rightmost point is denoted as P R , the top point is denoted as P U , the lowest point is denoted as P D , if there are multiple poles in a certain direction, the first point that appears in the vertex sequence is taken as the pole in this direction, where the subscripts L, R, U and D are the sequence numbers of the corresponding poles respectively; when walking on the polygon outline along the vertex sequence, the bounded area enclosed by the polygon outline is always on the left hand side, then the direction of the polygon is called counterclockwise; when walking on the polygon outline along the vertex sequence, the bounded area enclosed by the polygon outline is always on the right hand side, then the direction of the polygon is called clockwise; Then the direction of the polygon is determined by the order of the four pole numbers. The determination method is as follows: In the first case, when there are no overlapping points in P L , P R , P U and P D , the judgment rule is as follows: If L < U < R or R < D < L, the direction of the polygon is clockwise; if L < D < R or R < U < L, the direction of the polygon is counterclockwise. In the second case, when P L , P R , P U and P D only two points overlap, the judgment rule is: if D < L < R or L < R < D or L < U < R, the direction of the polygon is clockwise; if L > R > U or U > L > R or L > D > R, the direction of the polygon is also clockwise; if U < L < R or L < R < U or L < D < R, the direction of the polygon is counterclockwise; if L > R > D or D > L > R or R < U < L, the direction of the polygon is also counterclockwise; In the third case, when P L , P R , P U and P D If only two pairs of points overlap, remove one of the extreme points of the polygon and reduce the sequence number of the vertices after the extreme point by 1, and then repeat the process according to P. L , P R , P U and P D Determine the direction of the polygon when only two points overlap; ② If the internal angle formed by two adjacent sides intersecting a vertex is less than 180°, the vertex is called a convex vertex; if the internal angle formed by two adjacent sides intersecting a vertex is greater than 180°, the vertex is called a concave vertex; the internal angle refers to the angle within the bounded area surrounded by the polygon outline; For a polygon with n vertices, determine the vertex P i The method to find the convexity is: first, the vertex P i With its previous vertex P i-1 , the next vertex P i+1 Form a triangle, and then determine the direction of the triangle. The method to determine the direction of the triangle is: When the triangle can be regarded as a polygon with only two vertices overlapping, the method for determining the direction of the triangle is the same as the second case for determining the direction of the polygon; When the triangle can be regarded as a polygon with two pairs of overlapping poles, first connect the two pairs of overlapping poles with a straight line and regard the midpoint of the straight line as P L or P R Then, the direction of the triangle is determined according to the second case of determining the direction of the polygon; If the triangle orientation is consistent with the polygon orientation, then vertex P i is a convex vertex, otherwise it is a concave vertex.