A target object grasping angle determination method

Abstract

The application discloses a target object grabbing angle determination method. The target object grabbing angle determination method comprises the following steps: acquiring a detection image and a template image of a target object; determining a target object image region and the center coordinates of the target object in the detection image according to the detection image and the template image; determining the center coordinates of an angle mark in the detection image according to the target object image region and the template image; and determining the current grabbing angle of the target object according to the center coordinates of the target object and the center coordinates of the angle mark. The above technical scheme can improve the efficiency and accuracy of the determined grabbing angle.

Description

Technical Field

[0001] This invention relates to the field of visual positioning technology, and in particular to a method for determining the grasping angle of a target object. Background Technology

[0002] Visual positioning technology is widely used in industrial production because it has high positioning accuracy, high speed, high adaptability, and non-contact operation, which can meet the needs of real-time detection.

[0003] Currently, the commonly used visual positioning technology in industrial production involves acquiring corresponding detection images using industrial cameras and employing appropriate recognition and positioning algorithms to determine the actual rotation angle of the target object. However, when the target object has a special shape, such as the stator core of a flat wire motor, the angle recognition features are relatively small compared to the overall area of ​​the product because the stator core is a standard circle. This makes it difficult for conventional recognition and positioning algorithms to accurately determine the rotation angle information, thus adversely affecting the subsequent visual guidance and grasping of the stator core. Summary of the Invention

[0004] This invention provides a method for determining the grasping angle of a target object, thereby solving the technical problem in the prior art that it is impossible to accurately locate a target object with a special shape.

[0005] This invention provides a method for determining the grasping angle of a target object, wherein the target object is provided with an angle mark, and the method for determining the grasping angle of the target object includes:

[0006] Acquire the detection image and template image of the target object;

[0007] Based on the detected image and the template image, determine the target image region in the detected image and the center coordinates of the target object;

[0008] Based on the target image region and the template image, determine the center coordinates of the angle marker in the detection image;

[0009] The current grasping angle of the target object is determined based on the center coordinates of the target object and the center coordinates of the angle indicator.

[0010] The technical solution of this invention acquires a detection image and a template image of the target object. Based on the detection image and the template image, it determines the target object image region and the center coordinates of the target object in the detection image. Then, based on the target object image region and the template image, it determines the center coordinates of the angle marker in the detection image. The current grasping angle of the target object is determined based on both the center coordinates of the target object and the center coordinates of the angle marker. Thus, even if the target object has a special shape (e.g., a standard circle or square), the rotation angle of the target object can be determined based on the relative positional relationship between the center coordinates of the target object and the center coordinates of the angle marker, thereby determining the current grasping angle of the target object. Simultaneously, after determining the target object image region, the center coordinates of the angle marker in the detection image are determined based on the target object image region and the template image. This ensures that the center coordinates of the angle marker are determined within the target object image region, rather than throughout the entire detection image, which improves the accuracy of the determined center coordinates of the angle marker. Therefore, when determining the current grasping angle of the target object based on the center coordinates of the target object and the center coordinates of the angle marker, the accuracy of the determined current grasping angle is improved, thereby increasing the success rate of grasping the target object.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a method for determining the target object grasping angle provided in Embodiment 1 of the present invention;

[0014] Figure 2 This is a schematic diagram of the structure of an image acquisition device provided in an embodiment of the present invention;

[0015] Figure 3 This is a flowchart of a method for determining the target object grasping angle provided in Embodiment 2 of the present invention;

[0016] Figure 4 This is a schematic diagram of a detection image and a template image provided in an embodiment of the present invention;

[0017] Figure 5This is a flowchart of a method for determining the target object grasping angle provided in Embodiment 3 of the present invention;

[0018] Figure 6 This is a flowchart of a method for determining the target object grasping angle provided in Embodiment 4 of the present invention;

[0019] Figure 7 This is a schematic diagram of the structure of a target object grasping angle determination device provided in Embodiment 5 of the present invention;

[0020] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Embodiment Six of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising," "setting," and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] Example 1

[0024] Figure 1 This is a flowchart of a method for determining the grasping angle of a target object according to Embodiment 1 of the present invention. This embodiment is applicable to determining the grasping angle of a target object with a special shape. The target object is provided with an angle mark. This method can be executed by a target object grasping angle determining device, which can be implemented in hardware and / or software. The target object grasping angle determining device can be configured in an electronic device, such as a host computer. Figure 1 As shown, the method includes:

[0025] S110. Obtain the detection image and template image of the target object.

[0026] The detection image of the target object can be acquired using an industrial camera, which can be fixed to the robot used to grasp the target object. This ensures that the camera coordinate system of the industrial camera is relatively fixed to the coordinate system of the robot used to grasp the target object, facilitating subsequent coordinate transformation. The industrial camera can include, but is not limited to, a 2D industrial camera.

[0027] In one exemplary embodiment, Figure 2 This is a schematic diagram of the structure of an image acquisition device provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the image acquisition device can include an industrial camera and a light source. The light source is located on the side of the industrial camera closer to the work platform. The light source provides light to the target object on the work platform, so that the light reflected by the target object can be captured by the industrial camera, thereby obtaining a detection image in the industrial camera. Taking the stator core of a flat wire motor as an example, the upper and lower surfaces of the stator core are both standard annular shapes. During the loading and unloading of the stator core, it is necessary to determine the gripping angle of the stator core for subsequent assembly. At this time, an industrial camera can be used to capture the image of the upper surface of the stator core placed on the work platform as the detection image. The industrial camera can be fixed to the end of the robot used to grip the stator core. When using the industrial camera to acquire the image of the upper surface of the stator core, the parameters of the industrial camera and the light source need to be adjusted, such as the light intensity of the light source, the aperture size of the industrial camera, and the exposure time, to ensure that the image of the upper surface of the stator core is neither too dark nor too exposed. Simultaneously, when using an industrial camera to acquire images of the upper surface of the stator core, it is also necessary to adjust the height and focal length of the industrial camera to ensure that the image size, position, and clarity of the upper surface of the stator core are reasonable. For example, due to the large size of the stator core, a surface light source can be selected to ensure uniform illumination. The light-emitting surface of this surface light source covers the upper surface of the stator core, and the distance between the industrial camera and the upper surface of the stator core can be controlled at around 700mm to meet high imaging quality requirements. In addition, when using an industrial camera to acquire images of the upper surface of the stator core, the horizontal alignment of the industrial camera and the stator core must also be considered. This can be achieved by adjusting the horizontal position of the industrial camera to ensure that the image of the stator core is centered on the imaging plane of the industrial camera, facilitating subsequent image processing.

[0028] In an optional embodiment, the image of the target object acquired by the industrial camera can be an initial detection image. The initial detection image can also be preprocessed to obtain a detection image. The preprocessing process may include, for example, determining a region of interest of a standard size that includes the complete target object image based on the size of the target object image in the initial detection image acquired by the industrial camera, using the standard-sized region of interest as the detection image, and masking other areas outside the standard-sized region of interest to eliminate noise effects and improve the speed of image acquisition, image transmission and image processing.

[0029] It needs to be explained that, Figure 2 The method of acquiring the detection image is illustrated by taking the stator core of a flat-wire motor as an example only. However, in this embodiment of the invention, the target object can be any object with a special shape or a non-special shape, and this embodiment does not impose specific limitations on it. Furthermore, the method of acquiring the detection image of the target object can also be other, and this invention does not impose specific limitations on it either. For ease of description, unless otherwise specified, the following illustrative description uses the stator core of a flat-wire motor as an example.

[0030] Similarly, template images can also be acquired using industrial cameras. When acquiring template images, the parameters of the industrial camera and light source can be the same as those when acquiring detection images, so that the image ratio of the template image is consistent with that of the detection image, which facilitates subsequent image matching.

[0031] S120. Based on the detection image and the template image, determine the target image region and the center coordinates of the target object in the detection image.

[0032] In this embodiment, the template image contains a target object template. The target object template and the target object image in the detection image have the same size, orientation, and image elements. Using the region and center coordinates of the target object template as a reference, the target object image region to which the target object belongs, and the center coordinates of the target object within that region, can be located in the detection image. In an optional embodiment, template matching can be used to determine the target object image region and the center coordinates of the target object in the detection image. For example, a feature identifier is selected on the target object template, and this feature identifier is matched with the detection image using a sliding window. The corresponding target object similarity is calculated to determine the target object image region and the center coordinates of the target object in the detection image based on the target object similarity. It is understood that the feature identifier selected on the target object template can be a location point, location region, or vector, etc., and can be set according to actual needs. This embodiment of the invention does not impose specific limitations on this.

[0033] S130. Determine the center coordinates of the angle marker in the detection image based on the target image region and the template image.

[0034] The target object is also provided with an angle mark. The angle mark can be set on the target object. When the detection image and template image of the target object are collected, the angle mark on the target object can be collected at the same time. For example, when the target object is the stator core of a flat wire motor, the angle mark can be a notch, protrusion or groove set on the edge of the upper surface of the stator core, and the shape of the angle mark can be at least one of the circle, rectangle, etc., and the number of angle marks can be one or more. The embodiments of the present invention do not limit this in the most specific way.

[0035] Specifically, when acquiring images of the upper surface of the stator core, the image of the angle marker can be acquired simultaneously, and this angle marker image will be located within the image area of ​​the upper surface of the stator core. Therefore, the target template in the template image also has an angle marker template with the same size, orientation, and image elements as the angle marker image in the detection image. Using the area where this angle marker template is located and its center coordinates as a reference, the area to which the angle marker belongs is searched within the target image area. After finding the area to which the angle marker belongs, the center coordinates of the angle marker can be determined within that area. In this way, when determining the center coordinates of the angle marker, it is not necessary to search based on the entire detection image, but rather to search within the determined target image area, which can narrow the search area and improve the efficiency of determining the center coordinates of the angle marker. At the same time, by narrowing the search area for the center coordinates of the angle marker, the error of the determined center coordinates of the angle marker can be reduced, thus improving the accuracy of the determined center coordinates of the angle marker.

[0036] In an optional embodiment, template matching can be used to determine the center coordinates of the angle markers in the target image region. For example, a corresponding feature marker is selected on the angle marker template, and a sliding window matching is performed between the feature marker and the target image region. The similarity of the corresponding angle markers is then calculated to determine the center coordinates of the angle markers in the target image region based on the similarity. It is understood that the feature marker selected on the angle marker template can also be a location point, location region, or vector, etc., and can be set according to actual needs. This embodiment of the invention does not impose specific limitations on this.

[0037] S140. Determine the current grasping angle of the target object based on the center coordinates of the target object and the center coordinates of the angle indicator.

[0038] Specifically, after determining the center coordinates of the target object and the center coordinates of the angle marker, the orientation of the center coordinates of the angle marker relative to the center coordinates of the target object can be determined. By comparing this orientation with the orientation of the center coordinates of the angle marker relative to the center coordinates of the target object when the grasping angle is 0, the current rotation angle of the target object can be determined. This rotation angle is the grasping angle when the target object is grasped.

[0039] In an optional embodiment, after determining the center coordinates of the target object and the center coordinates of the angle marker, a coordinate transformation is required. The center coordinates of the target object and the center coordinates of the angle marker are transformed from the image coordinate system to the coordinate system of the industrial camera, and then from the coordinate system of the industrial camera to the coordinate system of the robot used to grasp the target object. This ensures that the center coordinates of the target object and the center coordinates of the angle marker after the coordinate transformation are the actual center coordinates of the target object and the angle marker in the robot coordinate system. This allows the robot to determine the actual rotation angle of the target object based on the center coordinates of the target object and the center coordinates of the angle marker after the coordinate transformation, enabling the robot to accurately grasp the target object based on the actual rotation angle and perform subsequent assembly and other processes.

[0040] The transformation relationships from the image coordinate system to the industrial camera's coordinate system, and from the industrial camera's coordinate system to the robot's coordinate system used for grasping the target object, can be pre-stored in corresponding memory. This allows the transformation relationships to be directly retrieved from memory during coordinate transformation. After the coordinate transformation is completed, the transformed center coordinates of the target object and the center coordinates of the angle marker can be sent to the robot via a corresponding communication protocol, which may include, but is not limited to, TCP / IP.

[0041] In an optional embodiment, the transformation relationship between the image coordinates and the coordinate system of the industrial camera, as well as the transformation relationship between the coordinate system of the industrial camera and the coordinate system of the robot, can be determined using a conventional nine-point calibration algorithm. Provided that coordinate transformation can be achieved, this embodiment of the invention does not impose specific limitations on this.

[0042] This embodiment acquires a detection image of the target object and a template image. Based on these images, it determines the target object image region and the center coordinates of the target object within the detection image. Then, based on the target object image region and the template image, it determines the center coordinates of an angle marker in the detection image. Using both the target object's center coordinates and the angle marker's center coordinates, it determines the target object's current grasping angle. This allows it to determine the target object's rotation angle based on the relative positional relationship between the target object's center coordinates and the angle marker's center coordinates, even if the target object has a unique shape (e.g., a standard circle or square). Furthermore, by determining the target object image region and then using the template image, it determines the center coordinates of the angle marker within the target object image region, rather than across the entire detection image. This improves the accuracy of the determined angle marker's center coordinates, thus increasing the success rate of target object grasping.

[0043] Example 2

[0044] Figure 3 This is a flowchart of a method for determining the target object grasping angle according to Embodiment 2 of the present invention. Based on the above embodiments, this embodiment describes the specific method for determining the target object image region and the center coordinates of the target object in the detection image, specifically including: "Determining a target reference vector in the template image from the center point of the target object template to the edge point of the target object template; performing sliding window matching between the target reference vector and the detection image, and calculating the target object similarity at each sliding window; determining the target object image region and the center coordinates of the target object in the detection image based on the similarity of each target object." Figure 3 As shown, the method includes:

[0045] S210. Obtain the detection image and template image of the target object.

[0046] S220. Determine the target reference vector in the template image, which points from the center point of the target template to the edge point of the target template.

[0047] Since the orientation and size of the target template in the template image are known, the coordinates of the center point and edge points of the target template can be determined. At this point, the line segment pointing from the center point to the edge point of the target template is the target reference vector. The target reference vector The starting coordinates are the coordinates of the center point of the target template, and this reference vector... The termination coordinates are the coordinates of the edge points of the target object template.

[0048] In an optional embodiment, since the target template in the template image has multiple edge points, multiple edge points can be selected on the edges of the target template, and multiple target reference vectors pointing from the center point of the target template to each edge point of the target template can be determined respectively.

[0049] For example, Figure 4 This is a schematic diagram of a detection image and a template image provided in an embodiment of the present invention, such as... Figure 4 As shown, the number of target reference vectors determined in the template image (a) is M, and these M target reference vectors are respectively Wherein, the target reference vector The target reference vector is a vector originating from the center point O1 of the target template and ending at the edge point A1 at the edge of the target template. Let O1 be the center point O1 of the target template and A2 be the edge point A2 at the edge of the target template, ..., the target reference vector. Let A1 be a vector originating from the center point O1 of the target template and ending at the edge point AM on the edge of the target template. Edge points A1, A2, ..., AM can be evenly distributed along the edge of the target template. When a template coordinate system is established in the template image (a) with the center point of the target template as the origin, each target reference vector can be represented by the coordinates of its corresponding edge point. For example, if the coordinates of edge point A1 are (Xa1, Ya1), A2 are (Xa2, Ya2), ..., AM are (Xam, Yam), then the target reference vector... They are (Xa1, Ya1), (Xa2, Ya2), ..., (Xam, Yam).

[0050] S230. Perform sliding window matching between the target reference vector and the detection image, and calculate the similarity of the target object at each sliding window.

[0051] Specifically, since the size of the detection image is larger than the size of the target image in the detection image, in order to determine the position of the target image in the detection image, it is necessary to compare the determined target reference vector with the vectors in each region of the detection image one by one. During the comparison process, it is necessary to calculate the similarity between the target reference vector and the vectors in each region of the detection image. This similarity can be used as the target similarity.

[0052] In an optional embodiment, the specific implementation of sliding window matching between the target reference vector and the detection image, and the calculation of the target object similarity at each sliding window, may include: determining the center coordinates of the image region corresponding to the first set sliding window on the detection image; determining the edge vector of the target object located within the image region based on the center coordinates of the image region and the target reference vector; calculating the target object similarity of the image region based on the target object edge vector and the target reference vector; adjusting the image region corresponding to the first set sliding window on the detection image by a first preset step size, and returning to execute each step from determining the center point coordinates of the region within the image region corresponding to the first set sliding window on the detection image to calculating the target object similarity of the image region based on the target object edge vector and the target reference vector, until the traversal of each image region of the detection image is completed.

[0053] The size of the first sliding window can be determined based on the dimensions of the target object. Figure 4 Taking the image shown as an example, when the outer contour of the target object template on the template image (a) is circular and the radius of the target object template is R, the area of ​​the first set sliding window can be set to 4(R+K1). 2 K1 is a parameter greater than 0. The value of this parameter can be set according to actual needs. Provided that the image area corresponding to the first set sliding window on the detection image is larger than the target object image and the size difference between the two is within a preset range, the embodiment of the present invention does not limit the value of K1. In an exemplary embodiment, K1 can be equal to 10.

[0054] Furthermore, the size of the first preset sliding window can also reflect the number of comparisons when traversing the detection image. For example, when the area of ​​the first preset sliding window is 4(R+10). 2 And when the size of the detection image is L*L, the number of comparisons n = L when traversing the detection image. 2 -4RL-40L+80R+4R 2 +400.

[0055] Understandably, since images are composed of pixels, and the size of a pixel is related to the image resolution, when the image resolution is fixed, the number of pixels per unit area is a fixed value, meaning the size of each pixel is a fixed value. Furthermore, once the image's coordinate system is determined, the coordinates of each pixel can be determined one-to-one. Therefore, the coordinates of the edge and center points of an object in the image represent the coordinates of the pixels representing the edge and center points of that object, respectively. In this case, the unit of measurement for the object's size in the image can be pixels. That is, when the radius of the object template is R, we can know that the radius of the object template is R pixels, thus enabling the accurate determination of the coordinates of the object's edge and center points in the image.

[0056] Specifically, with Figure 4 Taking the image shown as an example, after determining the target reference vector pointing from the center point of the target template to the edge point of the target template in the template image (a), the direction and magnitude of the target reference vector can be obtained. When establishing a template coordinate system with the starting point O2 at the upper left corner of the detection image (b) as the origin of the image coordinate system and the center point O1 of the target template in the template image (a) as the origin, if the center coordinates of the target in the detection image (b) are (X0, Y0), then the target reference vector... When the template coordinate system is (Xa, Ya), the target edge vector in image (b) is detected. It should be (Xa+X0, Ya+Y0). At this point, starting from the origin O2 of the image coordinate system of the detection image (b), the detection image is traversed, so that the initial position of the first set sliding window is image region P1 in the detection image (b). When the coordinates (X1, Y1) of the center point of this image region P1 are used as the center coordinates of the target object, based on the target reference vector... In the template coordinate system (Xa, Ya), the edge vector of the target object in the image region P1 can be obtained. Given (X1+Xa, Y1+Xa), we can calculate the similarity between the image region P1 and the target object template. After determining the similarity between image region P1 and the target object template, the first preset sliding window can be moved by a first preset step size, so that the position of the first preset sliding window becomes image region P2 in the detection image (b). When the coordinates (X2, Y2) of the center point of image region P2 are used as the center coordinates of the target object, the edge vector of the target object in image region P2 can be obtained. Given (X2+Xa, Y2+Xa), we can calculate the similarity between the image region P2 and the target object template. This process continues until the first set sliding window position is the last image region Pn in the detection image, completing the traversal of the detection image (b). The coordinates (Xn, Yn) of the center point of the image region Pn are used as the center coordinates of the target object to determine the edge vector of the target object in the image region Pn. Given (Xn+Xa, Yn+Xa), the similarity between the image region Pn and the target object template is...

[0057] In an optional embodiment, the number of target reference vectors determined in the template image is M, and each image region corresponding to the first set sliding window includes M target object edge vectors that correspond one-to-one with the M target reference vectors; M is a positive integer greater than or equal to 2; at this time, the implementation of calculating the target object similarity of each image region may include: calculating the target vector similarity between the target reference vector and the target object edge vector corresponding to the target reference vector; and taking the average of the M target vector similarities as the target object similarity of the image region.

[0058] Specifically, when the template image includes M target reference vectors At that time, M target edge vectors can be determined corresponding to each image region of the detected image. At this point, we can first calculate the similarity between the edge vectors of each target object and the reference vector of each target object in the same image region, that is... Then, the average of the target vector similarities within the same image region is calculated to determine the target object similarity within that image region.

[0059] For example, such as Figure 4 As shown, when the edge vectors of the M target objects in image region P1 are respectively At that time, the similarity of the target object in image region P1 When the edge vectors of the M objects in image region P2 are respectively At that time, the similarity of the target objects in image region P2 …; when the edge vectors of the M objects in the image region Pn are respectively At that time, the similarity of the target objects in image region Pn In this way, the similarity of the target objects in each image region is determined based on the average of the similarity of the target vectors of multiple target object edge vectors, thereby improving the accuracy of the similarity of the target objects in each image region.

[0060] S240. Based on the similarity of each target object, determine the target object image region and the center coordinates of the target object in the detected image.

[0061] Specifically, since the object similarity can represent the degree of similarity between the object image and the object template in the image region of the detection image, and generally the higher the object similarity, the higher the degree of similarity between the object image and the object template in the image region of the detection image, the image region corresponding to the highest value among the object similarities can be determined as the object image region in the detection image, and the coordinates of the center point of the object image region are the center coordinates of the object.

[0062] In an optional embodiment, the specific implementation of determining the target image region and the center coordinates of the target in the detection image based on the similarity of each target may include: determining the target similarity with the largest similarity among the target similarities as the first similarity; determining whether the first similarity is greater than a first preset similarity; if so, determining the image region corresponding to the first similarity as the target image region; and determining the center coordinates of the target image region as the center coordinates of the target.

[0063] The first preset similarity can be the similarity between the target object image in the detection image and the target object template in the template image when the matching degree is a preset matching degree. The preset matching degree can be determined according to the actual required matching accuracy. Correspondingly, the first preset similarity can be set according to the size of the target object template or the experience of the designer. This embodiment of the invention does not make specific limitations on this.

[0064] Specifically, when determining the maximum similarity among the determined similarities of each target object as the first similarity, it is necessary to judge whether the first similarity is greater than the first preset similarity. When the first similarity is greater than the first preset similarity, it can be determined that the matching degree between the target object image in the image region corresponding to the first similarity and the target object template meets the requirements. At this time, the image region corresponding to the first similarity can be directly determined as the target object image region, and the coordinates of the center point of the target object image region can be determined as the center coordinates of the target object. Conversely, when the first similarity is less than or equal to the first preset similarity, it can be determined that the matching degree between the target object image in the image region corresponding to the first similarity and the target object template does not meet the requirements. At this time, the detection image can be traversed again by adjusting the size of the first set sliding window, etc., until the first similarity is greater than the first preset similarity.

[0065] S250. Determine the center coordinates of the angle marker in the detection image based on the target image region and the template image.

[0066] S260. Determine the current grasping angle of the target object based on the center coordinates of the target object and the center coordinates of the angle indicator.

[0067] This embodiment determines the target reference vector in the template image, performs sliding window matching between the target reference vector and the detection image, and calculates the corresponding target object similarity. Based on the target object similarity, it can determine the matching between the image region in the detection image and the target object template in the template image. Thus, based on the target object similarity, it can accurately determine the target object image region to which the target object image in the detection image belongs, as well as the center coordinates of the target object.

[0068] Example 3

[0069] Figure 5 This is a flowchart of a method for determining the target object grasping angle according to Embodiment 3 of the present invention. Based on the above embodiments, this embodiment specifically describes the method for determining the center coordinates of the angle marker, specifically including: "determining a marker reference vector in the template image from the center point of the angle marker template to the edge point of the angle marker template; performing sliding window matching between the marker reference vector and the target object image area, and calculating the angle marker similarity at each sliding window; determining the center coordinates of the angle marker in the target object image area based on the similarity of each angle marker." Figure 5 As shown, the method includes:

[0070] S310. Obtain the detection image and template image of the target object.

[0071] S320. Based on the detected image and the template image, determine the target image region and the center coordinates of the target object in the detected image.

[0072] S330. Determine the identifier reference vector in the template image, which points from the center point of the angle identifier template to the edge point of the angle identifier template.

[0073] Since the orientation and size of the target template in the template image are known, the coordinates of the center point and the edge points of the angle marker template in the template image can be determined. At this point, the line segment pointing from the center point to the edge point of the angle marker template is the marker reference vector. This identifier reference vector The starting coordinates are the coordinates of the center point of the angle identifier template, and this reference vector... The termination coordinates are the coordinates of the edge points of the angle identifier template.

[0074] In an optional embodiment, since the angle marker template in the template image has multiple edge points, multiple edge points can be selected on the edges of the angle marker template, and multiple marker reference vectors pointing from the center point of the angle marker template to each edge point of the angle marker template can be determined respectively.

[0075] For example, such as Figure 4 As shown, the number of identifier reference vectors determined in the template image (a) is N, and these N identifier reference vectors are respectively Among them, the identification reference vector The reference vector is a vector that originates at the center point O3 of the angle identifier template and ends at the edge point C1 at the edge of the angle identifier template. Let O3 be the center point O3 of the angle identifier template and C2 be the edge point C2 at the edge of the angle identifier template, ..., the identifier reference vector. Let the vector originate from the center point O3 of the angle marker template and terminate at the edge point CN at the edge of the angle marker template. Edge points C1, C2, ..., CN can be evenly distributed along the edge of the angle marker template. When a template coordinate system is established in the template image (a) with the center point of the angle marker template as the origin, each marker reference vector can be represented by the coordinates of its corresponding edge point. For example, when the coordinates of edge point C1 are (Xc1, Yc1), C2 are (Xc2, Yc2), ..., CN are (Xcn, Ycn), the marker reference vector... They are (Xc1, Yc1), (Xc2, Yc2), ..., (Xcn, Ycn).

[0076] S340. Perform sliding window matching between the identifier reference vector and the target object image region, and calculate the similarity of the angular identifier at each sliding window.

[0077] Specifically, since the target image region is larger than the size of the angle marker image in the target image region, in order to determine the position of the angle marker image in the target image region, it is necessary to compare the determined marker reference vector with the vectors in each sub-region of the target image region one by one. During the comparison process, it is necessary to calculate the similarity between the marker reference vector and the vectors in each sub-region of the target image region. This similarity can be used as the angle marker similarity.

[0078] In an optional embodiment, the identification reference vector is matched with the target image region using a sliding window, and the angular identification similarity at each sliding window is calculated. This includes: determining the center coordinates of the sub-region corresponding to the second preset sliding window in the target image region; determining the angular identification edge vector within the sub-region based on the sub-region center coordinates and the identification reference vector; calculating the angular identification similarity of the sub-region based on the angular identification edge vector and the identification reference vector; adjusting the sub-region corresponding to the second preset sliding window in the target image region by a second preset step size, and returning to execute each step from determining the center coordinates of the sub-region corresponding to the second preset sliding window in the target image region to calculating the angular identification similarity of the sub-region based on the angular identification edge vector and the identification reference vector, until the traversal of each sub-region of the target image region is completed.

[0079] The size of the second sliding window can be determined based on the dimensions of the angle marking. Figure 4 Taking the image shown as an example, when the outer contour of the angle marker template on the template image (a) is a semicircle and the radius of the angle marker template is r, the area of ​​the second setting sliding window can be set to 4(r+K2). 2K2 is a parameter greater than 0. The value of this parameter can be set according to actual needs. Provided that the sub-region corresponding to the second sliding window on the target object image area is larger than the angle marker image and the size difference between them is within a preset range, the embodiment of the present invention does not limit the value of K2. In an exemplary embodiment, K2 can be equal to 10.

[0080] Understandably, the size of the second set sliding window can reflect the number of comparisons when traversing the target image region. In an optional embodiment, the area of ​​the second set sliding window is 4(r+10). 2 The area of ​​the target object image region is 4(R+10). 2 When traversing the target image region, the number of comparisons m = 4R 2 +4r 2 -8Rr. In this way, compared to the situation where the entire detection image needs to be compared when determining the center coordinates of the angle marker, the number of comparisons can be reduced, thereby improving the efficiency of determining the center coordinates of the angle marker.

[0081] Specifically, with Figure 4 Taking the image shown as an example, after determining the identifier reference vector pointing from the center point of the angle identifier template to the edge point of the angle identifier template in the template image (a), the direction and magnitude of the identifier reference vector can be obtained. When establishing a template coordinate system with the starting point O4 at the upper left corner of the target object image region (c) as the origin of the image coordinate system and the center point O3 of the angle identifier template in the template image (a) as the origin, if the center coordinates of the target object in the target object image region (c) are (x0, y0), then the identifier reference vector... When the coordinates are (xc, yc) in the template coordinate system, the angular marker edge vector in the target object image region (c) is... It should be (xc+x0, yc+y0). At this point, starting from the origin O4 of the image coordinate system of the target image region (c), the detection image is traversed, so that the initial position of the second set sliding window is sub-region Q1 in the target image region (c). When the coordinates (x1, y1) of the center point of this sub-region Q1 are used as the center coordinates of the angle marker, based on the marker reference vector... In the template coordinate system, (xc, yc), the angular marker edge vector in the sub-region Q1 can be obtained. Given (xc+x1, yc+y1), we can then calculate the similarity between the sub-region Q1 and the angle label of the angle label template. After determining the similarity between sub-region Q1 and the angle marker template, the second preset sliding window can be moved by a second preset step size, so that the position of the second preset sliding window becomes sub-region Q2 in the target image region (c). When the coordinates (x2, y2) of the center point of sub-region Q2 are used as the center coordinates of the angle marker, the edge vector of the angle marker in sub-region Q2 can be obtained. Given (xc+x2, yc+y2), we can then calculate the similarity between the sub-region Q2 and the angle label of the angle label template. This process continues until the second set sliding window position is the last sub-region Qm in the target image region (c). At this point, the traversal of the target image region (c) is complete. The coordinates (xm, ym) of the center point of this sub-region Qm are used as the center coordinates of the angle marker, and the edge vector of the angle marker in this sub-region Qm is determined. Given (xc+xm, yc+ym), the similarity between the sub-region Qm and the angle label of the angle label template is...

[0082] In an optional embodiment, the number of identifier reference vectors determined in the template image is N, and each sub-region corresponding to the second set sliding window includes N angular identifier edge vectors that correspond one-to-one with the N identifier reference vectors; N is a positive integer greater than or equal to 2; at this time, based on the angular identifier edge vectors and the identifier reference vectors, the angular identifier similarity of the sub-region is calculated, including: calculating the identifier vector similarity between the identifier reference vector and the angular identifier edge vector corresponding to the identifier reference vector; and taking the average of the N identifier vector similarities as the angular identifier similarity of the sub-region.

[0083] Specifically, when the template image includes N identifier reference vectors At that time, N angular marker edge vectors can be determined for each image region of the detected image. At this point, we can first calculate the similarity between the edge vectors of each angular marker and the reference vectors of each marker within the same sub-region, i.e. Then, the average of the target vector similarities within the same image region is calculated to determine the target object similarity within that image region.

[0084] For example, such as Figure 4 As shown, when the N angular marker edge vectors in sub-region Q1 are respectively At that time, the similarity of the angle labels of sub-region Q1 When the N angular marker edge vectors in sub-region Q2 are respectively At that time, the similarity of the target objects in image region P2 …;When the N target vectors in subregion Qm are respectively At that time, the similarity of the target objects in image region Pn In this way, the similarity of the angular labels of each sub-region is determined based on the average of the similarity of the label vectors of multiple angular label edge vectors, thereby improving the accuracy of the angular label similarity of each sub-region.

[0085] S350. Based on the similarity of each angle marker, determine the center coordinates of the angle markers in the target object image area.

[0086] Specifically, since angle marker similarity can represent the degree of similarity between the angle marker image and the angle marker template within a sub-region of the target image region, and generally, the higher the angle marker similarity, the higher the similarity between the angle marker image and the angle marker template within the sub-region of the target image region, the sub-region corresponding to the highest value among all angle marker similarities can be determined as the angle marker region in the target image region, and the coordinates of the center point of this angle marker region are the center coordinates of the angle marker.

[0087] In an optional embodiment, determining the center coordinates of the angle markers in the target image region based on the similarity of each angle marker includes: determining the angle marker similarity with the largest similarity among the angle markers as the second similarity; determining whether the second similarity is greater than a second preset similarity; if so, determining the sub-region corresponding to the second similarity as the angle marker region; and determining the center coordinates of the angle marker region as the center coordinates of the angle marker.

[0088] The second preset similarity can be the angle identifier similarity when the matching degree between the angle identifier image in the target object image region and the angle identifier template in the template image is a preset matching degree. The preset matching degree can be determined according to the actual required matching accuracy. Correspondingly, the second preset similarity can be set according to the size of the angle identifier template or the designer's experience. This embodiment of the invention does not specifically limit this.

[0089] Specifically, when determining the maximum similarity among the determined similarities of each angle marker as the second similarity, it is necessary to judge whether the second similarity is greater than the second preset similarity. When the second similarity is greater than the second preset similarity, it can be determined that the matching degree between the angle marker image and the angle marker template in the sub-region corresponding to the second similarity meets the requirements. At this time, the sub-region corresponding to the second similarity can be directly determined as the angle marker image region, and the coordinates of the center point of the angle marker image region can be determined as the center coordinates of the angle marker. Conversely, when the second similarity is less than or equal to the second preset similarity, it can be determined that the matching degree between the angle marker image and the angle marker template in the sub-region corresponding to the second similarity does not meet the requirements. At this time, the target image region can be traversed again by adjusting the size of the second set sliding window, etc., until the second similarity is greater than the second preset similarity.

[0090] S360. Determine the current grasping angle of the target object based on the center coordinates of the target object and the center coordinates of the angle indicator.

[0091] This embodiment determines the identifier reference vector in the template image, performs sliding window matching based on the identifier reference vector and the target object image region, and calculates the corresponding angle identifier similarity. Based on the similarity of the angle identifiers, it can determine the matching between the sub-region in the target object image region and the angle identifier template in the template image. Thus, based on the similarity of the angle identifiers, it can accurately and quickly determine the angle identifier image region to which the angle identifier image in the target object image region belongs, and then quickly and accurately determine the center coordinates of the angle identifier.

[0092] Example 4

[0093] Figure 6 This is a flowchart of a method for determining the target object's grasping angle according to Embodiment 4 of the present invention. Based on the above embodiments, this embodiment specifically describes the method for determining the current grasping angle of the target object, specifically including: "obtaining an initial positioning vector; determining a detection positioning vector with the center coordinates of the target object in the detection image as the starting point and the center coordinates of the angle marker in the detection image as the ending point; determining the current grasping angle of the target object based on the angle between the detection positioning vector and the initial positioning vector," as shown below. Figure 6 As shown, the method includes:

[0094] S410. Acquire the detection image and template image of the target object.

[0095] S420. Based on the detection image and the template image, determine the target image region and the center coordinates of the target object in the detection image.

[0096] S430. Determine the center coordinates of the angle marker in the detection image based on the target image region and the template image.

[0097] S440, Obtain the initial positioning vector.

[0098] Among them, such as Figure 4 As shown, the initial positioning vector This refers to the vector in the template image obtained when the capture angle is 0 degrees, which points from the center coordinates of the target object to the center coordinates of the angle marker.

[0099] S450. Determine the detection positioning vector with the center coordinates of the target object in the detection image as the starting point and the center coordinates of the angle marker in the detection image as the ending point.

[0100] S460. Determine the current grasping angle of the target object based on the angle between the detected positioning vector and the initial positioning vector.

[0101] Specifically, compared to the template image obtained when the capture angle is 0 degrees, the target object image in the detection image may have a rotation angle greater than 0 degrees. In this case, the detection localization vector in the detection image can be determined first, pointing from the center coordinates of the target object to the center coordinates of the angle marker. At this point, if the target image in the detected image and the target object on the worktable have a 1:1 ratio, then the detection localization vector... With the initial positioning vector The included angle is the current grasping angle. Alternatively, when the scale between the target image in the detection image and the target on the worktable is not 1:1, the center coordinates of the target and the center coordinates of the angle marker in the template image with a grasping angle of 0 degrees can be transformed to the coordinates of the robot used to grasp the target on the worktable. The initial positioning vector is then determined based on the transformed center coordinates of the target and the center coordinates of the angle marker. Similarly, based on coordinate transformation, the center coordinates of the target object and the center coordinates of the angle markers in the detection image are transformed to the coordinates of the robot used to grasp the target object on the worktable, and the detection positioning vector is determined based on the transformed center coordinates of the target object and the center coordinates of the angle markers. At this point, the initial positioning vector With detection localization vector The angle between them is the current grasping angle, which enables the robot to accurately grasp the target object based on the current grasping angle, facilitating subsequent assembly, etc.

[0102] In this embodiment, after obtaining the initial positioning coordinates, the vector pointing from the center coordinates of the target object to the center coordinates of the angle marker in the detection image is used as the detection positioning vector. Based on the angle between the initial positioning coordinates and the detection positioning vector, the current grasping angle for grasping the target object is determined, so that the robot can accurately grasp the target object based on the current grasping angle, which facilitates subsequent assembly, etc.

[0103] Example 5

[0104] Figure 7 This is a schematic diagram of a target object grasping angle determination device provided in Embodiment 5 of the present invention. Figure 7 As shown, the device includes: an image acquisition module 510, a target object determination module 520, an angle identification determination module 530, and a grasping angle determination module 540, wherein:

[0105] Image acquisition module 510 is used to acquire the detection image and template image of the target object;

[0106] The target object determination module 520 is used to determine the target object image region and the center coordinates of the target object in the detection image based on the detection image and the template image;

[0107] Angle identifier determination module 530 is used to determine the center coordinates of the angle identifier in the detection image based on the target image region and the template image;

[0108] The grasping angle determination module 540 is used to determine the current grasping angle of the target object based on the center coordinates of the target object and the center coordinates of the angle indicator.

[0109] Optionally, the target object determination module 520 includes a target reference vector determination unit, a target object similarity calculation unit, and a target object determination unit, wherein:

[0110] The target reference vector determination unit is used to determine the target reference vector in the template image, which points from the center point of the target template to the edge point of the target template.

[0111] The target object similarity calculation unit is used to perform sliding window matching between the target reference vector and the detection image, and to calculate the target object similarity at each sliding window.

[0112] The target object determination unit is used to determine the target object image region and the center coordinates of the target object in the detected image based on the similarity of each target object.

[0113] Optionally, the target object similarity calculation unit is specifically used for: determining the center coordinates of the image region corresponding to the first preset sliding window on the detection image; determining the edge vector of the target object located within the image region based on the center coordinates of the image region and the target reference vector; calculating the target object similarity of the image region based on the target object edge vector and the target reference vector; adjusting the image region corresponding to the first preset sliding window on the detection image with a first preset step size, and returning to execute each step from determining the center point coordinates of the region within the image region corresponding to the first preset sliding window on the detection image to calculating the target object similarity of the image region based on the target object edge vector and the target reference vector, until the traversal of each image region of the detection image is completed.

[0114] Optionally, the number of target reference vectors determined in the template image is M, and each first set sliding window corresponds to an image region containing M target object edge vectors that correspond one-to-one with the M target reference vectors; M is a positive integer greater than or equal to 2; in this case, based on the target object edge vectors and target reference vectors, the target object similarity of the image region is calculated, including: calculating the target vector similarity between the target reference vector and the target object edge vector corresponding to the target reference vector; and taking the average of the M target vector similarities as the target object similarity of the image region.

[0115] Optionally, the target object determination unit is specifically used to: determine the target object similarity with the largest similarity among all target object similarities as the first similarity; determine whether the first similarity is greater than the first preset similarity; if so, determine the image region corresponding to the first similarity as the target object image region; and determine the center coordinates of the target object image region as the center coordinates of the target object.

[0116] Optionally, the angle identifier determination module 530 includes an identifier reference vector determination unit, an angle identifier similarity determination unit, and an angle identifier determination unit, wherein:

[0117] The identifier reference vector determination unit is used to determine the identifier reference vector in the template image, which points from the center point of the angular identifier template to the edge point of the angular identifier template;

[0118] The angle label similarity determination unit is used to perform sliding window matching between the label reference vector and the target object image region, and to calculate the angle label similarity at each sliding window.

[0119] The angle marker determination unit is used to determine the center coordinates of the angle markers in the target object image region based on the similarity of each angle marker.

[0120] Optionally, the angle identifier similarity determination unit is specifically used for: determining the center coordinates of the sub-region corresponding to the second preset sliding window on the target object image region; determining the angle identifier edge vector located in the sub-region based on the center coordinates of the sub-region and the identifier reference vector; calculating the angle identifier similarity of the sub-region based on the angle identifier edge vector and the identifier reference vector; adjusting the sub-region corresponding to the second preset sliding window on the target object image region by a second preset step size, and returning to execute each step from determining the center coordinates of the sub-region corresponding to the second preset sliding window on the target object image region to calculating the angle identifier similarity of the sub-region based on the angle identifier edge vector and the identifier reference vector, until the traversal of each sub-region of the target object image region is completed.

[0121] Optionally, the number of identifier reference vectors determined in the template image is N, and each sub-region corresponding to the second set sliding window includes N angular identifier edge vectors that correspond one-to-one with the N identifier reference vectors; N is a positive integer greater than or equal to 2; based on the angular identifier edge vectors and identifier reference vectors, the angular identifier similarity of the sub-region is calculated, including: calculating the identifier vector similarity between the identifier reference vector and the angular identifier edge vector corresponding to the identifier reference vector; and taking the average of the N identifier vector similarities as the angular identifier similarity of the sub-region.

[0122] Optionally, the angle identifier determination unit is specifically used to: determine the angle identifier similarity with the largest similarity among all angle identifier similarities as the second similarity; determine whether the second similarity is greater than the second preset similarity; if so, determine the sub-region corresponding to the second similarity as the angle identifier region; and determine the center coordinates of the angle identifier region as the center coordinates of the angle identifier.

[0123] Optionally, the grasping angle determination module 540 includes an initial positioning vector determination unit, a detection positioning vector determination unit, and a grasping angle determination unit, wherein:

[0124] The initial positioning vector determination unit is used to obtain the initial positioning vector; the initial positioning vector is the vector in the template image obtained when the grasping angle is 0 degrees, which points from the center coordinates of the target object to the center coordinates of the angle marker.

[0125] The detection localization vector determination unit is used to determine the detection localization vector with the center coordinates of the target object in the detection image as the starting point and the center coordinates of the angle marker in the detection image as the ending point.

[0126] The grasping angle determination unit is used to determine the current grasping angle of the target object based on the angle between the detected positioning vector and the initial positioning vector.

[0127] The target object grasping angle determination device provided in the embodiments of the present invention can execute the target object grasping angle determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0128] Example 6

[0129] Figure 8 This is a schematic diagram of an electronic device according to Embodiment Six of the present invention. The electronic device is intended to represent various forms of digital computers, such as host computers, servers, or any device capable of controlling the grasping posture of a grasping machine used in industrial production. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0130] like Figure 8 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0131] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, touch screen, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0132] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, wherein the method may, for example, include: acquiring a detection image and a template image of the target object; determining, based on the detection image and the template image, the center coordinates of the target object image region and the target object in the detection image; determining, based on the target object image region and the template image, the center coordinates of an angle marker in the detection image; and determining, based on the center coordinates of the target object and the center coordinates of the angle marker, the current grasping angle of the target object.

[0133] In some embodiments, the target object grasping angle determination method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the target object grasping angle determination method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the target object grasping angle determination method by any other suitable means (e.g., by means of firmware).

[0134] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0135] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0136] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0137] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0138] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0139] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0140] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0141] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for determining the angle of grasping a target object, characterized in that, The target object is provided with an angle marker, and the method for determining the target object's grasping angle includes: Acquire the detection image and template image of the target object; Based on the detected image and the template image, determine the target image region in the detected image and the center coordinates of the target object; In the template image, an identifier reference vector is determined from the center point of the angle identifier template to the edge point of the angle identifier template; The identifier reference vector is matched with the target object image region using a sliding window, and the similarity of the angular identifier at each sliding window is calculated. Based on the similarity of each angle marker, determine the center coordinates of the angle markers in the target object image region; The current grasping angle of the target object is determined based on the center coordinates of the target object and the center coordinates of the angle indicator.

2. The method for determining the target object grasping angle according to claim 1, characterized in that, Determining the target object image region and the center coordinates of the target object in the detected image and the template image includes: In the template image, a target reference vector is determined from the center point of the target template to the edge point of the target template; The target reference vector is matched with the detected image using a sliding window, and the similarity of the target object at each sliding window is calculated. Based on the similarity of each target object, the target object image region and the center coordinates of the target object in the detected image are determined.

3. The method for determining the target object grasping angle according to claim 2, characterized in that, Perform sliding window matching between the reference vector and the detected image, and calculate the target object similarity at each sliding window, including: Determine the center coordinates of the image region corresponding to the first preset sliding window on the detected image; Based on the center coordinates of the image region and the target reference vector, determine the edge vector of the target object located within the image region; Based on the target object edge vector and the target reference vector, the similarity of the target objects in the image region is calculated; The image region corresponding to the first preset sliding window on the detection image is adjusted by a first preset step size, and then the process returns to the execution of each step from determining the coordinates of the center point of the region within the image region corresponding to the first preset sliding window on the detection image to calculating the target object similarity of the image region based on the target object edge vector and the target reference vector, until the traversal of each image region of the detection image is completed.

4. The method for determining the target object grasping angle according to claim 3, characterized in that, The number of target reference vectors determined in the template image is M, and each of the image regions corresponding to the first set sliding window includes M target object edge vectors that correspond one-to-one with the M target reference vectors; M is a positive integer greater than or equal to 2; Based on the target object edge vector and the target reference vector, the similarity of the target objects in the image region is calculated, including: Calculate the target vector similarity between the target reference vector and the target object edge vector corresponding to the target reference vector; The average of the similarities of the M target vectors is taken as the target object similarity of the image region.

5. The method for determining the target object grasping angle according to claim 2, characterized in that, Based on the similarity of each target object, the image regions of the target objects in the detected image and the center coordinates of the target objects are determined, including: The target object with the highest similarity among all the target object similarities is determined as the first similarity. Determine whether the first similarity is greater than the first preset similarity; If so, the image region corresponding to the first similarity is determined as the target object image region; The center coordinates of the target object image region are determined as the center coordinates of the target object.

6. The method for determining the target object grasping angle according to claim 1, characterized in that, The identification reference vector is matched with the target object image region using a sliding window method, and the similarity of the angular identifiers at each sliding window is calculated, including: Determine the center coordinates of the sub-region corresponding to the second set sliding window on the target object image region; Based on the center coordinates of the sub-region and the identifier reference vector, determine the angle identifier edge vector located within the sub-region; Based on the angular identifier edge vector and the identifier reference vector, the angular identifier similarity of the sub-region is calculated; The second preset step size is used to adjust the sub-region corresponding to the second preset sliding window on the target image area, and the process returns to the execution of each step from determining the center coordinates of the sub-region corresponding to the second preset sliding window on the target image area to calculating the angle label similarity of the sub-region based on the angle label edge vector and the label reference vector, until the traversal of each sub-region of the target image area is completed.

7. The method for determining the target object grasping angle according to claim 6, characterized in that, The number of the identifier reference vectors determined in the template image is N, and each sub-region corresponding to the second set sliding window includes N angle identifier edge vectors that correspond one-to-one with the N identifier reference vectors; N is a positive integer greater than or equal to 2; Based on the angular marker edge vector and the marker reference vector, the angular marker similarity of the sub-region is calculated, including: Calculate the similarity between the identifier reference vector and the identifier vector corresponding to the angular identifier edge vector; The average of the similarity of the N identifier vectors is used as the angular identifier similarity of the sub-region.

8. The method for determining the target object grasping angle according to claim 1, characterized in that, Determining the center coordinates of the angle markers in the target object image region based on the similarity of each angle marker includes: The angle identifier similarity with the highest value among all the angle identifier similarities is determined as the second similarity. Determine whether the second similarity is greater than the second preset similarity; If so, the sub-region corresponding to the second similarity is determined as the angle identification region; The center coordinates of the angle mark area are determined as the center coordinates of the angle mark.

9. The method for determining the target object grasping angle according to claim 1, characterized in that, Based on the center coordinates of the target object and the center coordinates of the angle indicator, the current grasping angle of the target object is determined, including: Obtain the initial positioning vector; the initial positioning vector is the vector in the template image obtained when the grasping angle is 0 degrees, which points from the center coordinates of the target object to the center coordinates of the angle marker. A detection positioning vector is determined with the center coordinates of the target object in the detection image as the starting point and the center coordinates of the angle marker in the detection image as the ending point. The current grasping angle of the target object is determined based on the angle between the detected positioning vector and the initial positioning vector.

Images