Workpiece positioning method and device, electronic equipment and storage medium

Abstract

The invention discloses a workpiece positioning method and device, electronic equipment and a storage medium, and belongs to the technical field of machine vision, the workpiece positioning method comprises the steps that a workpiece sample image set and a workpiece template image are acquired, and a detection frame and a plurality of key points located in the detection frame are marked on the workpiece template image; determining target labeling information of each workpiece sample graph in the workpiece sample graph set according to the workpiece template graph; training a key point identification model according to the workpiece sample atlas and the target labeling information; and according to the trained key point identification model and the workpiece template drawing, workpiece positioning is carried out, so that positioning of workpieces in images shot in various complex scenes can be efficiently and stably realized, the positioning accuracy and reliability are improved, the application range is wide, and the flexibility is high.

Description

Technical Field

[0001] The present application belongs to the field of machine vision technology, and in particular relates to a workpiece positioning method, device, electronic device and storage medium. Background Art

[0002] In recent years, with the development of industrial production automation and the increase in labor costs, robotics technology has been widely used in the production field.

[0003] In machine vision, visual positioning is an important function. By positioning the target (such as the edge position of the workpiece) in the captured workpiece image, it can provide positioning guidance for robots and other equipment, and can also perform further image processing on the positioning image. However, for existing visual positioning methods, when the contrast of the workpiece image is low, there are many edge noise points, or the shooting scene is complex, the positioning effect and accuracy are difficult to guarantee, and it is easy to have problems such as failure to position or positioning errors, which in turn affects the subsequent robot's operation of the workpiece. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a workpiece positioning method, device, electronic device and storage medium, which can achieve accurate positioning of the workpiece and improve the positioning effect.

[0005] In a first aspect, the present application provides a workpiece positioning method, comprising:

[0006] Acquire a workpiece sample atlas and a workpiece template map, wherein the workpiece template map is marked with a detection frame and a plurality of key points located in the detection frame;

[0007] Determine target annotation information of each workpiece sample image in the workpiece sample image set according to the workpiece template image;

[0008] Training a key point recognition model according to the workpiece sample atlas and the target annotation information;

[0009] Workpiece positioning is performed according to the trained key point recognition model and the workpiece template map.

[0010] In some embodiments, determining the target annotation information of each workpiece sample image in the workpiece sample image set according to the workpiece template image includes:

[0011] Using a geometric positioning tool, annotate each workpiece sample image in the workpiece sample image set according to the workpiece template image to obtain rough annotation information;

[0012] The coarse labeling information is fine-tuned to obtain target labeling information.

[0013] In some embodiments, the use of a geometric positioning tool to label each workpiece sample image in the workpiece sample image set according to the workpiece template image to obtain rough labeling information includes:

[0014] A geometric positioning tool is used to determine a mapping frame corresponding to the detection frame on the workpiece template image and a mapping point corresponding to the key point on each workpiece sample image, and the mapping frame and the mapping point are used as rough annotation information.

[0015] In some embodiments, the performing workpiece positioning according to the trained key point recognition model and the workpiece template map includes:

[0016] Extracting a target area image from the workpiece image to be located according to the workpiece template image;

[0017] The target area image is processed using the trained key point recognition model to identify key points in the target area image to obtain a first key point set.

[0018] In some embodiments, extracting the target area image from the to-be-located workpiece image according to the workpiece template image includes:

[0019] A geometric positioning tool is used to determine a target frame on the workpiece image to be positioned that corresponds to the detection frame on the workpiece template image, and an image corresponding to the target frame on the workpiece image to be positioned is extracted to obtain a target area image.

[0020] In some embodiments, the workpiece positioning method further includes:

[0021] Acquire a reference workpiece image, wherein the reference workpiece image is marked with a second key point set;

[0022] Determine a translation amount according to the first key point set and the second key point set;

[0023] The robot is controlled according to the translation amount.

[0024] In some embodiments, determining the translation amount according to the first key point set and the second key point set includes:

[0025] Respectively translating the geometric center corresponding to the first key point set and the geometric center corresponding to the second key point set to the origin to obtain a corresponding first translated point set and a second translated point set;

[0026] Determine a rotation matrix according to the first translation point set and the second translation point set;

[0027] Rotate the second key point set according to the rotation matrix;

[0028] The translation amount is determined according to the second key point set and the first key point set after rotation.

[0029] In a second aspect, the present application provides a workpiece positioning device, comprising:

[0030] An acquisition module, used for acquiring a workpiece sample atlas and a workpiece template map, wherein the workpiece template map is marked with a detection frame and a plurality of key points located in the detection frame;

[0031] A determination module, configured to determine target annotation information of each workpiece sample image in the workpiece sample image set according to the workpiece template image;

[0032] A training module, used for training a key point recognition model according to the workpiece sample atlas and the target annotation information;

[0033] The positioning module is used to locate the workpiece according to the trained key point recognition model and the workpiece template map.

[0034] In some embodiments, the determining module is specifically used to:

[0035] Using a geometric positioning tool, annotate each workpiece sample image in the workpiece sample image set according to the workpiece template image to obtain rough annotation information;

[0036] The coarse labeling information is fine-tuned to obtain target labeling information.

[0037] In some embodiments, the determining module is specifically used to:

[0038] A geometric positioning tool is used to determine a mapping frame corresponding to the detection frame on the workpiece template image and a mapping point corresponding to the key point on each workpiece sample image, and the mapping frame and the mapping point are used as rough annotation information.

[0039] In some embodiments, the positioning module is specifically used to:

[0040] Extracting a target area image from the workpiece image to be located according to the workpiece template image;

[0041] The target area image is processed using the trained key point recognition model to identify key points in the target area image to obtain a first key point set.

[0042] In some embodiments, the positioning module is specifically used to:

[0043] A geometric positioning tool is used to determine a target frame on the workpiece image to be positioned that corresponds to the detection frame on the workpiece template image, and an image corresponding to the target frame on the workpiece image to be positioned is extracted to obtain a target area image.

[0044] In some embodiments, the workpiece positioning device further includes a control module for:

[0045] Acquire a reference workpiece image, wherein the reference workpiece image is marked with a second key point set;

[0046] Determine a translation amount according to the first key point set and the second key point set;

[0047] The robot is controlled according to the translation amount.

[0048] In some embodiments, the control module is specifically used to:

[0049] Respectively translating the geometric center corresponding to the first key point set and the geometric center corresponding to the second key point set to the origin to obtain a corresponding first translated point set and a second translated point set;

[0050] Determine a rotation matrix according to the first translation point set and the second translation point set;

[0051] Rotate the second key point set according to the rotation matrix;

[0052] The translation amount is determined according to the second key point set and the first key point set after rotation.

[0053] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the above-described workpiece positioning methods.

[0054] In a fourth aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-described workpiece positioning methods when executing the program.

[0055] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements any of the above-mentioned workpiece positioning methods when executed by a processor.

[0056] The workpiece positioning method, device, storage medium, electronic device and computer program product provided in the embodiments of the present application obtain a workpiece sample atlas and a workpiece template map, on which a detection frame and multiple key points located in the detection frame are marked; target marking information of each workpiece sample map in the workpiece sample atlas is determined according to the workpiece template map; a key point recognition model is trained according to the workpiece sample atlas and the target marking information; workpiece positioning is performed according to the trained key point recognition model and the workpiece template map, that is, the workpiece sample atlas is marked according to the workpiece template map, and a model is trained using the marked workpiece sample atlas, and workpiece positioning is performed based on the training model and the workpiece template map, so that the positioning of the workpiece in the images taken in various complex scenes can be achieved efficiently and stably, and the positioning accuracy and reliability can be improved, and the application range is wide and the flexibility is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0058] Figure 1 It is a schematic diagram of the flow of the workpiece positioning method provided in an embodiment of the present application;

[0059] Figure 2 is another schematic diagram of the process of the workpiece positioning method provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram of the structure of a workpiece positioning device provided in an embodiment of the present application;

[0061] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0062] Figure 5 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0063] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0064] Embodiments of the present application provide a workpiece positioning method, device, storage medium, electronic device, and computer program product.

[0065] See also Figure 1, Figure 1 1 is a flow chart of a workpiece positioning method provided in an embodiment of the present application. The workpiece positioning method is applied to an electronic device and can be executed by the electronic device or a functional module or functional entity in the electronic device. The workpiece positioning method includes the following steps 101-104, wherein:

[0066] 101. Obtain a workpiece sample atlas and a workpiece template image, where a detection frame and a plurality of key points located in the detection frame are marked on the workpiece template image;

[0067] 102. Determine target annotation information of each workpiece sample image in the workpiece sample image set according to the workpiece template image;

[0068] 103. Training a key point recognition model according to the workpiece sample atlas and the target annotation information;

[0069] 104. Workpiece positioning is performed according to the trained key point recognition model and the workpiece template image.

[0070] The workpiece positioning method provided in the embodiment of the present application obtains a workpiece sample atlas and a workpiece template image, wherein the workpiece template image is marked with a detection frame and a plurality of key points located in the detection frame; determines the target marking information of each workpiece sample image in the workpiece sample atlas according to the workpiece template image; trains a key point recognition model according to the workpiece sample atlas and the target marking information; and performs workpiece positioning according to the trained key point recognition model and the workpiece template image, that is, annotates the workpiece sample atlas according to the workpiece template image, and uses the annotated workpiece sample atlas training model to perform workpiece positioning based on the training model and the workpiece template image, thereby being able to efficiently and stably realize the positioning of workpieces in images taken in various complex scenes, improve the positioning accuracy and reliability, and has a wide range of applications and high flexibility.

[0071] The above steps 101 - 104 will be further described in detail below.

[0072] 101. Obtain a workpiece sample atlas and a workpiece template image, where a detection frame and a plurality of key points located in the detection frame are marked on the workpiece template image.

[0073] Among them, the workpiece sample atlas and the workpiece template map can be images taken of the workpieces on the production line. The detection frame and key points on the workpiece template map are manually marked, and the number of key points is manually determined according to the actual scene. Usually, the key points are some points on the outer contour of the workpiece, and their positions can reflect the placement position and angle of the workpiece and other information. The area surrounded by the detection frame is a rectangular area including the key points, and its size can be manually set according to the actual workpiece size. In other embodiments, the detection frame can also be other shapes, such as a circle, which is not limited here.

[0074] 102. Determine target annotation information of each workpiece sample image in the workpiece sample image set according to the workpiece template image.

[0075] In some embodiments, see Figure 2 , Figure 2 1 is another flow chart of the workpiece positioning method provided in an embodiment of the present application, wherein the above step 102 specifically includes:

[0076] 1021. Using a geometric positioning tool, annotate each workpiece sample image in the workpiece sample image set according to the workpiece template image to obtain rough annotation information;

[0077] 1022. Fine-tune the rough annotation information to obtain target annotation information.

[0078] Among them, the geometric positioning tool is a technology that locates based on geometric information such as the shape, edge, and contour of an object. It locates the position of the object in the image by detecting the contour features of the object in the image and comparing it with the preset template. In industrial scenarios, the shapes, sizes, and fixed positions of the workpieces on the production line are roughly the same, so the workpiece template image with manually marked key points and detection frames can be used as a template, and the geometric positioning tool can be used to realize automatic annotation of the workpiece sample atlas, eliminating the need for manual annotation by each person, greatly saving annotation time and improving annotation speed. However, since geometric positioning is inaccurate in images taken in complex environments, the rough annotation information obtained by automatic annotation may need to be partially manually fine-tuned to finally obtain accurate and reliable target annotation information.

[0079] Furthermore, the above step 1021 specifically includes:

[0080] By using a geometric positioning tool, a mapping frame corresponding to a detection frame on the workpiece template image and a mapping point corresponding to a key point on each workpiece sample image are determined, and the mapping frame and the mapping point are used as coarse annotation information.

[0081] Among them, the geometric positioning tool can map the detection box and key points on the workpiece template image to the coordinate system of the workpiece sample image, thereby realizing automatic labeling of the detection box and key points on the workpiece sample image.

[0082] 103. Train a key point recognition model according to the workpiece sample atlas and the target annotation information.

[0083] Among them, the key point recognition model can be an RTMPose model. The key point recognition model is trained by using a sample atlas of artifacts with accurate target annotation information, that is, the model parameters are adjusted. The key point recognition model obtained can subsequently recognize artifact images taken in various complex scenes, thereby improving recognition accuracy and reliability.

[0084] 104. Workpiece positioning is performed according to the trained key point recognition model and the workpiece template image.

[0085] In some embodiments, see Figure 2 , the above step 104 specifically includes:

[0086] 1041. Extracting a target area image from the to-be-located workpiece image according to the workpiece template image;

[0087] 1042. Process the target area image using the trained key point recognition model to identify key points in the target area image and obtain a first key point set.

[0088] In some embodiments, the above step 1041 specifically includes:

[0089] By using the geometric positioning tool, a target frame on the workpiece image to be positioned corresponding to the detection frame on the workpiece template image is determined, and an image corresponding to the target frame on the workpiece image to be positioned is extracted to obtain a target area image.

[0090] That is, after the key point recognition model is trained, in the subsequent model application stage, if you want to locate the workpiece on the workpiece image to be located, you need to first use the workpiece template image used in the model training stage, and use the geometric positioning tool to map the detection box marked on the workpiece template image to the workpiece image to be located, and obtain the target frame. Then extract the image of the area surrounded by the target frame and input it into the key point recognition model for key point recognition. The identified key point position can determine the workpiece position and angle to achieve the positioning of the workpiece.

[0091] In other embodiments, if workpiece positioning is to be achieved, two methods can be used. One method is to specially train a recognition model for a target frame and a recognition model for key points, and use the recognition model of the target frame to identify the target frame on the workpiece image to be positioned, and use the recognition model of the key points to identify the key points in the target frame. That is, this workpiece positioning method requires the training of two recognition models in advance, which not only requires the preparation of a large number of training samples, but also the sequential operation of the two models in the model application stage increases the time consumption to a certain extent and reduces the recognition rate. Another method is to achieve it completely based on the traditional geometric positioning method, that is, using an image to create a template, and then by calculating the similarity between the workpiece image to be positioned and the edge features in the template image, the key points are recognized. Although this method can improve the recognition efficiency, if the contrast of the workpiece image to be positioned is low, there are many edge noise points, or the shooting scene is complex, the positioning effect and accuracy are difficult to guarantee, and it is easy to have problems such as failure to position or positioning errors.

[0092] The embodiment of the present application only needs to train one recognition model for key points. In the model application stage, the target frame in the workpiece image to be located is first recognized based on the geometric positioning method, and then the recognition model is used to recognize the key points in the target frame. This not only makes up for the problem of inaccurate recognition caused by relying solely on the geometric positioning method, but also has high recognition accuracy and reliability, and avoids multiple training and use of a recognition model, thereby improving recognition efficiency. After testing on the CPU, the recognition time of a single image can be reduced from the original approximately 70ms to 20ms.

[0093] In some embodiments, see Figure 2 After identifying the first key point set in the target area image, the workpiece positioning method may further include:

[0094] 1051. Obtain a reference workpiece image, where a second key point set is marked on the reference workpiece image;

[0095] 1052. Determine a translation amount according to the first key point set and the second key point set;

[0096] 1053. Control the robot according to the translation amount.

[0097] Among them, in the positioning task, it is necessary to output an affine transformation relationship, that is, the transformation relationship of the workpiece image to be positioned relative to the reference workpiece image. Based on this relationship, functions such as visual guidance can be realized, expanding the application scenarios. The reference workpiece image is an image of the workpiece at the target position. By calculating the translation amount between the first key point set in the workpiece image to be positioned and the second key point set in the reference workpiece image, and controlling the robot based on the translation amount, for example, the robot controls the mechanical arm to clamp and move the workpiece according to the translation amount, so as to clamp the workpiece from the current position to the target position.

[0098] In some embodiments, the above step 1052 specifically includes:

[0099] The geometric center corresponding to the first key point set and the geometric center corresponding to the second key point set are respectively translated to the origin to obtain the corresponding first translated point set and second translated point set;

[0100] Determine a rotation matrix according to the first translation point set and the second translation point set;

[0101] Rotate the second key point set according to the rotation matrix;

[0102] The translation amount is determined according to the rotated second key point set and the first key point set.

[0103] Wherein, assuming that the second key point set in the reference workpiece image is P, and the first key point set in the workpiece image to be positioned is Q, then the following steps 1) to 4) are performed to determine the translation amount.

[0104] 1) Translate the geometric centers of the two sets of points to the origin

[0105] For the two sets of point sets (P, Q) before translation, first calculate the geometric center (centroid) of each set of point sets, then subtract the coordinates of the corresponding geometric center from each point in each set of point sets, and then translate the geometric centers of the two sets of point sets to the origin. The relevant formula is as follows:

[0106]

[0107]

[0108] P-=centroid(P);

[0109] Q-=centroid(Q);

[0110] Among them, centroid(P) is the geometric center of point set P, centroid(Q) is the geometric center of point set Q, N is the number of key points, i∈[1,N].

[0111] 2) Calculate the covariance matrix

[0112] The calculation formula of the covariance matrix H is:

[0113] H=P T Q;

[0114] 3) Calculate the rotation matrix

[0115] First, perform singular value decomposition (SVD) on the covariance matrix H.

[0116] That is, H = USV T , U and V are orthogonal matrices.

[0117] Afterwards, in order to ensure that the coordinate system obtained after the rotation operation is still a right-handed coordinate system, it is necessary to perform a sign check on the SVD result.

[0118] That is, d = sign [Det (UV T )], where sign(x) refers to the sign of x. When x>0, sign(x)=1; when x<0, sign(x)=-1.

[0119] The optimal rotation matrix is

[0120]

[0121] Afterwards, P is rotated according to R, that is, P=PR, to obtain the rotated P.

[0122] 4) Calculate translation

[0123] Calculate the distance between each point of the rotated point set P in step 3) and the original point set Q, and then find the average to get the translation between the two point sets.

[0124] From the above, it can be seen that the workpiece positioning method provided in the embodiment of the present application obtains a workpiece sample atlas and a workpiece template map, and the workpiece template map is marked with a detection frame and a plurality of key points located in the detection frame; determines the target annotation information of each workpiece sample map in the workpiece sample atlas according to the workpiece template map; trains the key point recognition model according to the workpiece sample atlas and the target annotation information; and performs workpiece positioning according to the trained key point recognition model and the workpiece template map, that is, annotates the workpiece sample atlas according to the workpiece template map, and uses the annotated workpiece sample atlas training model to perform workpiece positioning based on the training model and the workpiece template map, so as to efficiently and stably realize the positioning of workpieces in images taken in various complex scenes, improve the positioning accuracy and reliability, and have a wide range of applications and high flexibility.

[0125] According to the method described in the above embodiment, the present application also provides a workpiece positioning device for performing the steps in the above workpiece positioning method. Figure 3 , Figure 3 2 is a schematic diagram of the structure of a workpiece positioning device 200 provided in an embodiment of the present application. The workpiece positioning device 200 is applied to an electronic device, and the workpiece positioning method can be executed by the electronic device or a functional module or functional entity in the electronic device.

[0126] Specifically, the workpiece positioning device 200 includes an acquisition module 201, a determination module 202, a training module 203 and a positioning module 204, wherein:

[0127] An acquisition module 201 is used to acquire a workpiece sample atlas and a workpiece template map, where a detection frame and a plurality of key points located in the detection frame are marked on the workpiece template map;

[0128] A determination module 202 is used to determine target annotation information of each workpiece sample image in the workpiece sample image set according to the workpiece template image;

[0129] A training module 203 is used to train a key point recognition model according to the workpiece sample atlas and the target annotation information;

[0130] The positioning module 204 is used to perform workpiece positioning according to the trained key point recognition model and the workpiece template image.

[0131] In some embodiments, the determination module 202 is specifically configured to:

[0132] Using a geometric positioning tool, each workpiece sample image in the workpiece sample image set is annotated according to the workpiece template image to obtain rough annotation information;

[0133] The coarse annotation information is fine-tuned to obtain the target annotation information.

[0134] In some embodiments, the determination module 202 is specifically configured to:

[0135] By using a geometric positioning tool, a mapping frame corresponding to the detection frame on the workpiece template image and a mapping point corresponding to the key point on each workpiece sample image are determined, and the mapping frame and the mapping point are used as coarse annotation information.

[0136] In some embodiments, the positioning module 204 is specifically used to:

[0137] Extracting a target area image from the workpiece image to be located according to the workpiece template image;

[0138] The target area image is processed using the trained key point recognition model to identify key points in the target area image to obtain a first key point set.

[0139] In some embodiments, the positioning module 204 is specifically used to:

[0140] By using a geometric positioning tool, a target frame on the workpiece image to be positioned corresponding to a detection frame on the workpiece template image is determined, and an image corresponding to the target frame on the workpiece image to be positioned is extracted to obtain a target area image.

[0141] In some embodiments, the workpiece positioning device 200 further includes a control module for:

[0142] Acquire a reference workpiece image, on which a second key point set is marked;

[0143] Determine a translation amount according to the first key point set and the second key point set;

[0144] The robot is controlled according to the translation amount.

[0145] In some embodiments, the control module is specifically used to:

[0146] The geometric center corresponding to the first key point set and the geometric center corresponding to the second key point set are respectively translated to the origin to obtain the corresponding first translated point set and second translated point set;

[0147] Determine a rotation matrix according to the first translation point set and the second translation point set;

[0148] Rotate the second key point set according to the rotation matrix;

[0149] The translation amount is determined according to the rotated second key point set and the first key point set.

[0150] It should be noted that the specific details of each module unit in the above-mentioned workpiece positioning device 200 have been described in detail in the embodiment of the above-mentioned workpiece positioning method, and will not be repeated here.

[0151] As can be seen from the above, the workpiece positioning device 200 provided in the embodiment of the present application obtains the workpiece sample atlas and the workpiece template map through the acquisition module 201, and the workpiece template map is marked with a detection frame and a plurality of key points located in the detection frame; the determination module 202 determines the target annotation information of each workpiece sample map in the workpiece sample atlas according to the workpiece template map; the training module 203 trains the key point recognition model according to the workpiece sample atlas and the target annotation information; the positioning module 204 performs workpiece positioning according to the trained key point recognition model and the workpiece template map, that is, annotates the workpiece sample atlas according to the workpiece template map, and uses the annotated workpiece sample atlas training model to perform workpiece positioning based on the training model and the workpiece template map, so as to efficiently and stably realize the positioning of the workpiece in the images taken in various complex scenes, improve the positioning accuracy and reliability, and has a wide range of applications and high flexibility.

[0152] In some embodiments, the workpiece positioning device in the embodiments of the present application can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a server, which can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The embodiments of the present application are not specifically limited.

[0153] In some embodiments, Figure 4 As shown, an embodiment of the present application further provides an electronic device 300, including a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the program is executed by the processor 301, each process of the above-mentioned workpiece positioning method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0154] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0155] Figure 5 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0156] The electronic device 400 includes but is not limited to: a radio frequency unit 401, a network module 402, an audio output unit 403, an input unit 404, a sensor 405, a display unit 406, a user input unit 407, an interface unit 408, a memory 409, a processor 410 and other components.

[0157] Those skilled in the art will appreciate that the electronic device 400 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 410 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 5 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0158] It should be understood that in the embodiment of the present application, the input unit 404 may include a graphics processor (Graphics Processing Unit, GPU) 4041 and a microphone 4042, and the graphics processor 4041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 406 may include a display panel 4061, and the display panel 4061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 407 includes a touch panel 4071 and at least one of other input devices 4072. The touch panel 4071 is also called a touch screen. The touch panel 4071 may include two parts: a touch detection device and a touch controller. Other input devices 4072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0159] The memory 409 can be used to store software programs and various data. The memory 409 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 409 may include a volatile memory or a non-volatile memory, or the memory 409 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 409 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0160] The processor 410 may include one or more processing units; the processor 410 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 410.

[0161] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned workpiece positioning method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0162] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0163] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned workpiece positioning method when executed by a processor.

[0164] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0165] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0166] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0167] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0168] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0169] In the description of the present application, “plurality” means two or more.

[0170] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0171] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A workpiece positioning method, characterized in that: include: Acquire a workpiece sample atlas and a workpiece template map, wherein the workpiece template map is marked with a detection frame and a plurality of key points located in the detection frame; Determine target annotation information of each workpiece sample image in the workpiece sample image set according to the workpiece template image; Training a key point recognition model according to the workpiece sample atlas and the target annotation information; Workpiece positioning is performed according to the trained key point recognition model and the workpiece template map.

2. The workpiece positioning method according to claim 1, characterized in that: The step of determining target annotation information of each workpiece sample image in the workpiece sample image set according to the workpiece template image includes: Using a geometric positioning tool, annotate each workpiece sample image in the workpiece sample image set according to the workpiece template image to obtain rough annotation information; The coarse labeling information is fine-tuned to obtain target labeling information.

3. The workpiece positioning method according to claim 2, characterized in that: The method of using a geometric positioning tool to mark each workpiece sample image in the workpiece sample image set according to the workpiece template image to obtain rough marking information includes: A geometric positioning tool is used to determine a mapping frame corresponding to the detection frame on the workpiece template image and a mapping point corresponding to the key point on each workpiece sample image, and the mapping frame and the mapping point are used as rough annotation information.

4. The workpiece positioning method according to claim 1, characterized in that: The step of positioning the workpiece according to the trained key point recognition model and the workpiece template map includes: Extracting a target area image from the workpiece image to be located according to the workpiece template image; The target area image is processed using the trained key point recognition model to identify key points in the target area image to obtain a first key point set.

5. The workpiece positioning method according to claim 4, characterized in that: The step of extracting a target area image from the workpiece image to be positioned according to the workpiece template image comprises: A geometric positioning tool is used to determine a target frame on the workpiece image to be positioned that corresponds to the detection frame on the workpiece template image, and an image corresponding to the target frame on the workpiece image to be positioned is extracted to obtain a target area image.

6. The workpiece positioning method according to claim 4, characterized in that: The workpiece positioning method also includes: Acquire a reference workpiece image, wherein the reference workpiece image is marked with a second key point set; Determine a translation amount according to the first key point set and the second key point set; The robot is controlled according to the translation amount.

7. The workpiece positioning method according to claim 6, characterized in that: The determining of the translation amount according to the first key point set and the second key point set includes: Respectively translating the geometric center corresponding to the first key point set and the geometric center corresponding to the second key point set to the origin to obtain a corresponding first translated point set and a second translated point set; Determine a rotation matrix according to the first translation point set and the second translation point set; Rotate the second key point set according to the rotation matrix; The translation amount is determined according to the second key point set and the first key point set after rotation.

8. A workpiece positioning device, characterized in that: include: An acquisition module, used for acquiring a workpiece sample atlas and a workpiece template map, wherein the workpiece template map is marked with a detection frame and a plurality of key points located in the detection frame; A determination module, configured to determine target annotation information of each workpiece sample image in the workpiece sample image set according to the workpiece template image; A training module, used for training a key point recognition model according to the workpiece sample atlas and the target annotation information; The positioning module is used to locate the workpiece according to the trained key point recognition model and the workpiece template map.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the workpiece positioning method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the workpiece positioning method according to any one of claims 1 to 7 is implemented.

Images