Multi-stage model workpiece positioning method, device, electronic equipment and storage medium

Abstract

This invention provides a multi-stage model for workpiece positioning, including a method, apparatus, electronic device, and storage medium. The method comprises: acquiring a processing image containing a workpiece to be positioned; identifying the first corner position of a corner of the workpiece in the processing image based on a first corner point model, and processing the processing image to obtain a corner point image based on the first corner point position; wherein the corner point image is a local image containing the corner points; processing the corner point image based on a second corner point model to obtain the second corner point position; determining the target corner point position based on the second corner point position and the first corner point position, and processing and positioning the workpiece based on the target corner point position. This invention enables more stable and reliable positioning of the pixel coordinates of the workpiece in the processing image, thereby supporting the precise operation of mechanical equipment.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a workpiece positioning method, apparatus, electronic device, and storage medium for a multi-stage model. Background Technology

[0002] Workpiece positioning refers to ensuring that all workpieces occupy the correct and consistent position according to processing requirements before processing, so as to achieve precise cutting, bending and other processing techniques.

[0003] Taking workpiece cutting as an example, workpiece manufacturers need to use lasers to precisely process workpieces sequentially downwards from the top of the workpiece stack. At this point, a worker can operate a workpiece electromagnet to attract the top workpiece from the stack and place it at the laser cutting position, thus cutting the workpiece.

[0004] However, manually operating the workpiece electromagnet for workpiece cutting can result in inaccurate cutting positions, leading to errors and a lower yield rate.

[0005] As a result, machine vision-based workpiece localization has emerged. Most of these methods typically use filtering to separate the workpiece image and then extract features from the image. This type of method requires edge sensitivity of the image, but when the workpiece's volume, background, lighting, or pose are affected, false detections are likely to occur during localization. Therefore, a more reliable image localization model (method) is urgently needed. Summary of the Invention

[0006] The present invention provides at least one workpiece positioning method, device, electronic device, and storage medium for a multi-stage model.

[0007] In a first aspect, embodiments of the present invention provide a workpiece positioning method based on a multi-stage model, comprising: acquiring a processing image containing a workpiece to be positioned; identifying a first corner point position of a corner point of the workpiece to be positioned in the processing image based on a first corner point model, and processing a corner point image in the processing image based on the first corner point position; wherein, the corner point image is a local image containing corner points; processing the corner point image based on a second corner point model to obtain a second corner point position of the corner point; determining a target corner point position of the corner point based on the second corner point position and the first corner point position, and processing and positioning the workpiece to be positioned based on the target corner point position.

[0008] In one optional implementation, the method of identifying the first corner position of the corner of the component to be located in the image to be processed based on the first corner model includes: performing semantic segmentation processing on the image to be processed based on the first corner model to obtain a first mask image of the corner region; wherein the corner region is a local region containing the corner; determining first binary contour information based on the first mask image, and determining the first corner position of the corner based on the first binary contour information.

[0009] In one optional implementation, determining binary contour information based on a first mask image includes: identifying false detection regions in the first mask image; wherein, a false detection region is a region in the image to be processed that does not belong to a corner region; filtering the false detection regions in the first mask image to obtain a second mask image; and determining first binary contour information based on the second mask image.

[0010] In one optional implementation, the first binary contour information includes the pixel coordinates of the contour pixels; determining the first corner point position based on the first binary contour information includes: determining the geometric centroid of the pixel coordinates of the contour pixels; and determining the first corner point position based on the geometric centroid.

[0011] In one optional implementation, the corner image is processed based on the second corner model to obtain the second corner position, including: performing semantic segmentation processing on the corner image based on the second corner model to obtain a third mask image of the corner region; wherein, the corner region is a local region containing the corner; performing image erosion processing on the third mask image to obtain a fourth mask image; determining second binary contour information based on the fourth mask image, and determining the second corner position of the corner based on the second binary contour information.

[0012] In one optional implementation, determining the target corner position of a corner point based on the second corner position and the first corner position includes: correcting the first corner position based on the second corner position to obtain the target corner position of the corner point.

[0013] In one optional implementation, the target corner position is obtained by correcting the first corner position based on the second corner position, including: determining the image center position of the corner image; determining the target offset position based on the position difference between the second corner position and the image center position; and correcting the first corner position based on the target offset position to obtain the target corner position.

[0014] Secondly, embodiments of the present invention provide a workpiece positioning device, comprising: an acquisition unit for acquiring a processing image containing a workpiece to be positioned; an identification unit for identifying a first corner position of a corner point of the workpiece to be positioned in the processing image based on a first corner point model; a processing unit for processing a corner point image in the processing image based on the first corner point position; wherein the corner point image is a local image containing corner points; a processing unit for processing the corner point image based on a second corner point model to obtain a second corner point position of the corner point; and a determination unit for determining a target corner point position of the corner point based on the second corner point position and the first corner point position, and processing and positioning the workpiece to be positioned based on the target corner point position.

[0015] Thirdly, embodiments of the present invention also provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the first aspect above, or any possible implementation of the first aspect, are performed.

[0016] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the first aspect or any possible implementation thereof.

[0017] In this embodiment of the invention, firstly, an image containing the part to be positioned is acquired. Then, based on a first corner point model, the first corner point position of the corner point of the part to be positioned can be identified in the image to be positioned, and a corner point image containing the corner point region can be obtained by processing the image to be positioned based on the first corner point position. Next, the corner point image can be processed based on a second corner point model to obtain the second corner point position of the corner point. Finally, the target corner point position of the corner point can be determined based on the first corner point position and the second corner point position, and the part to be positioned can be processed and positioned based on the target corner point position.

[0018] In the above implementation, firstly, a corner image containing corner regions is obtained from the image to be processed using a first corner model. Then, the corner positions are accurately estimated in the corner image using a second corner model. This processing method enables more stable and reliable positioning of the pixel coordinates of the part to be positioned in the image, thereby supporting the precise operation of mechanical equipment.

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to the present invention and, together with the specification, serve to explain the technical solutions of the present invention. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 A flowchart of a workpiece positioning method for a multi-stage model provided by an embodiment of the present invention is shown;

[0022] Figure 2 The flowchart illustrates a specific method for identifying the position of the first corner point of the workpiece to be positioned in the image to be processed based on the first corner point model in the workpiece positioning method of the multi-stage model provided in the embodiments of the present invention.

[0023] Figure 3 This diagram illustrates an image to be processed input to a first corner point model, as provided in an embodiment of the present invention.

[0024] Figure 4 This diagram illustrates the effect of positioning the first corner point of a component to be positioned according to an embodiment of the present invention.

[0025] Figure 5 The flowchart illustrates a specific method for processing corner images based on a second corner model to obtain the second corner position in the workpiece positioning method of the multi-stage model provided in this embodiment of the invention.

[0026] Figure 6 This diagram illustrates a corner image provided by an embodiment of the present invention;

[0027] Figure 7 This diagram illustrates the effect of a second corner point position provided by an embodiment of the present invention.

[0028] Figure 8 This diagram illustrates the effect of a target corner point position provided by an embodiment of the present invention.

[0029] Figure 9 A flowchart of another workpiece positioning method for a multi-stage model provided by an embodiment of the present invention is shown;

[0030] Figure 10 A schematic diagram of a workpiece positioning device provided in an embodiment of the present invention is shown;

[0031] Figure 11A schematic diagram of an electronic device provided by an embodiment of the present invention is shown. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0033] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0034] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0035] Research has revealed that during workpiece processing, manufacturers begin precise machining from the top of the workpiece stack downwards. Workers can manipulate electromagnets to attract the top workpiece and place it at the machining station for processing. However, this manual operation of the electromagnets results in inaccurate cutting positions, introducing errors and reducing yield. Current image recognition positioning methods lack sufficient compatibility and error tolerance for workpiece handling.

[0036] In this embodiment of the invention, the workpiece mentioned can be a basic processed part with a regular shape, such as steel, plastic sheet, or ceramic plate.

[0037] To address the aforementioned technical problems, this invention provides a workpiece positioning method, apparatus, electronic device, and storage medium using a multi-stage model. In this embodiment, firstly, an image containing the workpiece to be positioned is acquired. Then, based on a first corner point model, the first corner point position of the workpiece's corners can be identified in the image, and a corner point image containing the corner point region can be processed based on the first corner point position. Next, the corner point image can be processed based on a second corner point model to obtain the second corner point position. Finally, the target corner point position can be determined based on the first and second corner point positions, and the workpiece to be positioned can be processed and positioned based on the target corner point position.

[0038] In the above implementation, firstly, a corner image containing corner regions is obtained from the image to be processed using a first corner model. Then, the corner positions are accurately estimated in the corner image using a second corner model. This processing method enables more stable and reliable positioning of the pixel coordinates of the part to be positioned in the image, thereby supporting the precise operation of mechanical equipment.

[0039] To facilitate understanding of this embodiment, a detailed description of a multi-stage model workpiece positioning method disclosed in this invention will be provided first. The execution entity of the multi-stage model workpiece positioning method provided in this invention is generally a computer device with certain computing capabilities, such as a terminal device, server, or other processing device. In some possible implementations, this multi-stage model workpiece positioning method can be implemented by a processor calling computer-readable instructions stored in memory.

[0040] See also Figure 1 The diagram shows a flowchart of a multi-stage model workpiece positioning method provided by an embodiment of the present invention. The method includes steps S101 to S107, wherein:

[0041] S101: Obtain the image to be processed containing the part to be positioned.

[0042] Here, a camera device can be pre-installed above the area where the workpiece stack is located. The camera device can be set to capture images of the area at preset time intervals; or, it can be set to capture images of the area upon detecting a shooting command. This invention does not specifically limit the method of triggering the camera device to capture images, but only to the method that is feasible.

[0043] S103: Based on the first corner point model, identify the first corner point position of the corner point of the part to be located in the image to be processed, and process the corner point image in the image to be processed based on the first corner point position; wherein, the corner point image is a local image containing the corner point.

[0044] Here, the first corner point model can be a deep learning model, which adopts the Unet architecture, uses the EfficientNet model as the backbone (feature extraction model), and collects a large amount of data with various positions, lighting, and workpiece layers to train the initial version of the model.

[0045] After acquiring the image to be processed, it can be input into the first corner model for processing to obtain the corner recognition result of the first stage. Then, the first corner position of the corner of the part to be located can be determined based on the corner recognition result of the first stage.

[0046] Here, a corner point can be understood as a point that can be used to position and cut the part to be positioned. For example, the corner point can be a geometric corner point of the part to be positioned, or any vertex of the part to be positioned.

[0047] After identifying the location of the first corner point, a corner image containing the corner point region can be generated from the image to be processed based on the location of the first corner point. For example, a corner image of size (128x128) pixels centered on the location of the first corner point can be generated from the image to be processed.

[0048] S105: Process the corner image based on the second corner model to obtain the second corner position.

[0049] After obtaining the corner image, the corner image can be input into the second corner model for recognition processing, thereby obtaining the second corner position.

[0050] S107: Determine the target corner position based on the second corner position and the first corner position, and perform machining and positioning on the part to be positioned based on the target corner position.

[0051] Since the second corner position is estimated based on a small region of the image to be processed, it is more accurate than the first corner position. By combining the second and first corner positions, a more precise target corner position can be determined, enabling more stable and reliable machining and positioning of the part to be positioned.

[0052] In this embodiment of the invention, firstly, an image containing the part to be positioned is acquired. Then, based on a first corner point model, the first corner point position of the corner point of the part to be positioned can be identified in the image to be positioned, and a corner point image containing the corner point region can be obtained by processing the image to be positioned based on the first corner point position. Next, the corner point image can be processed based on a second corner point model to obtain the second corner point position of the corner point. Finally, the target corner point position of the corner point can be determined based on the first corner point position and the second corner point position, and the part to be positioned can be processed and positioned based on the target corner point position.

[0053] In the above implementation, firstly, a corner image containing corner regions is obtained from the image to be processed using a first corner model. Then, the corner positions are accurately estimated in the corner image using a second corner model. This processing method enables more stable and reliable positioning of the pixel coordinates of the part to be positioned in the image, thereby supporting the precise operation of mechanical equipment.

[0054] In one alternative implementation, such as Figure 2 As shown, step S103 above, based on the first corner point model, identifies the first corner point position of the corner point of the part to be located in the image to be processed, specifically including the following steps:

[0055] Step S201: Perform semantic segmentation on the image to be processed based on the first corner point model to obtain the first mask image of the corner point region; wherein, the corner point region is a local region containing corner points;

[0056] Step S202: Determine the first binary contour information based on the first mask image, and determine the first corner point position based on the first binary contour information.

[0057] In this embodiment of the invention, after acquiring the image to be processed, the image can be input into a first corner model for semantic segmentation processing to obtain a semantic segmentation result. This semantic segmentation result is used to indicate the reasoning result of the first corner model regarding the corner positions. For example, the semantic segmentation result can be a first mask image of the corner region in the image to be processed. For example, such as... Figure 3 The image to be processed, as shown in the input to the first corner point model, is as follows: Figure 4 The diagram shown illustrates the position of the first corner point of the part to be positioned.

[0058] When performing semantic segmentation on the image to be processed using the first corner model, non-corner regions are often incorrectly segmented into the first mask image. Therefore, in order to improve the accuracy of corner location recognition, it is necessary to filter the false detection regions in the first mask image. By filtering the false detection regions, the area of ​​the corner region in the first mask image can be further reduced, thereby improving the positioning accuracy of the corner.

[0059] Therefore, after obtaining the first mask image, morphological erosion processing can be performed on it. This morphological erosion process filters out false detection areas in the first mask image, thus obtaining the image erosion result. Then, binary contour information (i.e., the aforementioned first binary contour information) can be extracted based on the image erosion result. Finally, the first corner point position can be determined based on the first binary contour information.

[0060] In an optional implementation, step S202 determines the first binary contour information based on the first mask image, specifically including the following steps:

[0061] Step S2021: Identify false detection regions in the first mask image; wherein, the false detection region is a region in the image to be processed that does not belong to a corner region;

[0062] Step S2022: Filter the false detection areas in the first mask image to obtain the second mask image;

[0063] Step S2023: Determine the first binary contour information based on the second mask image.

[0064] As described above, when performing semantic segmentation on the image to be processed using the first corner model, non-corner regions are often incorrectly segmented into the first mask image. Therefore, it is necessary to identify and filter out falsely detected regions in the first mask image to obtain a second mask image with higher accuracy.

[0065] Here, false detection regions in the first mask image can be identified using a morphological erosion algorithm. The specific identification process is described below:

[0066] First, a convolutional kernel can be determined. Then, this kernel is used to traverse each pixel of the first mask image to obtain the image features of the corresponding pixels. Next, image erosion processing can be performed on the first mask image based on these image features to obtain the second mask image.

[0067] Specifically, for each pixel (x, y) in the first mask image, the product of the convolution kernel and the pixels surrounding the pixel (x, y) can be calculated and summed to obtain the result. If the calculated result is less than the sum of the values ​​of the neighboring pixels of that pixel, the pixel value of that pixel is set to 0. Through this processing method, false detection regions in the first mask image can be identified and filtered out.

[0068] After obtaining the second mask image, the outer edges of the connected regions in the second mask image can be determined, and the first binary contour information can be determined based on these outer edges. Here, the first binary contour information can also be understood as the coordinates of each pixel on the outer edge of the connected region.

[0069] Specifically, since the second mask image is a binary image, the region containing consecutive pixels with a pixel value of 0 can be considered as one connected region, and the region containing consecutive pixels with a pixel value of 1 can be considered as another connected region. Then, scanning can be performed clockwise / counterclockwise along the outermost perimeter of the connected regions, and the coordinates of the pixel boundary points with pixel values ​​of 1 / 0 can be used as the first binary contour information.

[0070] After determining the first binary contour information, the position of the first corner point can be determined based on the first binary contour information.

[0071] As described above, the first binary contour information includes the pixel coordinates of the contour pixels. In this case, step S202 above determines the position of the first corner point based on the first binary contour information, specifically including the following steps:

[0072] Step S2024: Determine the geometric centroid of the pixel coordinates of the contour pixels;

[0073] Step S2025: Determine the position of the first corner point based on the geometric centroid.

[0074] In this embodiment of the invention, the first binary contour information is a list of two-dimensional pixel coordinates, and the contour center is the geometric centroid of these two-dimensional coordinates. At this time, the geometric centroid of the pixel coordinates of the contour pixel can be determined, and then the geometric centroid is determined as the contour center, wherein the contour center is the first corner point coordinate of the corner point.

[0075] After determining the location of the first corner point, the corner point image can be obtained by processing the image to be processed based on the location of the first corner point. The specific process is described as follows:

[0076] First, determine the size information of the processing area. Then, using the first corner point as the center, determine two pixel coordinates based on this size information, for example, determine the upper left and lower right coordinates of the processing area. Next, process a small image, namely the corner point image, based on these two pixel coordinates.

[0077] For example, for a 3840x2160 image to be processed, a small 128x128 image can be created at the corner point as the corner image.

[0078] It's important to note here that since the processing area may extend beyond the boundaries, a larger image can be generated based on the image to be processed. For example, for an image to be processed that is 3840x2160, a zero-pixel image of size (3840+2*128)x(2160+2*128) can be generated first, and the image to be processed can be filled in the area of ​​(128~3840+128)x(128~2160+128). This way, when processing within the image area of ​​the image to be processed, the boundaries will not be exceeded.

[0079] After obtaining the corner image, the corner image can be processed based on the second corner model to obtain the second corner position.

[0080] In one alternative implementation, such as Figure 5 As shown, step S105 processes the corner image based on the second corner model to obtain the second corner position, specifically including the following steps:

[0081] Step S501: Perform semantic segmentation on the corner image based on the second corner model to obtain the third mask image of the corner region; wherein, the corner region is a local region containing corners;

[0082] Step S502: Perform image erosion processing on the third mask image to obtain the fourth mask image;

[0083] Step S503: Determine the second binary contour information based on the fourth mask image, and determine the second corner point position based on the second binary contour information.

[0084] After obtaining the corner image, it can be input into a second corner model for semantic segmentation to obtain the semantic segmentation result. This semantic segmentation result indicates the second corner model's inference of the corner position. For example, the semantic segmentation result can be a third mask image of the corner region in the corner image.

[0085] Here, the structures of the second corner point model and the first corner point model can be the same or different. This invention does not impose specific limitations on this, but rather focuses on what is feasible. The input image sizes for the first and second corner point models are different. The first corner point model can input very large images, which may contain many corner points; the second corner point model inputs very small images, typically containing only one corner point.

[0086] When performing semantic segmentation on corner images using the second corner model, non-corner regions are often incorrectly segmented into the third mask image. Therefore, it is necessary to identify and filter out falsely detected regions in the third mask image to obtain a fourth mask image with higher accuracy.

[0087] Here, false detection regions in the third mask image can be identified using a morphological erosion algorithm. The specific identification process is described below:

[0088] First, a convolutional kernel can be determined. Then, this kernel is used to traverse each pixel of the third mask image to obtain the image features of the corresponding pixels. Next, image erosion processing can be performed on the third mask image based on these image features to obtain the fourth mask image.

[0089] Specifically, for each pixel (x, y) in the third mask image, the product of the convolution kernel and the pixels surrounding the pixel (x, y) can be calculated and summed to obtain the result. If the calculated result is less than the sum of the values ​​of the neighboring pixels of that pixel, the pixel value of that pixel is set to 0. Through this processing method, false detection regions in the third mask image can be identified and filtered out.

[0090] After obtaining the fourth mask image, the outer edges of the connected regions in the second mask image can be determined, and the second binary contour information can be determined based on these outer edges. Here, the second binary contour information can also be understood as the coordinates of each pixel on the outer edge of the connected region.

[0091] Specifically, since the fourth mask image is a binary image, the region containing consecutive pixels with a pixel value of 0 can be considered as one connected region, and the region containing consecutive pixels with a pixel value of 1 can be considered as another connected region. Then, scanning can be performed clockwise / counterclockwise along the outermost perimeter of the connected regions, and the coordinates of the pixel boundary points with pixel values ​​of 1 / 0 can be used as the second binary contour information.

[0092] After determining the second binary contour information, the position of the second corner point can be determined based on the second binary contour information. For example, as... Figure 6 The image shown is a schematic diagram of a corner point image, such as... Figure 7 The diagram shown illustrates the effect of the second corner point position.

[0093] After determining the position of the second corner point, the target corner point position can be determined based on the positions of the second and first corner points, specifically including:

[0094] The position of the first corner point is corrected based on the position of the second corner point to obtain the target corner point position.

[0095] In this embodiment of the invention, the approximate area of ​​all corner points can be determined by the first corner point model, allowing for a certain error; the purpose of corner point positioning by the second corner point model is to find the precise coordinates within a single pixel error for all corner points, which is to correct the position of the first corner point identified by the first corner point model.

[0096] In an optional implementation, the above steps correct the position of the first corner point based on the position of the second corner point to obtain the target corner point position, specifically including the following steps:

[0097] First, determine the image center position of the corner point image;

[0098] Secondly, the target offset position is determined based on the positional difference between the second corner point position and the image center position;

[0099] Finally, the position of the first corner point is corrected based on the target offset position to obtain the target corner point position.

[0100] Here, the process of correcting the position of the first corner point by using the position of the second corner point involves identifying the position of the corner point within the corner image (i.e., the position of the second corner point). Specifically, the position of the first corner point can be adjusted based on the difference between the position of the second corner point and the center position of the corner image.

[0101] First, the image center position of the corner point image can be determined. Then, the position difference between the second corner point position and the image center position is calculated, for example, the coordinate difference. Then, the first corner point position and the target offset position (i.e., the coordinate difference) are summed to obtain the target corner point position.

[0102] For example, if the first corner point is identified in a 3840x2160 image to be processed, a 128x128 image (i.e., a corner point image) can be processed centered on this first corner point location. Then, this image can be used as input to the second corner point model to predict the coordinates of the second corner point. The final target corner point coordinates are then:

[0103] The formula is: First-stage corner coordinates + (Second-stage corner coordinates – Second-stage image center coordinates), which is also: First-stage corner coordinates + Second-stage offset; where the first-stage corner coordinates are the position of the first corner point, the second-stage corner coordinates are the position of the second corner point, and the second-stage image center coordinates are the position of the image center. For example... Figure 8 The image shown is a schematic diagram illustrating the effect of the target corner point position.

[0104] In the above implementation, firstly, a corner image containing corner regions is obtained from the image to be processed using a first corner model. Then, the corner positions are accurately estimated in the corner image using a second corner model. This processing method enables more stable and reliable positioning of the pixel coordinates of the part to be positioned in the image, thereby supporting the precise operation of mechanical equipment.

[0105] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0106] The following will combine Figure 9 The workpiece positioning method of the above multi-stage model is introduced below, and the specific process is described as follows:

[0107] S1: The image to be processed is captured.

[0108] Here, a camera device can be pre-installed above the area where the workpiece stack is located. The camera device can be set to capture images of the area at preset time intervals; or, it can be set to capture images of the area upon detecting a capture command, thereby obtaining the image to be processed.

[0109] S2: The image to be processed is semantically segmented using a one-stage corner model to obtain the corner region mask.

[0110] After acquiring the image to be processed, the image can be converted in resolution and then input into a first-stage corner model (i.e., the first corner model) for semantic segmentation to obtain the semantic segmentation result. For example, the image resolution of the image to be processed can be converted to 640*352. This semantic segmentation result is used to indicate the inference result of the first-stage corner model regarding the corner positions. For example, the semantic segmentation result can be a corner region mask in the image to be processed, where the corner region mask is the first mask image in the above embodiment.

[0111] S3: Perform image morphological erosion processing on the corner region mask.

[0112] When performing semantic segmentation on an image using a one-stage corner model, non-corner regions are often incorrectly segmented into the first mask image. Therefore, to improve the accuracy of corner location recognition, it is necessary to filter out false detection regions in the first mask image. By filtering out false detection regions, the area of ​​corner regions in the first mask image can be further reduced, thereby improving the accuracy of corner location.

[0113] Therefore, after obtaining the first mask image, morphological erosion processing can be performed on the first mask image. Through morphological erosion processing, false detection areas in the first mask image can be filtered out, thereby obtaining the image erosion result (i.e., the second mask image).

[0114] S4: Contour localization and contour center calculation based on image erosion results.

[0115] Here, binary contour information (i.e., the first binary contour information mentioned above) can be extracted based on the image erosion results. Then, the position of the first corner point can be determined based on the first binary contour information.

[0116] Specifically, after obtaining the second mask image, the outer edges of the connected regions in the second mask image can be determined, and the first binary contour information can be determined based on these outer edges. Here, the first binary contour information can also be understood as the coordinates of each pixel on the outer edge of the connected region.

[0117] Here, the first binary contour information is a list of two-dimensional pixel coordinates, and the contour center is the geometric centroid of these two-dimensional coordinates. At this point, the geometric centroid of the pixel coordinates of the contour pixels can be determined, and then this geometric centroid is determined as the contour center, where the contour center is the first corner point coordinate of the corner point.

[0118] S5: Input the processing result of the first-stage corner point model, i.e., the position of the first corner point.

[0119] S6: Image processing.

[0120] The corner image is obtained by processing the image to be processed based on the position of the first corner point.

[0121] First, determine the size information of the processing area. Then, using the first corner point as the center, determine two pixel coordinates based on this size information, for example, determine the upper left and lower right coordinates of the processing area. Next, process a small image, namely the corner point image, based on these two pixel coordinates.

[0122] For example, for a 3840x2160 image to be processed, a small 128x128 image can be created at the corner point as the corner image.

[0123] It's important to note here that since the processing area may extend beyond the boundaries, a larger image can be generated based on the image to be processed. For example, for an image to be processed that is 3840x2160, a zero-pixel image of size (3840+2*128)x(2160+2*128) can be generated first, and the image to be processed can be filled in the area of ​​(128~3840+128)x(128~2160+128). This way, when processing within the image area of ​​the image to be processed, the boundaries will not be exceeded.

[0124] S7: The corner image is semantically segmented using a two-stage corner model to obtain the third mask image of the corner region.

[0125] After obtaining the corner image, it can be input into a second corner model (i.e., a two-stage corner model) for semantic segmentation to obtain the semantic segmentation result. This semantic segmentation result indicates the second corner model's inference of the corner position. For example, the semantic segmentation result can be a third mask image of the corner region in the corner image.

[0126] S8: Perform image morphological erosion processing on the third mask image.

[0127] Here, false detection regions in the third mask image can be identified using a morphological erosion algorithm. The specific identification process is described below:

[0128] First, a convolutional kernel can be determined. Then, this kernel is used to traverse each pixel of the third mask image to obtain the image features of the corresponding pixels. Next, image erosion processing can be performed on the third mask image based on these image features to obtain the fourth mask image.

[0129] Specifically, for each pixel (x, y) in the third mask image, the product of the convolution kernel and the pixels surrounding the pixel (x, y) can be calculated and summed to obtain the result. If the calculated result is less than the sum of the values ​​of the neighboring pixels of that pixel, the pixel value of that pixel is set to 0. Through this processing method, false detection regions in the third mask image can be identified and filtered out.

[0130] S9: Contour mask extraction.

[0131] After obtaining the fourth mask image, the outer edges of the connected regions in the second mask image can be determined, and the second binary contour information can be determined based on these outer edges. Here, the second binary contour information can also be understood as the coordinates of each pixel on the outer edge of the connected region.

[0132] S10: Calculate the geometric centroid of the maximum profile.

[0133] After determining the second binary contour information, the position of the second corner point can be determined based on the second binary contour information.

[0134] S11: Coordinate correction.

[0135] The position of the first corner point is corrected based on the position of the second corner point to obtain the target corner point position.

[0136] In the above implementation, firstly, a corner image containing corner regions is obtained from the image to be processed using a first corner model. Then, the corner positions are accurately estimated in the corner image using a second corner model. This processing method enables more stable and reliable positioning of the pixel coordinates of the part to be positioned in the image, thereby supporting the precise operation of mechanical equipment.

[0137] Based on the same inventive concept, this embodiment of the invention also provides a workpiece positioning device corresponding to the workpiece positioning method of the multi-stage model. Since the principle of the device in this embodiment of the invention is similar to the workpiece positioning method of the multi-stage model described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0138] Reference Figure 10 The diagram shown is a schematic representation of a workpiece positioning device according to an embodiment of the present invention. The device includes: an acquisition unit 10, an identification unit 20, a processing unit 30, a handling unit 40, and a determination unit 50; wherein,

[0139] The acquisition unit 10 is used to acquire an image to be processed containing the part to be positioned;

[0140] The recognition unit 20 is used to identify the first corner point position of the corner point of the part to be located in the image to be processed based on the first corner point model;

[0141] The processing unit 30 is used to process the image to be processed based on the position of the first corner point to obtain a corner point image; wherein, the corner point image is a local image containing the corner points;

[0142] Processing unit 40 is used to process the corner image based on the second corner model to obtain the second corner position of the corner.

[0143] The determining unit 50 is used to determine the target corner position of the corner based on the second corner position and the first corner position, and to process and position the part to be positioned based on the target corner position.

[0144] In this embodiment of the invention, firstly, an image containing the part to be positioned is acquired. Then, based on a first corner point model, the first corner point position of the corner point of the part to be positioned can be identified in the image to be positioned, and a corner point image containing the corner point region can be obtained by processing the image to be positioned based on the first corner point position. Next, the corner point image can be processed based on a second corner point model to obtain the second corner point position of the corner point. Finally, the target corner point position of the corner point can be determined based on the first corner point position and the second corner point position, and the part to be positioned can be processed and positioned based on the target corner point position.

[0145] In the above implementation, firstly, a corner image containing corner regions is obtained from the image to be processed using a first corner model. Then, the corner positions are accurately estimated in the corner image using a second corner model. This processing method enables more stable and reliable positioning of the pixel coordinates of the part to be positioned in the image, thereby supporting the precise operation of mechanical equipment.

[0146] In one possible implementation, the identification unit is further configured to: perform semantic segmentation processing on the image to be processed based on the first corner point model to obtain a first mask image of the corner point region; wherein, the corner point region is a local region containing corner points; determine first binary contour information based on the first mask image, and determine the first corner point position of the corner point based on the first binary contour information.

[0147] In one possible implementation, the identification unit is further configured to: identify false detection regions in the first mask image; wherein the false detection regions are regions in the image to be processed that do not belong to corner regions; filter the false detection regions in the first mask image to obtain a second mask image; and determine first binary contour information based on the second mask image.

[0148] In one possible implementation, the identification unit is further configured to: determine the geometric centroid of the pixel coordinates of the contour pixel when the first binary contour information contains the pixel coordinates of the contour pixel; and determine the position of the first corner point based on the geometric centroid.

[0149] In one possible implementation, the processing unit is further configured to: perform semantic segmentation processing on the corner image based on the second corner model to obtain a third mask image of the corner region; wherein the corner region is a local region containing corners; perform image erosion processing on the third mask image to obtain a fourth mask image; determine second binary contour information based on the fourth mask image, and determine the second corner position of the corner based on the second binary contour information.

[0150] In one possible implementation, the determining unit is further configured to: correct the position of the first corner point based on the position of the second corner point to obtain the target corner point position.

[0151] In one possible implementation, the determining unit is further configured to: determine the image center position of the corner point image; determine the target offset position based on the position difference between the second corner point position and the image center position; and correct the first corner point position based on the target offset position to obtain the target corner point position.

[0152] The description of the processing flow of each module in the device and the interaction flow between each module can be found in the relevant descriptions in the above method embodiments, and will not be detailed here.

[0153] Corresponding to Figure 1 The invention also provides an electronic device 700, such as a multi-stage model for workpiece positioning. Figure 11 The diagram shown is a structural schematic of an electronic device 700 provided in an embodiment of the present invention, comprising:

[0154] The system includes a processor 71, a memory 72, and a bus 73. The memory 72 stores execution instructions and includes main memory 721 and external memory 722. The main memory 721, also called internal memory, temporarily stores the computational data in the processor 71 and the data exchanged with external memory 722 such as a hard disk. The processor 71 exchanges data with the external memory 722 through the main memory 721. When the electronic device 700 is running, the processor 71 communicates with the memory 72 through the bus 73, causing the processor 71 to execute the following instructions:

[0155] Obtain the image to be processed containing the part to be positioned;

[0156] Based on the first corner point model, the first corner point position of the corner point of the component to be located is identified in the image to be processed, and a corner point image is obtained by processing the image to be processed based on the first corner point position; wherein, the corner point image is a local image containing the corner point;

[0157] The corner image is processed based on the second corner model to obtain the second corner position of the corner.

[0158] The target corner position of the corner point is determined based on the second corner position and the first corner position, and the part to be positioned is processed and positioned based on the target corner position.

[0159] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program performs the steps of the workpiece positioning method for a multi-stage model described in the above method embodiments. The storage medium can be either volatile or non-volatile computer-readable storage.

[0160] This invention also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the workpiece positioning method of the multi-stage model described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0161] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0162] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided by this invention, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0163] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0164] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0165] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0166] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A workpiece positioning method using a multi-stage model, characterized in that, include: Obtain the image to be processed containing the part to be positioned; Based on the first corner point model, the first corner point position of the corner point of the component to be located is identified in the image to be processed. A corner image is obtained by processing the image to be processed based on the position of the first corner point; wherein, the corner image is a local image containing the corner point; The corner image is processed based on the second corner model to obtain the second corner position of the corner. The target corner position of the corner point is determined based on the second corner position and the first corner position, and the part to be positioned is processed and positioned based on the target corner position; The step of identifying the first corner point position of the corner point of the component to be located in the image to be processed based on the first corner point model includes: Based on the first corner model, semantic segmentation processing is performed on the image to be processed to obtain a first mask image of the corner region; wherein, the corner region is a local region containing the corner; First binary contour information is determined based on the first mask image, and the first corner point position of the corner point is determined based on the first binary contour information; Determining the first binary contour information based on the first mask image includes: Identify false detection regions in the first mask image; wherein, the false detection regions are regions in the image to be processed that do not belong to corner regions; Filter out the false detection areas in the first mask image to obtain the second mask image; The first binary contour information is determined based on the second mask image; The first binary contour information includes the pixel coordinates of the contour pixels; determining the first corner point position based on the first binary contour information includes: Determine the geometric centroid of the pixel coordinates of the contour pixels; The position of the first corner point is determined based on the geometric centroid.

2. The method according to claim 1, characterized in that, The step of processing the corner image based on the second corner model to obtain the second corner position of the corner includes: Based on the second corner point model, semantic segmentation processing is performed on the corner point image to obtain a third mask image of the corner point region; wherein, the corner point region is a local region containing the corner point; The third mask image is subjected to image erosion processing to obtain the fourth mask image; The second binary contour information is determined based on the fourth mask image, and the second corner point position of the corner point is determined based on the second binary contour information.

3. The method according to claim 1, characterized in that, Determining the target corner position based on the second corner position and the first corner position includes: The first corner point position is corrected based on the second corner point position to obtain the target corner point position.

4. The method according to claim 3, characterized in that, The step of correcting the position of the first corner point based on the position of the second corner point to obtain the target corner point position includes: Determine the image center position of the corner point image; The target offset position is determined based on the positional difference between the second corner point position and the image center position; The position of the first corner point is corrected based on the target offset position to obtain the target corner point position.

5. A workpiece positioning device, characterized in that, include: The acquisition unit is used to acquire an image of the part to be positioned that is to be processed. The identification unit is used to identify the first corner point position of the corner point of the component to be located in the image to be processed based on the first corner point model. A processing unit is configured to process the image to be processed based on the position of the first corner point to obtain a corner point image; wherein, the corner point image is a local image containing the corner point; The processing unit is used to process the corner image based on the second corner model to obtain the second corner position of the corner. The determining unit is configured to determine the target corner position of the corner point based on the second corner position and the first corner position, and to perform processing and positioning on the part to be positioned based on the target corner position; The step of identifying the first corner point position of the corner point of the component to be located in the image to be processed based on the first corner point model includes: Based on the first corner model, semantic segmentation processing is performed on the image to be processed to obtain a first mask image of the corner region; wherein, the corner region is a local region containing the corner; First binary contour information is determined based on the first mask image, and the first corner point position of the corner point is determined based on the first binary contour information; Determining the first binary contour information based on the first mask image includes: Identify false detection regions in the first mask image; wherein, the false detection regions are regions in the image to be processed that do not belong to corner regions; Filter out the false detection areas in the first mask image to obtain the second mask image; The first binary contour information is determined based on the second mask image; The first binary contour information includes the pixel coordinates of the contour pixels; determining the first corner point position based on the first binary contour information includes: Determine the geometric centroid of the pixel coordinates of the contour pixels; The position of the first corner point is determined based on the geometric centroid.

6. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the workpiece positioning steps as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the workpiece positioning steps as described in any one of claims 1 to 4.

Images