Labeling and localization method based on linemod matching and Hough transform

Abstract

This application relates to a labeling and positioning method based on Linemod matching and Hough transform, as well as an apparatus, electronic device, and storage medium. The method includes: acquiring a target image to be labeled and positioned; processing the target image using a multi-template matching positioning model to obtain a weld target region in the target image where a weld seam appears, wherein the multi-template matching positioning model is trained and generated based on a preset training template image and measured information of the weld seam appearing in the preset training template image; performing edge extraction on the target image to obtain a first metal can image; and performing vertical line detection within the region corresponding to the weld target region in the first metal can image to obtain the weld center coordinate data corresponding to the target image and determine the labeling position. This application solves the problems in related technologies where it is difficult to achieve label positioning by rotating and scanning large-mass metal cans, and where some positioning is inaccurate, achieving the beneficial effect of efficient and accurate labeling and positioning for various types of metal cans.

Description

Technical Field

[0001] This application relates to the field of labeling and positioning technology, and in particular to a labeling and positioning method based on linemod matching and Hough transform. Background Technology

[0002] Metal cans are widely used in industrial production, such as filling chemical coatings and paints, and packaging food products like milk powder, tea, and biscuits. Therefore, labeling metal cans has become an indispensable basic process in production lines. Currently, factories mainly use two methods. The first is manual labeling of the sides of metal cans. This method is not only labor-intensive but also inefficient. Furthermore, factors such as visual fatigue can cause inaccurate labeling, leading to missed or incorrect labels, making it difficult to meet the high precision and efficiency requirements of manufacturing enterprises. With the rapid development of modern industrial production, automatic labeling systems have emerged. Automatic labeling systems are devices that can affix rolls of paper or foil labels to specified packaging containers or products. The labels have adhesive backing and are arranged regularly on a glossy backing paper. The peeling mechanism on the automatic labeling system automatically peels them off. Labeling can perform various operations such as flat labeling, single-sided or multi-sided labeling of packaging, cylindrical labeling, partial or full coverage of cylindrical labels, and labeling of recessed and corner areas.

[0003] Automatic labeling systems for metal cans must be fast, accurate, and automate the labeling process. Therefore, mainstream automatic labeling systems in the industry employ automatic photoelectric tracking sensors, featuring functions such as no labeling without a label, automatic label correction, and automatic label detection to avoid label waste and omissions. However, a problem exists: while small metal cans can be positioned using rotational scanning, large metal cans, such as paint buckets, cannot be easily rotated, making label positioning difficult. Furthermore, considering the front and back sides of the can, photoelectric recognition becomes challenging, and colors like white, yellow, and transparent can cause sensor malfunctions, leading to inaccurate label positioning.

[0004] Currently, no effective solution has been proposed for the problems of difficulty in rotating and scanning large metal cans to achieve label positioning and inaccurate positioning in related technologies. Summary of the Invention

[0005] This application provides a labeling positioning method, apparatus, electronic device, and storage medium based on linemod matching and Hough transform, to at least solve the problems in related technologies such as difficulty in rotating and scanning large-mass metal cans to achieve label positioning and inaccurate positioning in some cases.

[0006] In a first aspect, embodiments of this application provide a labeling location method based on linemod matching and Hough transform, comprising: acquiring a target image to be labeled and located, wherein the target image is used to represent the body image of the metal can; processing the target image using a multi-template matching location model to obtain a weld target region in the target image where a weld seam appears, wherein the multi-template matching location model is trained and generated based on a preset training template image and measured region information of the weld seam appearing in the preset training template image; performing edge extraction on the target image to obtain a first metal can image; performing vertical line detection in the region of the first metal can image corresponding to the weld target region to obtain weld center coordinate data corresponding to the target image; and determining the labeling position based on the weld center coordinate data.

[0007] Secondly, embodiments of this application provide a labeling and positioning device based on linemod matching and Hough transform, comprising:

[0008] The acquisition module is used to acquire the target image to be labeled and positioned, wherein the target image is used to represent the body image of the metal can.

[0009] The matching module is used to process the target image using a multi-template matching localization model to obtain the weld target region in the target image where the weld seam appears. The multi-template matching localization model is generated by training based on a preset training template image and the measured region information of the weld seam in the preset training template image.

[0010] The positioning module is used to extract edges from the target image to obtain a first metal can image, perform vertical line detection in the area of ​​the first metal can image corresponding to the weld target area to obtain the weld center coordinate data corresponding to the target image, and determine the labeling position based on the weld center coordinate data.

[0011] Thirdly, embodiments of this application provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the labeling and positioning method based on linemod matching and Hough transform as described in the first aspect.

[0012] Fourthly, embodiments of this application provide a storage medium storing a computer program that, when executed by a processor, implements the labeling and positioning method based on linemod matching and Hough transform as described in the first aspect above.

[0013] Compared to related technologies, the labeling positioning method, apparatus, electronic device, and storage medium based on linemod matching and Hough transform provided in this application embodiment acquire a target image to be labeled, wherein the target image is used to represent the body image of the metal can; process the target image using a multi-template matching positioning model to obtain the weld target area in the target image where the weld seam appears, wherein the multi-template matching positioning model is trained and generated based on a preset training template image and the measured area information of the weld seam appearing in the preset training template image; perform edge extraction on the target image to obtain a first metal can image; perform vertical line detection in the area of ​​the first metal can image corresponding to the weld target area to obtain the weld center coordinate data corresponding to the target image, and determine the labeling position based on the weld center coordinate data; solve the problems of difficulty in rotating and scanning to achieve label positioning for large-mass metal cans and inaccurate positioning in related technologies, and achieve the beneficial effect of efficient and accurate labeling positioning for various metal cans.

[0014] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0016] Figure 1 This is a hardware structure block diagram of a terminal based on the labeling and positioning method of linemod matching and Hough transform according to an embodiment of this application.

[0017] Figure 2 This is a flowchart of a labeling and positioning method based on linemod matching and Hough transform according to an embodiment of this application;

[0018] Figure 3 This is a flowchart illustrating the construction process of a multi-template matching and localization model according to an embodiment of this application;

[0019] Figure 4 This is a flowchart of a labeling and positioning method based on linemod matching and Hough transform according to a preferred embodiment of this application;

[0020] Figure 5 This is a structural block diagram of a labeling and positioning device based on linemod matching and Hough transform according to an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application. Furthermore, it is understood that although the efforts made in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, modifications to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0022] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0023] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application means two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The terms “first,” “second,” “third,” etc., used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0024] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. Taking running on a terminal as an example, Figure 1 This is a hardware structure block diagram of the terminal for the labeling and positioning method based on linemod matching and Hough transform according to an embodiment of this application. Figure 1 As shown, terminal 10 may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, terminal 10 may also include components that are larger than those described above. Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0025] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the labeling and positioning method based on linemod matching and Hough transform in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0026] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0027] This embodiment provides a labeling and localization method based on linemod matching and Hough transform that runs on the aforementioned terminal. Figure 2 This is a flowchart of a labeling and positioning method based on linemod matching and Hough transform according to an embodiment of this application, as shown below. Figure 2As shown, the process includes the following steps:

[0028] Step S201: Obtain the target image to be labeled and positioned, wherein the target image is used to represent the body image of the metal can.

[0029] In this embodiment, by acquiring the target image to be labeled and positioned, the actual production model is converted into image data that can be recognized and processed by a computer. This allows for further processing and calculation of the image data using a multi-template matching positioning model, which is the foundation for achieving automatic labeling and positioning and ultimately improves the efficiency and accuracy of labeling and positioning. The image of the can body refers to the image of the side of the metal can, that is, when the metal can is placed horizontally, the camera lens is kept parallel to the horizontal plane. Since the labeling position in production is generally on the side of the metal can (i.e., the can body), directly using the image of the can body to capture the target image can not only eliminate interference and improve the accuracy of positioning, but also simplify the actual operation, reduce the amount of calculation, and improve the efficiency of positioning. Considering that under actual production conditions, the metal cans to be labeled and positioned are mostly in a state of continuous movement on the assembly line, when the weld seam is outside the field of view of the lens (or the weld seam is in the field of view of the lens but the angle is relatively large relative to the lens), it is necessary to rotate the metal can for recognition to improve the recognition accuracy, which will seriously affect the production efficiency. Therefore, it is preferable to use multiple angles (covering as much as possible 360° of the side perimeter) to acquire multiple target images at the same time. This not only solves the problem of difficulty in rotating and scanning large metal cans to achieve label positioning, but also allows for image capture without stopping, improving production efficiency and recognition accuracy.

[0030] Step S202: The target image is processed using a multi-template matching localization model to obtain the target area of ​​the weld seam in the target image. The multi-template matching localization model is generated by training based on the preset training template image and the measured area information of the weld seam in the preset training template image.

[0031] In this embodiment, by using a multi-template matching localization model to process the target image, the target area of ​​the weld seam in the target image is obtained, which reduces the amount of computation and achieves efficient and accurate weld seam area recognition with short training time.

[0032] Step S203: Edge extraction is performed on the target image to obtain the first metal can image. Vertical line detection is performed in the area corresponding to the weld target area in the first metal can image to obtain the weld center coordinate data corresponding to the target image, and the labeling position is determined based on the weld center coordinate data.

[0033] In this embodiment, by performing edge extraction on the target image, a first metal can image is obtained. Within the area of ​​the first metal can image corresponding to the weld target region, vertical line detection is performed to obtain the weld center coordinate data corresponding to the target image. The labeling position is then determined based on the weld center coordinate data, which makes the labeling positioning more efficient. Due to the structure of the metal can itself, there are few basic straight line features. Using edge extraction and vertical line detection can efficiently identify the straight line features of the weld, reducing the amount of computation and improving the efficiency of labeling positioning.

[0034] Through the above steps S201 to S203, the target image to be labeled and positioned is obtained, wherein the target image is used to represent the body image of the metal can; the target image is processed using a multi-template matching positioning model to obtain the weld target area in the target image where the weld seam appears, wherein the multi-template matching positioning model is trained and generated based on the preset training template image and the measured area information of the weld seam in the preset training template image; edge extraction is performed on the target image to obtain the first metal can image; vertical line detection is performed in the area of ​​the first metal can image corresponding to the weld seam target area to obtain the weld center coordinate data corresponding to the target image; and the labeling position is determined based on the weld center coordinate data. This solves the problem in related technologies that it is difficult to achieve label positioning by rotating and scanning large-mass metal cans and that some positioning is inaccurate, and achieves the beneficial effect of efficient and accurate labeling and positioning of various metal cans.

[0035] It should be noted that in this embodiment, the automatic recognition of the metal can image based on machine vision and digital image processing technology is used to locate the weld position of the metal can in real time. The rotation angle of the can required for labeling is calculated based on the weld position, which is the prerequisite for the labeling system to perform the labeling operation. This achieves automatic labeling by determining the labeling position after image acquisition and recognition, which not only improves labeling efficiency and accuracy but also reduces label waste and thus lowers costs. By simultaneously capturing multiple target images from different angles, the problem of difficulty in rotating and scanning large metal cans for label positioning is solved. Furthermore, the conveyor belt can pass through without stopping during image capture, only slowing down, improving production efficiency. It also addresses the influence of factors such as the height of the metal can, distortion, and exposure in the image under different environments and perspectives, improving recognition accuracy. The Linemod multi-template matching method requires no lengthy training time and has the advantages of real-time performance and fast algorithm speed, fully meeting the efficiency requirements of production. After obtaining the target area of ​​the weld, it is used as a mask. Combined with basic image processing techniques such as image grayscale conversion, Canny operator and Hough transform, not only are unnecessary details in the metal can image eliminated, making it highly interpretable, but also requiring less computation, thus achieving fast and reliable labeling positioning.

[0036] In some embodiments, the base uses a multi-template matching localization model to process the target image to obtain the weld target region in the target image where the weld seam appears, including the following steps:

[0037] Step 1: Blur the target image to obtain the second metal can image. The blurring process includes Gaussian blurring.

[0038] Step 2: Process the image of the second metal can using the Linemod multi-template matching algorithm to obtain the matching degree between each region of the target image and the preset weld template. The weld template is generated by the multi-template matching localization model during training.

[0039] Step 3: Select the area where the matching degree exceeds the matching degree threshold as the target area for the weld.

[0040] The second metal can image is obtained by blurring the target image in the above steps, where the blurring includes Gaussian blurring. The second metal can image is then processed using the Linemod multi-template matching algorithm to obtain the matching degree between each region of the target image and the preset weld template, where the weld template is generated during the training of the multi-template matching localization model. Regions with matching degrees exceeding the matching degree threshold are selected as weld target regions, which reduces the computational load of the model and achieves efficient and accurate weld target region recognition, thereby improving the efficiency of labeling and localization.

[0041] In some embodiments, the Linemod multi-template matching algorithm is used to process the image of the second metal can to obtain the matching degree between each region of the target image and the preset weld template, including the following steps:

[0042] Step 1: Using multiple weld templates, perform horizontal and vertical sliding window matching sequentially in the image of the second metal can to obtain multiple similarity values. Each sliding window matching generates one similarity value.

[0043] Step 2: Superimpose all similarity values ​​obtained based on the same weld template to obtain the matching degree of the target image to the weld template.

[0044] By utilizing multiple weld seam templates in the above steps, horizontal and vertical sliding window matching is sequentially performed in the second metal can image to obtain multiple similarity values. Each sliding window matching generates one similarity value. All similarity values ​​obtained based on the same weld seam template are superimposed to obtain the matching degree of the target image to that weld seam template. This achieves rapid identification of the weld seam target area, while the accuracy is within an acceptable range, thereby improving the efficiency of labeling and positioning.

[0045] In some embodiments, edge extraction is performed on the target image to obtain a first metal can image, including the following steps:

[0046] Step 1: Convert the target image to grayscale to obtain the image of the third metal can.

[0047] Step 2: Extract the edges of the third metal can image using an edge detection operator to obtain the first metal can image. The edge detection operator includes the Canny operator.

[0048] The target image is converted to grayscale through the above steps to obtain the third metal can image; the edge of the third metal can image is extracted by the edge detection operator to obtain the first metal can image. The edge detection operator includes the Canny operator, and the low error rate of the Canny operator ensures the accuracy and reliability of the labeling positioning.

[0049] In some embodiments, vertical line detection is performed within the region of the first metal can image corresponding to the weld target area to obtain the weld center coordinate data corresponding to the target image, including the following steps:

[0050] Step 1: Use the target area of ​​the weld as a mask and perform an AND operation with the first metal can image to obtain the fourth metal can image.

[0051] Step 2: Perform vertical line detection on the image of the fourth metal can to obtain the weld center coordinate data corresponding to the target image. Vertical line detection includes Hough transform line detection.

[0052] By using the target area of ​​the weld as a mask in the above steps and performing an AND operation with the first metal can image, a fourth metal can image is obtained. Vertical line detection is then performed on the fourth metal can image to obtain the weld center coordinate data corresponding to the target image. Vertical line detection includes Hough transform line detection. After obtaining the target area of ​​the weld, it is used as a mask. Combined with basic image processing techniques such as image grayscale conversion, Canny operator, and Hough transform, this method not only offers high interpretability and accuracy but also requires less computation, thus improving the efficiency of labeling and positioning.

[0053] In some embodiments, determining the labeling position based on weld center coordinate data includes the following steps:

[0054] Step 1: Based on the distance and angle data set when the target image was captured, convert the weld center coordinate data into spatial position information; wherein, the distance and angle data includes: the distance between the lens and the metal can when the target image was captured, and the angle between the lens and the direction of movement of the metal can.

[0055] Step 2: Determine the labeling location based on spatial location information and the height data corresponding to the labeling area.

[0056] By converting the image-based data obtained from the model into actual three-dimensional spatial position, labeling positioning can be realized. At this time, the labeling mechanism is rotated by a corresponding angle or the metal can is rotated by a corresponding angle to find the horizontal position of the labeling area. Then, the height is adjusted according to the vertical distance of the labeling area, so that the labeling mechanism and the labeling area are perfectly aligned to complete the labeling task.

[0057] In some embodiments, the weld center coordinates corresponding to the target image are converted into spatial position information based on the distance and angle data set when the target image was captured, including the following steps:

[0058] Step 1: By searching the preset calibration point coordinate and calibration angle correspondence table, obtain the two calibration points that are closest to the horizontal coordinate in the weld center coordinate data.

[0059] Step 2: Use a linear interpolation algorithm to obtain the physical angle between the horizontal coordinate of the weld center coordinate data and the forward direction of the metal can by applying a linear interpolation algorithm to the calibration angles corresponding to the two calibration points.

[0060] Step 3: Generate corresponding spatial location information based on the physical angle and the distance and angle data set when the target image was captured.

[0061] Since the lens focal length, the distance from the metal can, and the angle between the lens and the direction of the metal can's movement are all preset when the target image is captured, the relevant spatial position information can be calculated based on the imaging principle and physical angles. Although the physical angles obtained through linear interpolation algorithms inevitably have some errors, they are within the expected range. This process reduces the amount of computation, which can improve the calculation speed and thus improve the efficiency of labeling and positioning.

[0062] The following section will introduce the construction and training methods of neural network models (multi-template matching localization models). Figure 3 This is a flowchart illustrating the construction process of a multi-template matching and localization model according to an embodiment of this application, such as... Figure 3 As shown, it includes the following steps:

[0063] Step S301: Obtain the preset training template image. Before obtaining the preset training template image, the direction and zero point are calibrated: the direction perpendicular to the screen outward is defined as 90°, and the direction parallel to the screen to the left is defined as 0°. The measurement range is [36°, -144°], with a counterclockwise step of 1°. Starting from 36°, the measurement is moved sequentially with a step of 1° until 144°, capturing training template images with and without the can handle occlusion. The naming format of the captured images is strictly defined to obtain a data image set. Two images with and two without the can handle occlusion are taken from each angle, for a total of 218 images.

[0064] Step S302: Construct a multi-template matching localization model.

[0065] Step S303: Train the constructed multi-template matching and positioning model to extract the weld template.

[0066] The training process primarily uses the two-dimensional Linemod algorithm, which includes feature point feature vector calculation, coordinate calculation, and information storage during the training phase, thereby further improving the matching speed.

[0067] Among them, the reference image {O} is aligned according to the given two-dimensional image. m} m Based on the gradient and normal vector direction features of ∈M, the following template feature set acquisition formula is defined:

[0068] Γ=({O m} m ∈M,P(r,m))

[0069] Where O represents the template feature set, including gradient direction features and normal vector features; m represents an image in the data image set; M represents the data image set; P represents the image feature location tuple, including the location r of the feature and the corresponding preset training template image m.

[0070] Step S304 yields the trained multi-template matching localization model.

[0071] The embodiments of this application will be described and illustrated below through preferred embodiments.

[0072] Figure 4 This is a flowchart of a labeling and positioning method based on linemod matching and Hough transform according to a preferred embodiment of this application. Figure 4 As shown, the labeling positioning method includes the following steps:

[0073] Step S401: Obtain the preset training template image. First, calibrate the direction and zero point: take the direction perpendicular to the screen outward as 90°, and the direction parallel to the screen to the left as 0°, and take the measurement range [36°, -144°] in counterclockwise increments of 1°. Start measuring from 36°, and move sequentially in increments of 1° until 144° to capture training template images with and without the can handle occlusion. Strictly define the naming format of the captured images to obtain a data image set. Take two images with and two images without the can handle occlusion for each angle, for a total of 218 images.

[0074] Step S402: Construct a multi-template matching localization model.

[0075] Step S403: Train the constructed multi-template matching and positioning model, extract the weld template, and obtain the trained multi-template matching and positioning model.

[0076] Step S404: Acquire the target image to be labeled and positioned. Considering the difficulty of angle conversion and without affecting the normal operation of the conveyor belt, four cameras are used simultaneously to capture the target image to be labeled and positioned. The four cameras are placed adjacent to each other perpendicularly, each at a 45° angle to the conveyor belt, and horizontally relative to the metal can. To capture more features of the weld area and reduce errors caused by camera lens distortion, the metal can is positioned as centrally as possible to obtain the largest possible image area.

[0077] Step S405: Apply Gaussian blur to the target image to obtain the second metal can image. Gaussian blur is an image blur filter that uses a normal distribution to calculate the transformation of each pixel in the image. The formula for calculating Gaussian blur is:

[0078]

[0079] Where u is the object distance, v is the image distance, and θ is the blur radius. 2 =u 2 +v 2 σ is the standard deviation of the normal distribution, e is the natural logarithm, and π is pi. In two-dimensional space, the contour lines of the surface generated by this formula are concentric circles that are normally distributed starting from the center. The convolution matrix composed of pixels with non-zero distributions is used to transform the original image. The value of each pixel is a weighted average of the values ​​of its surrounding pixels. Gaussian blur is used here to reduce the image resolution and improve the computation speed.

[0080] Step S406: The Linemod multi-template matching algorithm is used to process the image of the second metal can to obtain the target area of ​​the weld seam of the metal can. The input test image will be processed by simply determining the width and height of the image. The calculation of the feature vectors and coordinate positions of each pixel is the same as in the training stage. After obtaining the feature vectors and coordinate positions of the pixels, the Linemod multi-template matching algorithm pre-creates n0 lookup tables for each discrete direction for the subsequent matching search process with binary strings. The index number of the lookup table corresponds to the string, and the index value corresponds to the cosine value of the pixel position and the discrete feature direction. This completes the pixel feature processing, direction diffusion, response table construction, and linearized storage of gradient direction quantization values ​​of the target image to be labeled and located.

[0081] The similarity calculation involved involves creating a lookup table and storing the gradient direction quantization values. The Linemod algorithm then extracts the weld template feature point information from the training phase. By performing horizontal and vertical sliding window matching on the test image using the weld template, it compares the gradient value differences between the weld template feature points and the corresponding pixels in the target image to be labeled with the gradient in the lookup table, generating a two-dimensional similarity matrix and completing one sliding window matching of the test image.

[0082] The test image is continuously matched with the weld template information at different pyramid levels using a sliding window, resulting in different similarities (gradient direction cosine similarity). These similarities are then superimposed based on the same template to obtain the overall similarity. In this way, a target image to be labeled and located receives different overall matching scores for different positions, directions, and angles of the weld template. The matching score is used to determine whether a match is successful.

[0083] The formula for calculating similarity in the location matching process is shown below.

[0084] ε({L m} m ∈M,Γ,c)=∑ (r，m)∈P |max t∈R (c+r)f m (O m (r), L m (t))|

[0085] In the formula: ε represents similarity; Γ represents the template feature set collection formula; L m Let m be the input image for the matching process, where m represents an image in the image dataset; M represents the image dataset; c represents the positions of image m; f m R represents the function used to calculate the similarity between position r in the template image and position t in the input image; that is, it is a similarity function that calculates the cosine of the angle between the gradient directions or the angle between the normal vector directions of the two positions. R represents a window region, and its calculation formula is... This represents a region of size T centered at position c+r in the input image Lm; t represents a certain position in region R; the matching degree is compared with a pre-set threshold τ. When the matching degree is higher than the threshold τ, the template matching is successful, the primary template matching stage is completed, and the trained feature vector and feature point positions are returned. Then, the training information and region positions of the entire template are returned.

[0086] Step S407: Perform grayscale processing on the target image to obtain the third metal can image.

[0087] Step S408: Edge extraction is performed on the third metal can image using the Canny operator to obtain the first metal can image.

[0088] Step S409: Use the target area of ​​the weld as a mask and perform an AND operation with the first metal can image to obtain the fourth metal can image.

[0089] Step S410: Perform Hough transform on the image of the fourth metal can to detect vertical lines and obtain the weld center coordinate data corresponding to the target image. The processed image has been cropped, so post-processing is required to convert it to the coordinates of the original image for easier angle calibration later. The returned coordinate data is shown in Table 1 below:

[0090]

[0091]

[0092] Table 1

[0093] Step S411: Determine the labeling position based on the weld center coordinates data corresponding to the target image. This involves finding the two closest calibration points to the horizontal coordinate in the weld center coordinate data by searching a pre-defined table of calibration point coordinates and calibration angles. A linear interpolation algorithm is then used to obtain the physical angle between the current weld horizontal coordinate and the direction of travel of the metal can. Based on this physical angle and the distance-angle data set when the target image was captured, corresponding spatial position information is generated. While the physical angle obtained through linear interpolation inevitably contains some error, this is within the expected range. This is because the Linemod template does not guarantee that the weld is centered on the template image during extraction, and the template center coordinates are subsequently used as the weld center coordinates, leading to a deviation. Therefore, the angle inherently has some error when establishing the calibration point coordinates and calibration angle correspondence table. This process reduces computational load and increases calculation speed, thereby improving the efficiency of labeling positioning. Further, more precise template extraction and improved interpolation methods can make the results more accurate. Increasing the number of template images in subsequent steps can also improve matching accuracy and obtain more accurate weld center point coordinates. As shown in the image above, the handle's obstruction does not affect the recognition effect at these two angles, and the recognition results at the same angle are quite similar. Because the lens captures the metal can at an angle greater than 90°, the weld seam may be recognized simultaneously in two target images for labeling. The closer to the image center, the more sparse the calibration points, and the higher the accuracy after angle conversion. Therefore, when the weld seam is recognized in two target images, the result closer to the image center is taken.

[0094] It should be noted that the steps shown in the above process or in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here. For example, steps S401 and S402, steps S401 and S404, and steps S402 and S404.

[0095] This embodiment also provides a labeling and positioning device based on Linemod matching and Hough transform. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0096] Figure 5 This is a structural block diagram of a labeling and positioning device based on linemod matching and Hough transform according to an embodiment of this application, as shown below. Figure 5 As shown, the device includes: an acquisition module 51, a matching module 52, and a positioning module 53.

[0097] The acquisition module 51 is used to acquire the target image to be labeled and positioned, wherein the target image is used to represent the image of the metal can body.

[0098] The matching module 52 is coupled to the acquisition module 51 and is used to process the target image using a multi-template matching localization model to obtain the target area of ​​the weld seam in the target image. The multi-template matching localization model is trained and generated based on the preset training template image and the measured area information of the weld seam in the preset training template image.

[0099] The positioning module 53 is coupled to the matching module 52 and is used to extract the edge of the target image to obtain the first metal can image. In the area of ​​the first metal can image corresponding to the weld target area, vertical line detection is performed to obtain the weld center coordinate data corresponding to the target image, and the labeling position is determined according to the weld center coordinate data.

[0100] The labeling and positioning device based on Linemod matching and Hough transform provided in this application embodiment acquires a target image to be labeled and positioned, wherein the target image represents the body image of a metal can; processes the target image using a multi-template matching positioning model to obtain the weld target area where a weld seam appears in the target image, wherein the multi-template matching positioning model is trained and generated based on a preset training template image and the measured area information of the weld seam appearing in the preset training template image; performs edge extraction on the target image to obtain a first metal can image; performs vertical line detection within the area corresponding to the weld seam target area in the first metal can image to obtain the weld center coordinate data corresponding to the target image; and determines the labeling position based on the weld center coordinate data. This solves the problems in related technologies where it is difficult to achieve label positioning by rotating and scanning large-mass metal cans, and where some positioning is inaccurate, achieving the beneficial effect of efficient and accurate labeling and positioning for various types of metal cans.

[0101] In some embodiments, the matching module 52 further includes:

[0102] The first processing unit is used to blur the target image to obtain the second metal can image, wherein the blurring process includes Gaussian blurring.

[0103] The first computing unit, coupled to the first processing unit, is used to process the image of the second metal can using the multi-template matching algorithm Linemod to obtain the matching degree between each region of the target image and the preset weld template, wherein the weld template is generated by the multi-template matching localization model during the training process.

[0104] The first selection unit, coupled to the first calculation unit, is used to select regions with a matching degree exceeding the matching degree threshold as the target area of ​​the weld.

[0105] In some embodiments, the first computing unit further includes:

[0106] The matching component is used to perform horizontal and vertical sliding window matching sequentially in the second metal can image using multiple weld templates to obtain multiple similarity values. Each sliding window matching generates one similarity value.

[0107] The overlay component, coupled to the matching component, is used to overlay all similarity values ​​obtained based on the same weld template to obtain the matching degree of the target image corresponding to the weld template.

[0108] In some embodiments, the first selection unit further includes:

[0109] The grayscale processing component is used to perform grayscale processing on the target image to obtain the image of the third metal can;

[0110] An edge extraction component, coupled to a grayscale processing component, extracts edges from a third metal can image using an edge detection operator to obtain a first metal can image. The edge detection operator includes the Canny operator.

[0111] In some embodiments, the first selection unit is used to use the weld target area as a mask and perform an AND operation with the first metal can image to obtain a fourth metal can image; the fourth metal can image is subjected to vertical line detection to obtain the weld center coordinate data corresponding to the target image, wherein the vertical line detection includes Hough transform line detection.

[0112] In some embodiments, the first selection unit is used to convert the weld center coordinate data into spatial position information based on the distance and angle data set when the target image is captured; wherein, the distance and angle data includes: the distance between the lens and the metal can when the target image is captured, and the angle between the lens and the forward direction of the metal can; and the labeling position is determined based on the spatial position information and the height data corresponding to the labeling area.

[0113] In some embodiments, the first selection unit is used to find the two calibration points closest to the horizontal coordinate in the weld center coordinate data by looking up a preset calibration point coordinate and calibration angle correspondence table; to use a linear interpolation algorithm on the calibration angles corresponding to the two calibration points to obtain the physical angle between the horizontal coordinate in the weld center coordinate data and the forward direction of the metal can; and to generate corresponding spatial position information based on the physical angle and the distance angle data set when the target image was captured.

[0114] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0115] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0116] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0117] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0118] S1, Obtain the target image to be labeled and positioned, wherein the target image is used to represent the body image of the metal can.

[0119] S2, the target image is processed using a multi-template matching localization model to obtain the target area of ​​the weld seam in the target image. The multi-template matching localization model is generated by training based on the preset training template image and the measured area information of the weld seam in the preset training template image.

[0120] S3. Edge extraction is performed on the target image to obtain the first metal can image. Vertical line detection is performed in the area corresponding to the weld target area in the first metal can image to obtain the weld center coordinate data corresponding to the target image, and the labeling position is determined based on the weld center coordinate data.

[0121] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0122] Furthermore, in conjunction with the labeling and positioning methods based on linemod matching and Hough transform in the above embodiments, this application embodiment can provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the labeling and positioning methods based on linemod matching and Hough transform in the above embodiments.

[0123] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A labeling positioning method based on linemod matching and Hough transform, applied to a metal can labeling production line, characterized in that, include: Obtain the target image to be labeled and positioned, wherein the target image is used to represent the body image of the metal can; The target image is processed using a multi-template matching localization model to obtain the weld target region in the target image where the weld seam appears. The multi-template matching localization model is generated by training based on a preset training template image and the measured region information of the weld seam in the preset training template image. Edge extraction is performed on the target image to obtain a first metal can image. Vertical line detection is then performed within the region of the first metal can image corresponding to the weld target area to obtain the weld center coordinate data corresponding to the target image. The labeling position is then determined based on the weld center coordinate data. Determining the labeling position based on the weld center coordinate data includes: Based on the distance and angle data set when the target image was captured, the weld center coordinate data is converted into spatial position information; wherein, the distance and angle data includes: the distance between the lens and the metal can when the target image was captured, and the angle between the lens and the forward direction of the metal can; The labeling position is determined based on the spatial location information and the height data corresponding to the labeling area.

2. The method according to claim 1, characterized in that, The target image is processed using a multi-template matching localization model to obtain the weld target region in the target image where the weld seam appears, including: The target image is blurred to obtain a second metal can image, wherein the blurring process includes Gaussian blurring. The second metal can image is processed using the Linemod multi-template matching algorithm to obtain the matching degree between each region of the target image and a preset weld template, wherein the weld template is generated by the multi-template matching localization model during training. The region with a matching degree exceeding the matching degree threshold is selected as the target region of the weld.

3. The method according to claim 2, characterized in that, The second metal can image is processed using the Linemod multi-template matching algorithm to obtain the matching degree between each region of the target image and a preset weld template, including: Using multiple weld templates, horizontal and vertical sliding window matching is performed sequentially in the second metal can image to obtain multiple similarity values. Each sliding window matching generates one similarity value. The matching degree of the target image to the weld template is obtained by superimposing all the similarity values ​​obtained based on the same weld template.

4. The method according to claim 1, characterized in that, Edge extraction is performed on the target image to obtain a first metal can image, including: The target image is converted to grayscale to obtain the third metal can image; The first metal can image is obtained by extracting edges from the third metal can image using an edge detection operator, wherein the edge detection operator includes the Canny operator.

5. The method according to claim 1, characterized in that, Within the region of the first metal can image corresponding to the target area of ​​the weld, vertical line detection is performed to obtain the weld center coordinate data corresponding to the target image, including: The target area of ​​the weld is used as a mask and ANDed with the first metal can image to obtain a fourth metal can image; Vertical line detection is performed on the image of the fourth metal can to obtain the coordinate data of the weld center corresponding to the target image, wherein the vertical line detection includes Hough transform line detection.

6. The method according to claim 1, characterized in that, Based on the distance and angle data set when the target image was captured, the weld center coordinate data corresponding to the target image is converted into spatial position information, including: By searching a preset table of calibration point coordinates and calibration angles, the two calibration points closest to the horizontal coordinate in the coordinate data of the weld center are obtained; A linear interpolation algorithm is used to obtain the physical angle between the horizontal coordinate of the weld center coordinate data and the forward direction of the metal can by applying a linear interpolation algorithm to the calibration angles corresponding to the two calibration points. Based on the physical angle and the distance and angle data set when the target image was captured, corresponding spatial location information is generated.

7. A labeling and positioning device based on Linemod matching and Hough transform, applied to a metal can labeling production line, characterized in that, include: An acquisition module is used to acquire a target image of the location to be labeled, wherein the target image is used to represent the body image of the metal can; The matching module is used to process the target image using a multi-template matching localization model to obtain the weld target region in the target image where the weld seam appears. The multi-template matching localization model is generated by training based on a preset training template image and the measured region information of the weld seam in the preset training template image. The positioning module is used to extract edges from the target image to obtain a first metal can image. Within the region of the first metal can image corresponding to the weld target area, vertical line detection is performed to obtain the weld center coordinate data corresponding to the target image. The labeling position is determined based on the weld center coordinate data. The positioning module is also used to convert the weld center coordinate data into spatial position information based on the distance and angle data set when the target image was captured. The distance and angle data includes: the distance between the lens and the metal can when the target image was captured, and the angle between the lens and the forward direction of the metal can. The labeling position is determined based on the spatial position information and the height data corresponding to the labeling area.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the labeling and positioning method based on linemod matching and Hough transform as described in any one of claims 1 to 6.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the labeling and positioning method based on linemod matching and Hough transform as described in any one of claims 1 to 6.

Images