Image multi-template matching method, storage medium and computer

Through the multi-template matching method of image, edge points are extracted and filtered and template lists are generated, which solves the problem that traditional template matching algorithms cannot match multiple graphic objects, and realizes efficient and accurate matching of multiple template images, improving industrial detection efficiency .

CN114913351BActive Publication Date: 2024-11-12DSTEK CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202210470413.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-28
Publication Date
2024-11-12
Estimated Expiration
2042-04-28

AI Technical Summary

Technical Problem

The traditional template matching algorithm can only support the matching of single template images and cannot effectively match multiple graphic objects, such as circles, triangles or squares, resulting in the need to train and load the detection model multiple times, and the operation efficiency is low.

Method used

A multi-template matching method for image is proposed. By forming multi-template model files, extracting and filtering edge points, calculating gradient directions, generating template lists, supporting training and matching of multiple template images, achieving efficient matching of any number of template images .

Benefits of technology

High accuracy and high efficiency matching of multiple template images is achieved, detection time is reduced, industrial detection efficiency is improved, and rotation, scaling and affine transformation is supported.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114913351B_ABST
    Figure CN114913351B_ABST
Patent Text Reader

Abstract

The present invention provides an image multi-template matching method, a storage medium and a computer. The image multi-template matching method comprises: forming a model file including multiple templates; processing a detection image to obtain an edge feature map of the detection image; matching each template in the model file with the edge feature map to obtain matching areas of multiple templates in the detection image. The method can simultaneously match multiple different target areas on the detection image, has high accuracy and high efficiency, and can effectively improve industrial detection efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image multi-template matching method, a storage medium and a computer. Background Art

[0002] Template matching is a commonly used algorithm in machine vision. It is mainly used to match and locate specific images in images. It can generally support the rotation, scaling, and some affine transformations of specific template images. Generally, only a single template image can be loaded. However, if we need to match more types of images in the image, such as matching circles, triangles, or squares in the positioning image, the traditional template matching algorithm cannot support it. It is necessary to train and load the detection model multiple times according to the graphic objects to be matched, and the operation efficiency is low.

[0003] Therefore, it is necessary to propose a new image template matching scheme. Summary of the invention

[0004] The object of the present invention is to provide an image multi-template matching method, a storage medium and a computer, which can simultaneously match multiple different target areas on a detection image, have high accuracy and high efficiency, and can effectively improve industrial detection efficiency.

[0005] To achieve the purpose of the invention, according to one aspect of the present invention, the present invention provides an image multi-template matching method, which includes: forming a model file including multiple templates; processing a detection image to obtain an edge feature map of the detection image; and matching each template in the model file with the edge feature map of the detection image to obtain matching areas of multiple templates in the detection image. Wherein, the formation of the model file including multiple templates includes: extracting edge points of each template image in multiple template images; screening the edge points of each template image, filtering out edge points with edge gradients weaker than a predetermined value, and obtaining the screened edge points of each template image; discretizing and extracting the screened edge points of each template image based on a predetermined sampling interval to obtain the sampled edge points of each template image; calculating the gradient direction of each sampled edge point of each template image, and dividing the gradient direction of each sampled edge point into one of multiple direction intervals, thereby obtaining the direction interval and coordinates of each sampled edge point; generating a template from the direction intervals and coordinates of all sampled edge points of each template image, and forming a template list from multiple templates of multiple template images; forming the model file based on the template list;

[0006] According to another aspect of the present invention, the present invention provides a storage medium storing program instructions, wherein the above-mentioned image multi-template matching method is executed when the program is executed.

[0007] According to yet another aspect of the present invention, the present invention provides a computer, comprising a processor and a memory, wherein the memory stores program instructions, and the processor runs the program instructions to execute the above-mentioned image multi-template matching method.

[0008] Compared with the prior art, the present invention can support the training of any number of template images, and only needs to load the detection model once after the training is completed, so as to match and locate all the template images for training in the detection image. Moreover, the detection time is not much different from that of single template matching. It has high accuracy and efficiency, and can effectively improve the efficiency of industrial detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A schematic diagram of a flow chart of an image multi-template matching method in one embodiment of the present invention;

[0010] Figure 2 A flowchart of a method for forming a model file including multiple templates in one embodiment. DETAILED DESCRIPTION

[0011] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0012] In view of the problem of low efficiency of template matching when multiple template objects need to be detected, the present invention provides an image multi-template matching method, which supports the use of multiple template images for training. On the basis of general template images supporting rotation, scaling and affine transformation matching and positioning, it can efficiently match and locate multiple template images in the detection image, and the detection speed is much better than multiple training and detection of a single template image. And it can accurately obtain the position, angle, scaling, and template category of the template matching positioning area in the image.

[0013] The present invention provides an image multi-template matching method, which can simultaneously match multiple different target areas on a detection image, has high accuracy and high efficiency, and can effectively improve industrial detection efficiency.

[0014] Figure 1 FIG. 1 is a flow chart of the image multi-template matching method 100 in one embodiment of the present invention. Figure 1 As shown, the image multi-template matching method 100 includes the following steps.

[0015] Step 110, forming a model file including multiple templates.

[0016] Before matching begins, the model file is loaded to initialize the detection model. The model file includes multiple templates, each of which is based on a template image form. In this way, multiple templates can be matched at the same time, which is fast.

[0017] Step 120: Process the detection image to obtain an edge feature map of the detection image.

[0018] The detection image is an image to be detected or matched, and there may be multiple images.

[0019] In one embodiment, step 120 specifically includes: extracting edge points in the detection image; screening the edge points of the detection image, filtering out edge points whose edge gradients are weaker than a predetermined value, and obtaining the screened edge points of the detection image; performing point diffusion on the screened edge points of the detection image to eliminate noise enhancement features, and obtaining an edge feature map of the detection image.

[0020] Step 130, matching each template in the model file with the edge feature map of the detection image to obtain matching areas of multiple templates in the detection image.

[0021] In one embodiment, step 130 includes: traversing the edge feature map of the detection image by moving the window, comparing the similarity between each template in the model file and the area in the window, and obtaining candidate areas with higher similarity; using similarity threshold screening and NMS (Non-Maximum Suppression) operation to screen the candidate areas with higher similarity and finally obtaining matching areas of multiple templates in the detection image.

[0022] One of the key points of the present invention is to use a specific traditional image processing method to perform efficient multi-template image matching. First, the edge points of multiple template images are processed, the gradient characteristics of each edge point are recorded, all edge point information of each image is stored in a data structure, and finally the edge information of each template image is saved in the form of a node in a file.

[0023] The present invention can support the training of any number of template images, and only needs to load the detection model once after the training is completed, so that all the template images for training can be matched and located in the detection image. In addition, the detection time is not much different from that of single template matching. It has high accuracy and efficiency, and can effectively improve industrial detection efficiency.

[0024] Figure 2 FIG. 1 is a flow chart of step 110 of forming a model file including multiple templates in one embodiment. Figure 2 As shown, step 110 includes the following sub-steps.

[0025] Step 111 , extracting edge points of each template image among multiple template images.

[0026] Step 112 , screening the edge points of each template image, filtering out edge points whose edge gradient is weaker than a predetermined value, and obtaining screened edge points of each template image.

[0027] Step 113, based on the predetermined sampling interval, the screening edge points of each template image are discretized and extracted to obtain the sampling edge points of each template image. The sampling edge points of each template image are evenly distributed. It is ensured that the number of sampling edge points of each template image is roughly the same, such as 128 sampling edge points for each template image. The number of sampling edge points can also be set according to the needs of the project.

[0028] Step 114 , calculating the gradient direction of each sampled edge point of each template image, and dividing the gradient direction of each sampled edge point into one of a plurality of direction intervals, thereby obtaining the direction interval and coordinates of each sampled edge point.

[0029] The value range of the gradient direction is divided into M direction intervals. According to the value of the gradient direction of each sampling edge point, it is divided into one of the multiple direction intervals, so that the direction interval and coordinates of each sampling edge point are obtained, and M is a natural number greater than 1.

[0030] For example, M=16, the gradient direction uses 22.5° as a direction interval, and the value range of the gradient direction is quantized to 16 direction intervals represented by 0 to 15. The direction interval number is used to represent the sampled edge point according to the direction interval number of the gradient direction of the edge point, that is, each sampled edge point can be represented in the form of coordinates + direction interval of the edge gradient, such as feature point (x=300, y=320, angle=5).

[0031] Step 115, generate a template from the direction intervals and coordinates of all sampled edge points of each template image, and form a template list with multiple templates of multiple template images. Each node in the template list represents a template.

[0032] Step 116: forming the model file based on the template list.

[0033] Specifically, after the template list is represented by an XML structure, the model file is generated after saving, wherein the root node is the entry of the template list composed of all templates, and the first-level child nodes of the root node are the constituent nodes of each template.

[0034] In this way, the model file may include a template formed based on a plurality of template images.

[0035] Specific Example 1

[0036] A company's workshop production site PCB component defect detection, the components on the PCB board are of various types, complex and changeable, the shooting environment in the workshop is poor, there may be stains on the lens surface, and the lighting conditions around the camera are sometimes bright and sometimes dark. The detection is required to be compatible with these interference factors.

[0037] Detection difficulties: poor shooting environment conditions, poor imaging quality, a wide variety of components, large interference factors from similar components, and it is very easy to make mistakes when using traditional template matching to find the target component.

[0038] Collect template images of multiple components, use multi-template matching algorithm to generate model files, and use image multi-template matching method to load the generated model files to process the detection images, which can detect multiple target components efficiently and accurately.

[0039] According to another aspect of the present invention, the present invention provides a storage medium storing program instructions, which are executed to execute the above-mentioned image multi-template matching method 100. For simplicity, the specific content of the image multi-template matching method 100 is not repeated here.

[0040] According to another aspect of the present invention, the present invention provides a computer, comprising a processor and a memory, wherein the memory stores program instructions, and the processor runs the program instructions to execute the above-mentioned image multi-template matching method 100. For simplicity, the specific content of the image target matching and positioning method is not repeated here.

[0041] In this document, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion of elements other than those listed and may also include additional elements not expressly listed.

[0042] In this document, the directional words such as front, back, top, and bottom are defined by the positions of the components in the drawings and the positions of the components relative to each other, and are only for the sake of clarity and convenience in expressing the technical solution. It should be understood that the use of the directional words should not limit the scope of protection claimed in this application.

[0043] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0044] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0045] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0046] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0047] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

Claims

1. A method for matching multiple image templates, characterized in that: It includes: A model file including multiple templates is formed, which includes: Extract edge points of each template image from multiple template images; The edge points of each template image are screened, and the edge points whose edge gradient is weaker than a predetermined value are filtered out to obtain the screened edge points of each template image; Discretizing and extracting the screening edge points of each template image based on a predetermined sampling interval to obtain the sampling edge points of each template image; Calculate the gradient direction of each sampled edge point of each template image, and divide the gradient direction of each sampled edge point into one of a plurality of direction intervals, thereby obtaining the direction interval and coordinates of each sampled edge point; Generate a template from the direction intervals and coordinates of all sampled edge points of each template image, and combine multiple templates of multiple template images into a template list; forming the model file based on the template list; Processing the detection image to obtain an edge feature map of the detection image; and Each template in the model file is matched with the edge feature map of the detection image to obtain matching areas of multiple templates in the detection image.

2. The image multi-template matching method according to claim 1, characterized in that: The processing of the detection image to obtain the edge feature map of the detection image comprises: Extracting edge points in the detection image; Screening the edge points of the detection image, filtering out edge points whose edge gradient is weaker than a predetermined value, and obtaining screened edge points of the detection image; Point diffusion is performed on the screened edge points of the detection image to obtain an edge feature map of the detection image.

3. The image multi-template matching method according to claim 1, characterized in that: The step of matching each template in the model file with the edge feature map of the detection image to obtain matching areas of multiple templates in the detection image includes: Traversing the edge feature map of the detection image by moving the window, comparing the similarity between each template in the model file and the area in the window, and obtaining a candidate area with a higher similarity; The similarity threshold screening and non-maximum suppression operation are used to screen the candidate areas with higher similarity, and finally the matching areas of multiple templates are obtained in the detection image.

4. The image multi-template matching method according to claim 1, characterized in that: The value range of the gradient direction is divided into M direction intervals. According to the value of the gradient direction of each sampling edge point, it is divided into one of the multiple direction intervals, so that the direction interval and coordinates of each sampling edge point are obtained, and M is a natural number greater than 1.

5. The image multi-template matching method according to claim 1, characterized in that: After the template list is represented by an XML structure, the model file is generated after saving, wherein the root node is the entry of the template list composed of all templates, and the first-level child nodes of the root node are the constituent nodes of each template. The sampling edge points of each template image are evenly distributed. Before matching, the model file is loaded to initialize the detection model.

6. A storage medium, characterized in that: It stores program instructions, and the program is executed to execute the image multi-template matching method as claimed in any one of claims 1-5.

7. A computer, characterized in that: It includes a processor and a memory, wherein the memory stores program instructions, and the processor runs the program instructions to execute the image multi-template matching method as described in any one of claims 1-5.

Citation Information

Patent Citations

  • Matching method based on gradient information

    CN111079802A

  • Template matching method based on gradient information

    CN111079803A

  • Template matching method, device and equipment and computer readable storage medium

    CN113033640A

  • Discrete edge binning template matching system, method and computer readable medium

    EP3140783A4

  • Template matching algorithm based on edge and gradient feature

    WO2021017361A1