Material image matching method and device, computer equipment and storage medium

Abstract

The invention relates to a material image matching method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring edge points of a material template image, normal vectors of the edge points, a material image of a to-be-detected material, and transformation parameters of the material template image and the material image; transforming the coordinates and normal vectors of the edge points based on the transformation parameters to obtain transformed coordinates and normal vectors of the edge points; detecting a connected domain of the material image and a centroid coordinate of the connected domain, and respectively converting the transformed coordinate of the edge point and the coordinate of the pixel point in the material image into a coordinate system taking the centroid coordinate as an original point to obtain a first coordinate of the transformed edge point and a coordinate of the transformed pixel point; and matching the material template image and the material image based on the first coordinate of the edge point, the transformed normal vector and a target edge point corresponding to the edge point in the transformed material image to obtain a matching result. The method is beneficial to improving the image matching efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a material image matching method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] Industrial automated production lines typically encompass multiple core functions such as automated material sorting, cutting, and grinding. They often require photographing and recognizing the materials to be inspected to determine their position and orientation. Template matching is a common machine vision method used in this field. Generally, template matching involves pre-creating a template identical to the material to be inspected, and then extracting feature information from both the template and the material surface for similarity matching.

[0003] Template matching schemes in related technologies mainly include the following: feature point-based matching methods, which extract key points in the image and template and calculate their local descriptors, and then align the coordinate systems of the template image and the target image by determining the transformation matrix for feature comparison; and object shape contour matching, which captures invariant features under shape transformation, such as Hu moments and shape context, and compares them directly.

[0004] However, feature-point-based matching methods rely on surface texture features, and the descriptor computation is relatively large; shape contour-based matching methods are mainly of two types: Hu moments and shape context. Hu moments are easily affected by noise and shape details, while shape context has high computational complexity and is sensitive to noise; deep learning requires a large amount of labeled data, and model training and deployment are relatively complex.

[0005] It is evident that template-based image matching methods in related technologies suffer from low matching efficiency. Summary of the Invention

[0006] Therefore, it is necessary to provide a material image matching method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve image matching efficiency in response to the above-mentioned technical problems.

[0007] In a first aspect, this application provides a material image matching method, including:

[0008] Obtain the edge points and normal vectors of the material template image, the material image of the material to be tested, and the transformation parameters between the material template image and the material image;

[0009] Based on the transformation parameters, the coordinates and normal vectors of the edge points are transformed to obtain the transformed coordinates and normal vectors of the edge points.

[0010] Detect the connected components of the material image, determine the centroid coordinates of the connected components, and transform the coordinates of the transformed edge points and the coordinates of the pixels in the material image to a coordinate system with the centroid coordinates as the origin, so as to obtain the first coordinates of the transformed edge points and the coordinates of the transformed pixels.

[0011] Based on the first coordinates of the edge point, the transformed normal vector, and the target edge point corresponding to the edge point in the transformed material image, the material template image and the material image are matched to obtain the matching result.

[0012] In one embodiment, obtaining the edge points and normal vectors of the edge points of the material template image includes:

[0013] Obtain the material template image, perform edge detection on the material template image, and obtain the edge detection map and gradient map;

[0014] Based on a preset pixel interval value, multiple candidate edge points are extracted from the edge detection map;

[0015] Based on the gradient map, determine the gradient value and normal vector of each candidate edge point;

[0016] Candidate edge points whose gradient values ​​are greater than or equal to a preset gradient threshold are identified as edge points.

[0017] In one embodiment, after obtaining the normal vectors of the edge points of the material template image, the method further includes:

[0018] Detect the connected components of the material template image, determine the centroid coordinates of the connected components, and transform the coordinates of the edge points to a coordinate system with the centroid coordinates as the origin, thus obtaining the second coordinates of the transformed edge points;

[0019] If an edge point is detected where the normal vector does not point to a connected component, the normal vector of the edge point to be corrected is corrected to update the normal vector of the edge point.

[0020] Based on the transformation parameters, the coordinates and normal vectors of the edge points are transformed to obtain the transformed coordinates and normal vectors of the edge points, including:

[0021] Based on the transformation parameters, the second coordinates and the updated normal vector of the edge point are transformed to obtain the transformed coordinates and normal vector of the edge point.

[0022] In one embodiment, when an edge point to be corrected is detected whose normal vector does not point to a connected component, the normal vector of the edge point to be corrected is corrected, including:

[0023] The target pixel in the material template image is extracted along the direction of the normal vector of the edge point. If the target pixel is a pixel outside the connected component, it is determined that the normal vector of the edge point does not point to the connected component, and the edge point is identified as the edge point to be corrected.

[0024] Invert the normal vector of the edge point to be corrected to obtain the corrected normal vector.

[0025] In one embodiment, based on the first coordinates of the edge point and the transformed normal vector, and the target edge point in the transformed material image corresponding to the edge point, the material template image and the material image are matched to obtain a matching result, including:

[0026] For each target edge point, the first pixel is extracted from the material image along the positive direction of the normal vector corresponding to the target edge point, and the second pixel is extracted from the material image along the negative direction of the normal vector corresponding to the target edge point.

[0027] Determine the difference between the pixel values ​​of the first pixel and the second pixel. If the difference is greater than a preset difference threshold, determine that the target edge point matches the edge point.

[0028] The similarity between the material template image and the material image is determined based on the proportion of the number of target edge points that match the edge points to the total number of target edge points.

[0029] The matching results include similarity scores.

[0030] In one embodiment, before extracting edge points from the material template image, the method further includes:

[0031] The material template image is preprocessed, including at least one of binarization and image denoising.

[0032] Secondly, this application also provides a material image matching device, comprising:

[0033] The data acquisition module is used to acquire the edge points and normal vectors of the material template image, the material image of the material to be tested, and the transformation parameters between the material template image and the material image;

[0034] The data transformation module is used to transform the coordinates and normal vectors of edge points based on transformation parameters, so as to obtain the transformed coordinates and normal vectors of the edge points;

[0035] The coordinate transformation module is used to detect the connected components of the material image, determine the centroid coordinates of the connected components, and transform the coordinates of the transformed edge points and the coordinates of the pixels in the material image to a coordinate system with the centroid coordinates as the origin, so as to obtain the first coordinates of the transformed edge points and the coordinates of the transformed pixels.

[0036] The template matching module is used to match the material template image and the material image based on the first coordinates of the edge point, the transformed normal vector, and the target edge point in the transformed material image corresponding to the edge point, to obtain the matching result.

[0037] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above-described material image matching method embodiments.

[0038] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the above-described material image matching method embodiments.

[0039] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in any of the above-described material image matching method embodiments.

[0040] The aforementioned material image matching method, apparatus, computer equipment, computer-readable storage medium, and computer program product first pre-acquire the edge points and normal vectors of the material template image as feature information of the material template image. Then, they acquire the material image of the material to be tested and the transformation parameters between the material template image and the material image. Subsequently, based on the transformation parameters, they transform the coordinates and normal vectors of the edge points of the material template image to obtain the transformed coordinates and normal vectors of the edge points. This eliminates the need to transform the entire material template image, improving the efficiency of geometric transformation. Matching the material template image and the material image based on the transformed coordinates and normal vectors of the edge points helps reduce the possibility of mismatches caused by factors such as viewpoint and scale, thus improving matching accuracy. Simultaneously, it helps achieve geometric transformation invariance during the matching process and improves the handling of complex material images. Similar to the matching fitness of the matching scene, the connected components of the material image are detected, the centroid coordinates of the connected components are determined, and the coordinates of the transformed edge points and the coordinates of the pixels in the material image are transformed into a coordinate system with the centroid coordinates as the origin. This yields the first coordinates of the transformed edge points and the coordinates of the transformed pixels, which helps to quickly locate the target edge points in the material image corresponding to the edge points, further reducing the possibility of mismatches and improving matching accuracy. Finally, based on the first coordinates of the edge points, the transformed normal vectors, and the target edge points in the transformed material image corresponding to the edge points, the material template image and the material image are matched to obtain the matching results. In this way, the matching operation of the entire image is transformed into matching between a small number of pixels using the coordinates of the edge points and the normal vectors, which greatly reduces the amount of computation, improves the template matching efficiency, and further helps to improve the material processing efficiency. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is an application environment diagram of the material image matching method in one embodiment;

[0043] Figure 2 This is a flowchart illustrating a material image matching method in one embodiment;

[0044] Figure 3 This is a flowchart illustrating the material image matching method in another embodiment;

[0045] Figure 4This is a flowchart illustrating the material image matching method in yet another embodiment;

[0046] Figure 5 This is a schematic diagram of the normal vectors of the edge points of a material template image in one embodiment.

[0047] Figure 6 This is a flowchart illustrating the material image matching method in another embodiment;

[0048] Figure 7 This is a schematic diagram of a material image, a material template image, and a matching result in one embodiment;

[0049] Figure 8 This is a structural block diagram of a material image matching device in one embodiment;

[0050] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] The material image matching method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the image acquisition terminal 102 communicates with the server 104 via a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated onto the server 104, or it can be located in the cloud or on another network server.

[0053] Specifically, the image acquisition terminal 102 can upload the acquired material image data of the material to be tested to the server 104. The server 104 can acquire the material image of the material to be tested and obtain the edge points and normal vectors of the material template image, as well as the transformation parameters between the material template image and the material image from the data storage system. Next, based on the transformation parameters, the coordinates and normal vectors of the edge points are transformed to obtain the transformed coordinates and normal vectors of the edge points. Then, the connected components of the material image are detected, the centroid coordinates of the connected components are determined, and the coordinates of the transformed edge points and the coordinates of the pixels in the material image are transformed to a coordinate system with the centroid coordinates as the origin to obtain the first coordinates of the transformed edge points and the coordinates of the transformed pixels. Finally, based on the first coordinates of the edge points, the transformed normal vectors, and the target edge points in the transformed material image corresponding to the edge points, the material template image and the material image are matched to obtain the matching result.

[0054] Among them, server 104 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0055] In one exemplary embodiment, such as Figure 2 As shown, a material image matching method is provided, which is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps (hereinafter referred to as S): S100 to S400. Wherein:

[0056] S100, obtain the edge points and normal vectors of the material template image, the material image of the material to be tested, and the transformation parameters between the material template image and the material image.

[0057] The materials to be tested can include, but are not limited to, metals, plastics, electronic components, and cardboard boxes, depending on the actual product requirements. A material image is an image containing the material to be tested, while a material template image is an image containing standard materials. This template image can be created by the operator pre-collecting images of different types of materials to be identified based on material matching requirements, thus obtaining a material template image containing standard materials.

[0058] The transformation parameters between the material template image and the material image can include rotation angle and scaling ratio, which are used to adjust the rotation angle and scaling ratio of the material template image to match those of the material image. The transformation parameters can be derived from the camera pose parameters of the material template image and the material image acquired by the image acquisition device. Alternatively, with the pose of the image acquisition device used to acquire the material to be tested unchanged, a material image is taken, and feature points are identified in both the material image and the material template image based on feature extraction algorithms (such as SIFT (Scale Invariant Feature Transform) and SURF (Speeded Robust Feature Transform)). Feature points are then matched using Fast Approximate Nearest Neighbor (FLANN) search, and the transformation parameters are calculated based on the matched feature point pairs. These transformation parameters are then stored in a database.

[0059] Among them, the normal vector of the edge point can be a vector that represents the direction of the fastest change in gray value at the edge point, that is, the direction of the edge normal.

[0060] In practical applications, the operator uploads a material template image to the server. The server binarizes the image and then performs edge detection using an edge detection algorithm to obtain an edge detection map. Edge points are then extracted from this map. Next, a gradient vector detection algorithm is used to determine the gradient vectors of the edges in the material template image. The edge detection algorithms can include, but are not limited to, the Laplacian edge detection algorithm, the Robert edge detection operator, the Sobel edge detection operator, the Prewitt edge detection operator, and the Canny edge detection operator. Gradient vector detection algorithms can include, but are not limited to, the Sobel operator and the Scharr operator. The edge points and normal vectors of the material template image constitute its feature information. Therefore, after obtaining the edge points and normal vectors, the server can establish a correlation between these parameters and the material template image and store them in a database.

[0061] In practice, multiple image acquisition devices can be deployed in the factory. The image acquisition devices acquire material images of the material to be tested. Then, the material images are transmitted to the server via the network. The server obtains the material images and retrieves the edge points and normal vectors of the material template image from the database, as well as the transformation parameters between the material template image and the material image, for subsequent material image matching.

[0062] For example, taking the application scenario of sorting vehicle parts (such as bolts, connectors, etc.) as an example, an image acquisition device deployed next to the material conveying mechanism acquires material images of the material to be tested and transmits the material images to the server. The server obtains the material images and retrieves the edge points and edge point normal vectors of the material template image from the database, as well as the transformation parameters between the material template image and the material image.

[0063] S200, based on the transformation parameters, transform the coordinates of the edge points and the normal vector to obtain the transformed coordinates and normal vectors of the edge points.

[0064] In practical implementation, following the example above, the server can rotate the coordinates and normal vectors of the edge points based on the rotation angle in the transformation parameters, and scale the coordinates and normal vectors of the edge points based on the scaling ratio in the transformation parameters to obtain the transformed coordinates and normal vectors of the edge points, which helps to perform template matching for material images with different rotation angles and scaling relationships.

[0065] S300: Detect the connected components of the material image, determine the centroid coordinates of the connected components, and transform the coordinates of the transformed edge points and the coordinates of the pixels in the material image to a coordinate system with the centroid coordinates as the origin, thereby obtaining the first coordinates of the transformed edge points and the coordinates of the transformed pixels.

[0066] A connected region is a region connected by adjacent pixels under a specific pixel similarity criterion. The centroid of a connected region is the average position of the pixels in that region, used to characterize the center of the connected region.

[0067] In practice, the server can binarize the material image, detect connected components of the material image using the OpenCV library, and determine the centroid coordinates of these components. Then, for the material image, a translation transformation is performed on the coordinates of each pixel, centered on the centroid coordinates, to convert the pixel coordinates to a coordinate system with the centroid coordinates as the origin, resulting in the converted pixel coordinates. Similarly, for the converted edge point coordinates, a translation transformation is performed on the edge point coordinates, centered on the centroid coordinates, to convert the edge point coordinates to a coordinate system with the centroid coordinates as the origin, resulting in the first coordinates of the converted edge points. This completes the normalization process for the coordinates of the edge points and the pixel coordinates of the material image, aligning the material to be tested with the standard material in the same coordinate system.

[0068] S400, based on the first coordinates of the edge point and the transformed normal vector, as well as the target edge point in the transformed material image corresponding to the edge point, matches the material template image and the material image to obtain the matching result.

[0069] The matching results can include whether the material image matches the material template image, and the type of standard material in the material template image.

[0070] In practical implementation, since the above steps have transformed the edge points and the pixels of the material image into the same coordinate system, the first coordinate of the edge point is consistent with the coordinate of the target edge point. Therefore, the server can first determine the target edge point in the material image with the same first coordinate as the edge point based on the first coordinate of the edge point. Subsequently, based on the first coordinate of the edge point, the transformed normal vector, and the target edge point in the transformed material image that matches the edge point, the material template image and the material image are matched. This can be done by extracting the pixel values ​​of a preset number of pixels from the material image along the opposite direction of the normal vector of the edge that matches the target edge point for each target edge point. Based on the extracted pixel values, it is determined whether the target edge point matches the corresponding edge point. The number of matching target edge points and edge points is counted, and the matching of the material image and the material template image is determined based on the number. The preset number is determined according to the detection requirements, such as 1, 5, or 10, and this application does not limit this.

[0071] For example, for each target edge point, the pixel values ​​of 10 pixels are extracted from the material image along the opposite direction of the normal vector of the edge that matches the target edge point. The extracted 10 pixel values ​​are compared with a preset pixel value threshold. A first number of pixel values ​​not higher than the preset pixel value threshold is counted. If the proportion of the first number to the preset number is greater than a preset first proportion threshold, the target edge point is determined to match the corresponding edge point. Then, the mutually matching target edge points and edge points are taken as a pair of matching points. The number of matching point pairs is counted, and the proportion of the number of matching point pairs to the total number of pairs is calculated. If the proportion of the number of matching point pairs to the total number of pairs is greater than a preset second proportion threshold, the material image is determined to match the material template image. The type of standard material in the material template image is determined as the type of the material to be tested in the material image, and the matching result is obtained.

[0072] In other embodiments, the material template image and the material image are matched based on the first coordinates of the edge point, the transformed normal vector, and the target edge point in the transformed material image that matches the edge point. Alternatively, for each target edge point, a preset number of pixel values ​​are extracted from the material image along the positive direction of the normal vector of the edge that matches the target edge point. Based on the extracted pixel values, it is determined whether the target edge point matches the corresponding edge point. The number of matching target edge points and edge points is counted, and the material image and the material template image are determined based on the number. Specifically, for each target edge point, along the positive direction of the normal vector of the edge that matches the target edge point, a preset number (e.g., 10) of pixel values ​​are extracted from the material image. The extracted pixel values ​​are compared with a preset pixel value threshold. A second number of pixel values ​​higher than the preset pixel value threshold is counted. If the proportion of the second number to the preset number is greater than a preset first proportion threshold, the target edge point is determined to match the corresponding edge point. Then, the mutually matching target edge points and edge points are taken as a pair of matching points. The number of matching point pairs is counted, and the proportion of the number of matching point pairs to the total number of pairs is calculated. If the proportion of the number of matching point pairs to the total number of pairs is greater than a preset second proportion threshold, the material image is determined to match the material template image. The type of standard material in the material template image is determined as the type of the material to be tested in the material image, and the matching result is obtained.

[0073] In the aforementioned material image matching method, firstly, the edge points and normal vectors of the material template image are pre-acquired as feature information of the material template image. Then, the material image of the material to be tested and the transformation parameters between the material template image and the material image are obtained. Subsequently, based on the transformation parameters, the coordinates and normal vectors of the edge points of the material template image are transformed to obtain the transformed coordinates and normal vectors of the edge points. This eliminates the need to transform the entire material template image, improving the efficiency of geometric transformation. Therefore, matching the material template image and the material image based on the transformed edge point coordinates and normal vectors helps reduce the possibility of mismatches caused by factors such as viewpoint and scale, thus improving matching accuracy. Simultaneously, it helps achieve geometric transformation invariance during the matching process and improves the matching adaptability to complex material image matching scenarios. Next, the connected components of the material image are detected, and the centroid coordinates of the connected components are determined. The coordinates of the transformed edge points and the coordinates of the pixels in the material image are then transformed into a coordinate system with the centroid coordinates as the origin. This yields the first coordinates of the transformed edge points and the coordinates of the transformed pixels, which helps to quickly locate the target edge points in the material image corresponding to the edge points. This further reduces the possibility of mismatches and improves matching accuracy. Finally, based on the first coordinates of the edge points, the transformed normal vectors, and the target edge points in the transformed material image corresponding to the edge points, the material template image and the material image are matched to obtain the matching results. In this way, the matching operation of the entire image is transformed into matching between a small number of pixels using the edge point coordinates and normal vectors, which greatly reduces the amount of computation, improves the template matching efficiency, and further helps to improve the material processing efficiency.

[0074] In one exemplary embodiment, such as Figure 3 As shown, the edge points and normal vectors of the material template image are obtained based on the following steps S120 to S180:

[0075] S120: Obtain the material template image, perform edge detection on the material template image, and obtain the edge detection map and gradient map.

[0076] In practice, the method for obtaining the material template image can refer to the method for obtaining the material template image in the above embodiments, and will not be repeated here.

[0077] In practice, extracting edge points from the material template image can be achieved by the server detecting edge features in the material template image using the Canny edge detection operator to obtain an edge detection map, and then extracting edge points from the edge detection map. The normal vector of each edge point can be determined by the server calculating the gradient of the material template image in the x and y directions using the Sobel operator, obtaining the gradient vector of the edge point, which is the normal vector of the edge point. In the edge detection map, white pixels represent detected candidate edge points, and black pixels represent non-candidate edge points.

[0078] In other implementations, edge detection of the material template image can also be performed using deep learning to obtain edge points. Specifically, an initial edge detection model can be pre-built based on a convolutional neural network. Multiple material template images containing standard materials are acquired, and the material template images are labeled (the coordinates and labels of edge points are marked as edge points). The initial edge detection model is iteratively trained using the labeled material template images to determine the model's loss function value. The model parameters are then adjusted based on the loss function value until a preset training termination condition is met, resulting in a trained edge detection model. The preset training termination condition can be that the loss function value is less than a preset loss threshold for a preset number of consecutive times.

[0079] S140, based on a preset pixel interval value, extracts multiple candidate edge points from the edge detection map.

[0080] The pixel interval value represents the number of pixels between the candidate edge points extracted in two adjacent extraction operations.

[0081] In practical applications, to further improve template matching efficiency, pixel interval values ​​are set in advance according to matching requirements, such as 5 pixels. This is just an example and does not represent the only limitation.

[0082] In practice, the server traverses the edge detection image along the x-axis and y-axis. During the traversal, it determines whether the number of steps has reached a preset pixel interval value. If the preset interval value is reached, it checks whether the current pixel is a candidate edge point. If it is, the candidate edge point is extracted; otherwise, the traversal continues. This sparse sampling method obtains multiple candidate edge points from the edge detection image, reducing the number of edge points while maintaining certain edge features.

[0083] S160, based on the gradient map, determines the gradient value and normal vector of each candidate edge point.

[0084] In practice, the server can obtain the gradient value of each candidate edge point based on the gradient value of each candidate edge point in the two gradient maps, according to the pixel gradient value calculation method of the Sobel operator. Then, for each candidate edge point, the gradient vector of the candidate edge point is obtained based on the gradient value of the candidate edge point, which is the normal vector.

[0085] S180, candidate edge points whose gradient values ​​are greater than or equal to a preset gradient threshold are determined as edge points.

[0086] The gradient threshold is set according to the matching accuracy and is used to filter out the true edge points.

[0087] In practice, the server compares the gradient value of each candidate edge point with a preset gradient threshold. Candidate edge points with gradient values ​​greater than or equal to the preset gradient threshold are identified as edge points and retained, while candidate edge points with gradient values ​​less than the preset gradient threshold are removed.

[0088] In this embodiment, sparse sampling of candidate edge points in the edge detection image using a preset pixel interval value is beneficial to improving template matching efficiency, and filtering edge points based on gradient threshold is beneficial to improving template matching accuracy.

[0089] In one exemplary embodiment, such as Figure 4 As shown, after obtaining the normal vectors of the edge points of the material template image, the method further includes steps S192 to S194:

[0090] S192, detect the connected components of the material template image, determine the centroid coordinates of the connected components, and transform the coordinates of the edge points to a coordinate system with the centroid coordinates as the origin, to obtain the second coordinates of the transformed edge points.

[0091] In practice, the server can perform binarization on the material template image, detect the connected components of the material template image using the OpenCV library, and determine the centroid coordinates of the connected components. Then, with the centroid coordinates as the center, the coordinates of the edge points are translated and transformed into a coordinate system with the centroid coordinates as the origin, thus obtaining the second coordinates of the transformed edge points.

[0092] S194, if an edge point to be corrected is detected where the normal vector does not point to a connected region, the normal vector of the edge point to be corrected is corrected to update the normal vector of the edge point.

[0093] In practice, for each edge point, one or more pixels can be extracted from the material template image along the positive direction of the edge point's normal vector. Based on the position information of the detected connected components, it can be determined whether all the extracted pixels belong to the connected components. If all the extracted pixels belong to the connected components, it is determined that the normal vector of the edge point points to the connected components of the material template image. Otherwise, it is determined that the normal vector of the edge point does not point to the connected components, and the edge point is the edge point of the normal vector to be corrected.

[0094] Subsequently, the normal vector of the edge point to be corrected is corrected. This can be done by determining the vector from the edge point to the centroid coordinates based on the coordinates of the edge point to be corrected and the centroid coordinates of the connected region of the material template image, and then using this vector to update the normal vector of the edge point.

[0095] In an exemplary embodiment, the coordinates and normal vectors of the edge points are transformed based on transformation parameters to obtain the transformed coordinates and normal vectors of the edge points, including S220:

[0096] S220, based on the transformation parameters, transforms the second coordinates and the updated normal vector of the edge point to obtain the transformed coordinates and normal vector of the edge point.

[0097] In specific implementation, the second coordinates and the updated normal vector of the edge point are transformed based on the transformation parameters to obtain the transformed coordinates and normal vector of the edge point. The implementation method of transforming the coordinates and normal vector of the edge point based on the transformation parameters in the above embodiment to obtain the transformed coordinates and normal vector of the edge point will not be repeated here.

[0098] In this embodiment, by transforming the coordinates of the edge points, performing normal vector correction detection, and correcting the normal vector, it is beneficial to improve the efficiency of template matching and the adaptability to matching different material images.

[0099] In an exemplary embodiment, when an edge point to be corrected is detected whose normal vector does not point to a connected component, the normal vector of the edge point to be corrected is corrected, including:

[0100] The target pixel in the material template image is extracted along the direction of the normal vector of the edge point. If the target pixel is a pixel outside the connected component, it is determined that the normal vector of the edge point does not point to the connected component, and the edge point is identified as the edge point to be corrected.

[0101] In practice, for each edge point, one or more pixels can be extracted from the material template image along the positive direction of the edge point's normal vector. Based on the position information of the detected connected components, it can be determined whether all the extracted pixels are outside the connected components. If all the extracted pixels are outside the connected components, it is determined that the normal vector of the edge point does not point to the connected components of the material template image, and the edge point is the edge point of the normal vector to be corrected. Otherwise, it is determined that the normal vector of the edge point points to the connected components.

[0102] Invert the normal vector of the edge point to be corrected to obtain the corrected normal vector.

[0103] In practice, the server can take the negative values ​​of each component of the normal vector of the edge point to be corrected to obtain the corrected normal vector. For example, as shown below... Figure 5 As shown, Figure 5-1 This is a schematic diagram of the normal vectors of the edge points before normal vector correction. The arrows in the diagram represent the positive direction of the normal vectors. Figure 5-2 A schematic diagram representing the normal vectors of edge points after normal vector correction.

[0104] In this embodiment, normal vector correction detection and normal vector correction help improve the accuracy and efficiency of template matching.

[0105] In one exemplary embodiment, such as Figure 6 As shown, based on the first coordinates of the edge point, the transformed normal vector, and the target edge point in the transformed material image corresponding to the edge point, the material template image and the material image are matched to obtain the matching result, including S420 to S460:

[0106] S420: For each target edge point, extract the first pixel from the material image along the positive direction of the normal vector corresponding to the target edge point, and extract the second pixel from the material image along the negative direction of the normal vector corresponding to the target edge point.

[0107] The normal vector corresponding to the target edge point is the normal vector of the edge point corresponding to the target edge point.

[0108] In a specific implementation, for each target edge point, a predetermined number of first pixels can be taken from the material image along the positive direction of the normal vector corresponding to the target edge point. These first pixels can represent the pixels inside the region enclosed by the target edge point in the material image. Then, a predetermined number of second pixels can be taken from the material image along the negative direction of the normal vector corresponding to the target edge point. These second pixels can represent the pixels outside the region enclosed by the target edge point. The predetermined number can be one or more, and this application does not limit this number.

[0109] S440, determine the difference between the pixel values ​​of the first pixel and the second pixel, and if the difference is greater than a preset difference threshold, determine that the target edge point matches the edge point.

[0110] In practical applications, the preset difference threshold is set according to the matching requirements.

[0111] In practice, for each target edge point, with a preset number of one, the difference between the pixel value of the first pixel point and the pixel value of the second pixel point is calculated. The absolute value of the difference is taken and compared with a preset difference threshold. If the absolute value of the difference is greater than the preset difference threshold, it indicates that the first pixel point and the second pixel point meet the edge feature, that is, the pixel value of the first pixel point and the pixel value of the second pixel point have the feature of changing from "bright" to "dark", thus determining that the target edge point matches the edge point.

[0112] S460, based on the proportion of the number of target edge points that match the edge points to the total number of target edge points, determine the similarity between the material template image and the material image, and the matching result includes the similarity.

[0113] In practice, the server counts the number of target edge points with matching edge points, as well as the total number of target edge points, and determines the proportion of the target edge points to the total number. This proportion is the similarity between the material template image and the material image.

[0114] In this embodiment, pixel points are extracted from the material image based on the edge point coordinates and normal vectors for matching, thereby transforming the matching operation of the entire image into pixel-value-based matching, which greatly reduces the amount of computation and improves the template matching efficiency.

[0115] In one exemplary embodiment, before extracting edge points from the material template image, the method further includes:

[0116] The material template image is preprocessed, including at least one of binarization and image denoising.

[0117] Image denoising includes, but is not limited to, methods such as neighborhood averaging, median filtering, or bilateral filtering (border-preserving filtering) to eliminate noise in images. Binarization processing includes, but is not limited to, bimodal methods, P-parameter methods, iterative methods, and maximum inter-class variance methods.

[0118] In practice, to improve the efficiency and accuracy of material template image processing, the material template image can be binarized based on the maximum inter-class variance and the image can be denoised based on the median filtering method.

[0119] In other embodiments, extracting edge points from the material template image includes: extracting edge points from the preprocessed material template image. The method for extracting edge points here refers to the method for extracting edge points from the material template image described in the above embodiments, and will not be repeated here.

[0120] In this embodiment, binarization and image denoising preprocessing help improve the accuracy of edge point extraction, thereby improving the accuracy of template matching.

[0121] To provide a clearer explanation of the material image matching method provided in this application, a specific embodiment is described below, which includes the following steps:

[0122] S1. Obtain the material template image, preprocess the material template image, including at least one of binarization and image denoising, perform edge detection on the material template image, and obtain the edge detection map and gradient map.

[0123] S2, based on the preset pixel interval value, extract multiple candidate edge points from the edge detection map, and based on the gradient map, determine the gradient value and normal vector of each candidate edge point, and determine the candidate edge points whose gradient value is greater than or equal to the preset gradient threshold as edge points.

[0124] S3, detect the connected components of the material template image, determine the centroid coordinates of the connected components, and transform the coordinates of the edge points to a coordinate system with the centroid coordinates as the origin, to obtain the second coordinates of the transformed edge points.

[0125] S4. Extract the target pixel in the material template image along the direction of the normal vector of the edge point. If the target pixel is a pixel outside the connected component, it is determined that the normal vector of the edge point does not point to the connected component. The edge point is determined as the edge point to be corrected. The normal vector of the edge point to be corrected is inverted to obtain the corrected normal vector, so as to update the normal vector of the edge point.

[0126] S5, acquire the material image of the material to be tested, and the transformation parameters between the material template image and the material image.

[0127] S6, based on the transformation parameters, transforms the second coordinates and the updated normal vector of the edge point to obtain the transformed coordinates and normal vector of the edge point.

[0128] S7, detect the connected components of the material image, determine the centroid coordinates of the connected components, and transform the coordinates of the transformed edge points and the coordinates of the pixels in the material image to a coordinate system with the centroid coordinates as the origin, so as to obtain the first coordinates of the transformed edge points and the coordinates of the transformed pixels.

[0129] S8. For each target edge point, extract the first pixel from the material image along the positive direction of the normal vector corresponding to the target edge point, and extract the second pixel from the material image along the negative direction of the normal vector corresponding to the target edge point. Determine the difference between the pixel values ​​of the first pixel and the second pixel. If the difference is greater than a preset difference threshold, determine that the target edge point matches the edge point. Based on the proportion of the number of target edge points that match the edge point to the total number of target edge points, determine the similarity between the material template image and the material image. The matching result includes the similarity.

[0130] In one example, the comparison results of the material image matching effect using the above material image matching method with the template matching effect of related technologies are as follows:

[0131] The table below shows the test time for matching a single frame of a material image with rotation and scaling with a material template image using the above-mentioned material image matching method, the OpenCV-based grayscale value matching method, and ORB matching (an image matching technique combining corner detection and feature descriptor generation).

[0132]

[0133] The above-described material image matching method can achieve more accurate matching for materials with fine edges compared to shape contour matching methods. For example... Figure 7 As shown, Figure 7-1 Images of the materials used in the test. Figure 7-2 For material template images, Figure 7-3 The recognition result is the material image matching method described above.

[0134] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0135] In one exemplary embodiment, such as Figure 8 As shown, a material image matching device 600 is provided, including: a data acquisition module 610, a data transformation module 620, a coordinate transformation module 630, and a template matching module 640, wherein:

[0136] The data acquisition module 610 is used to acquire the edge points of the material template image and the normal vector of the edge points, the material image of the material to be tested, and the transformation parameters between the material template image and the material image;

[0137] The data transformation module 620 is used to transform the coordinates and normal vector of the edge point based on the transformation parameters to obtain the transformed coordinates and normal vector of the edge point;

[0138] The coordinate transformation module 630 is used to detect the connected components of the material image, determine the centroid coordinates of the connected components, and transform the coordinates of the transformed edge points and the coordinates of the pixels in the material image to a coordinate system with the centroid coordinates as the origin, so as to obtain the first coordinates of the transformed edge points and the coordinates of the transformed pixels.

[0139] The template matching module 640 is used to match the material template image and the material image based on the first coordinates of the edge point and the transformed normal vector, as well as the target edge point in the transformed material image corresponding to the edge point, to obtain a matching result.

[0140] In an exemplary embodiment, the material image matching device 600 further includes a feature extraction module 650, wherein:

[0141] The data acquisition module 610 is also used to acquire material template images;

[0142] The feature extraction module 650 is used to perform edge detection on the material template image to obtain an edge detection map and a gradient map; based on a preset pixel interval value, it extracts multiple candidate edge points from the edge detection map; based on the gradient map, it determines the gradient value and normal vector of each candidate edge point; and it determines the candidate edge points whose gradient values ​​are greater than or equal to a preset gradient threshold as edge points.

[0143] In an exemplary embodiment, the material image matching device 600 further includes a normal vector correction module 660, wherein:

[0144] The coordinate transformation module 630 is also used to detect the connected components of the material template image, determine the centroid coordinates of the connected components, and transform the coordinates of the edge points to a coordinate system with the centroid coordinates as the origin, so as to obtain the transformed second coordinates of the edge points;

[0145] The normal vector correction module 660 is used to correct the normal vector of the edge point to be corrected when it is detected that there is an edge point to be corrected whose normal vector does not point to the connected domain, so as to update the normal vector of the edge point.

[0146] The data transformation module 620 is also used to transform the second coordinates and the updated normal vector of the edge point based on the transformation parameters, so as to obtain the transformed coordinates and normal vector of the edge point.

[0147] In an exemplary embodiment, the normal vector correction module 660 is further configured to extract the target pixel in the material template image along the direction of the normal vector of the edge point; if the target pixel is a pixel outside the connected component, it is determined that the normal vector of the edge point does not point to the connected component, and the edge point is determined as the edge point to be corrected; the normal vector of the edge point to be corrected is inverted to obtain the corrected normal vector.

[0148] In an exemplary embodiment, the template matching module 640 is further configured to, for each target edge point, extract a first pixel from the material image along the positive direction of the normal vector corresponding to the target edge point, and extract a second pixel from the material image along the negative direction of the normal vector corresponding to the target edge point; determine the difference between the pixel values ​​of the first pixel and the second pixel; if the difference is greater than a preset difference threshold, determine that the target edge point matches the edge point; determine the similarity between the material template image and the material image based on the proportion of the number of target edge points that match the edge point to the total number of target edge points; the matching result includes the similarity.

[0149] In an exemplary embodiment, the material image matching device 600 further includes an image preprocessing module 670, which is further configured to preprocess the material template image, the preprocessing including at least one of binarization and image denoising.

[0150] Each module in the aforementioned material image matching device 600 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0151] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a material image matching method.

[0152] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0153] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above-described material image matching method embodiments.

[0154] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the above-described material image matching method embodiments.

[0155] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in any of the above-described material image matching method embodiments.

[0156] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0157] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0158] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0159] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A material image matching method characterized by, The method comprises: obtaining edge points of a material template image and normal vectors of the edge points, a material image of a material to be detected, and transformation parameters between the material template image and the material image; transforming coordinates of the edge points and the normal vectors based on the transformation parameters to obtain transformed coordinates and normal vectors of the edge points; detecting a connected domain of the material image, determining a centroid coordinate of the connected domain, and converting the transformed coordinates of the edge points and coordinates of pixel points in the material image to a coordinate system with the centroid coordinate as an origin to obtain first coordinates of the transformed edge points and coordinates of the converted pixel points; matching the material template image and the material image based on the first coordinates of the edge points and the transformed normal vectors and target edge points corresponding to the edge points in the converted material image to obtain a matching result.

2. The method of claim 1, wherein, The method comprises: obtaining a material template image, performing edge detection on the material template image to obtain an edge detection image and a gradient image; extracting a plurality of candidate edge points from the edge detection image based on a preset pixel interval value; determining gradient values and normal vectors of the candidate edge points based on the gradient image; determining the edge points as candidate edge points with gradient values greater than or equal to a preset gradient threshold value.

3. The method of claim 2, wherein, After obtaining the normal vectors of the edge points of the material template image, the method further comprises: detecting a connected domain of the material template image, determining a centroid coordinate of the connected domain, and converting the coordinates of the edge points to a coordinate system with the centroid coordinate as an origin to obtain second coordinates of the converted edge points; in a case where it is detected that there are edge points to be corrected in which the normal vectors do not point to the connected domain, correcting the normal vectors of the edge points to be corrected to update the normal vectors of the edge points; The method comprises: transforming the second coordinates of the edge points and the updated normal vectors based on the transformation parameters to obtain transformed coordinates and normal vectors of the edge points.

4. The method of claim 3, wherein, The method comprises: extracting target pixel points in the material template image in the direction of the normal vector of the edge point, and if the target pixel points are pixel points outside the connected domain, determining that the normal vector of the edge point does not point to the connected domain, and determining the edge point as an edge point to be corrected; inverting the normal vector of the edge point to be corrected to obtain a corrected normal vector.

5. The method according to any one of claims 1 to 4, characterized in that, The method comprises: For each target edge point, a first pixel point is extracted from the material image in a positive direction of a normal vector corresponding to the target edge point, and a second pixel point is extracted from the material image in a reverse direction of the normal vector corresponding to the target edge point; A difference between pixel values of the first pixel point and the second pixel point is determined, and in a case where the difference is greater than a preset difference threshold, it is determined that the target edge point matches the edge point; Based on a proportion of a number of target edge points matching edge points to a total number of the target edge points, a similarity between the material template image and the material image is determined. The matching result includes the similarity.

6. The method of claim 5, wherein, Before the edge points in the material template image are extracted, the method further includes: The material template image is preprocessed, and the preprocessing includes at least one of binarization and image denoising.

7. A material image matching apparatus characterized by comprising: The apparatus includes: A data acquisition module is configured to acquire edge points of a material template image and normal vectors of the edge points, a material image of a material to be tested, and a transformation parameter between the material template image and the material image; A data transformation module is configured to transform coordinates of the edge points and the normal vectors based on the transformation parameter to obtain transformed coordinates of the edge points and the normal vectors; A coordinate conversion module is configured to detect a connected domain of the material image, determine a centroid coordinate of the connected domain, and respectively convert the transformed coordinates of the edge points and coordinates of pixel points in the material image to a coordinate system with the centroid coordinate as an origin to obtain first coordinates of the converted edge points and coordinates of the converted pixel points; A template matching module is configured to match the material template image and the material image based on the first coordinates of the edge points and the transformed normal vectors and target edge points corresponding to the edge points in the converted material image to obtain a matching result. 8.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-7. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

Images