Image alignment method, device, storage medium and electronic device

Abstract

The present application discloses an image alignment method, device, storage medium and electronic device. The method comprises: determining the profile contour information of a target image and a template image, wherein the profile contour information comprises: template contour information and target contour information; filtering the profile contour information according to a preset filtering method to obtain the filtered profile contour information; matching the filtered profile contour information according to a preset matching method to obtain target matching information, wherein the preset matching method comprises at least one of the following: a first matching method and a second matching method, and the target matching information comprises at least one of the following: initial matching information and final matching information, wherein the matching accuracy of the first matching method is lower than the matching accuracy of the second matching method; mapping the target image to the template image based on the target matching information. Through the present application, the problem of low efficiency of image alignment in the related art is solved.

Description

Background Art

[0002] In the related technology, there are two main types of automatic image alignment technology. One is based on feature points, which calculates the homography matrix by finding the same-name points in the two images to complete the automatic alignment of the two images. The other is based on area image alignment, which rotates and moves the reference image to find the best matching position with the minimum absolute error or the maximum correlation coefficient as the measurement standard.

[0003] However, in the method based on feature point alignment, taking the working method of line laser scanning of workpieces as an example, the generated depth map has few texture features and is prone to noise and data holes, which limits the extraction of feature points and is not suitable for feature point matching. In the area-based image alignment method, the target position is mainly detected by the similarity between the sub-image and the original image. However, under interference factors such as rotation of the target, deformation caused by local defects, and noise, the accuracy of automatic alignment will be greatly reduced.

[0004] With regard to the problem of low efficiency of image alignment in related technologies, no effective solution has been proposed so far. Summary of the invention

[0005] The main purpose of the present application is to provide an image alignment method, device, storage medium and electronic device to solve the problem of low image alignment efficiency in related technologies.

[0006] In order to achieve the above-mentioned purpose, according to the first aspect of the present application, an image alignment method is provided. The method includes: determining the profile contour information of the target image and the template image, wherein the profile contour information includes: template contour information and target contour information; filtering the profile contour information according to a preset filtering method to obtain the filtered profile contour information, wherein the preset filtering method includes at least one of the following: line length filtering method and similarity filtering method, and the filtered profile contour information includes at least one of the following: initial profile contour information and final profile contour information; matching the filtered profile contour information according to a preset matching method to obtain target matching information, wherein the preset matching method includes at least one of the following: a first matching method and a second matching method, and the target matching information includes at least one of the following: initial matching information and final matching information, wherein the matching accuracy of the first matching method is lower than the matching accuracy of the second matching method; mapping the target image to the template image based on the target matching information.

[0007] Furthermore, determining the profile contour information of the target image and the template image includes: calculating the depth histogram of the target image and the template image, wherein the depth histogram includes: a template depth interval and a target depth interval; extracting the interval contour information of the template depth interval and the target depth interval, and using the interval contour information as the profile contour information of the target image and the template image, wherein the profile contour information includes: template profile contour information and target profile contour information.

[0008] Furthermore, the preset filtering method is a line length filtering method, and the filtered profile contour information is the initial profile contour information. The profile contour information is filtered according to the preset filtering method to obtain the filtered profile contour information, including: extracting the contour line length of the profile contour information; if the contour line length is detected to be greater than the preset contour line length, the profile contour information corresponding to the contour line length is used as the initial profile contour information, wherein the initial profile contour information includes: initial target profile contour information and initial template profile contour information.

[0009] Furthermore, the preset filtering method is a similarity filtering method, and the filtered profile profile information is the final profile profile information. The profile profile information is filtered according to the preset filtering method to obtain the filtered profile profile information, including: determining the Hu moment of the initial profile profile information; determining the similarity factor of the initial profile profile information based on the Hu moment; if it is detected that the similarity factor is greater than the preset similarity factor, the profile profile information corresponding to the similarity factor is used as the final profile profile information, wherein the final profile profile information includes: the final target profile profile information and the final template profile information.

[0010] Furthermore, the preset matching method is the first matching method, the target matching information is the initial matching information, and the filtered profile contour information is matched according to the preset matching method to obtain the target matching information, including: calculating the contour moment of the final profile contour information, wherein the contour moment is used to describe the global characteristics of the final profile contour information; calculating the parameter difference of the final profile contour information based on the contour moment, wherein the parameter difference includes: the center of mass position difference, the rotation angle difference; determining the initial matching information of the final profile contour information based on the parameter difference, wherein the initial matching information includes: the first transformation matrix.

[0011] Furthermore, the preset matching method is the second matching method, the target matching information is the final matching information, and the filtered profile contour information is matched according to the preset matching method to obtain the target matching information, including: inputting the first transformation matrix in the initial matching information into the target point cloud registration algorithm to obtain the second transformation matrix; determining the contour line length weight of the final profile contour information, and multiplying and superimposing the contour line length weight and the second transformation matrix to obtain the final matching information.

[0012] Furthermore, determining the contour line length weight of the final profile profile information includes: obtaining the contour line length of the final profile profile information; and normalizing the contour line length to obtain the contour line length weight.

[0013] Furthermore, before determining the contour information of the target image and the template image, the method further includes: performing filtering processing on the initial image and the initial template respectively to obtain the target image and the template image.

[0014] In order to achieve the above-mentioned purpose, according to the second aspect of the present application, an image alignment device is provided. The device includes: a determination unit, which is used to determine the profile contour information of the target image and the template image, wherein the profile contour information includes: template contour information and target contour information; a screening unit, which is used to screen the profile contour information according to a preset screening method to obtain the screened profile contour information, wherein the preset screening method includes at least one of the following: line length screening method and similarity screening method, and the screened profile contour information includes at least one of the following: initial profile contour information and final profile contour information; a matching unit, which is used to match the screened profile contour information according to a preset matching method to obtain target matching information, wherein the preset matching method includes at least one of the following: a first matching method and a second matching method, and the target matching information includes at least one of the following: initial matching information and final matching information, wherein the matching accuracy of the first matching method is lower than the matching accuracy of the second matching method; and a mapping unit, which is used to map the target image to the template image based on the target matching information.

[0015] According to a third aspect of an embodiment of the present application, an electronic device is provided, including: a processor; and a memory, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned image alignment methods is implemented.

[0016] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image alignment method according to any one of the above items is implemented.

[0017] Through the present application, the following steps are adopted: determining the profile contour information of the target image and the template image, wherein the profile contour information includes: template contour information and target contour information; filtering the profile contour information according to a preset filtering method to obtain the filtered profile contour information, wherein the preset filtering method includes at least one of the following: line length filtering method and similarity filtering method, and the filtered profile contour information includes at least one of the following: initial profile contour information and final profile contour information; matching the filtered profile contour information according to a preset matching method to obtain target matching information, wherein the preset matching method includes at least one of the following: a first matching method and a second matching method, and the target matching information includes at least one of the following: initial matching information and final matching information, wherein the matching accuracy of the first matching method is lower than the matching accuracy of the second matching method; matching the target image and the template image based on the target matching information, and the present application solves the problem of low image alignment efficiency in the related art by filtering the contour and subsequently matching the contour. Thus, the effect of improving the image alignment efficiency is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a flowchart of an image alignment method provided according to an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of an image alignment device provided according to an embodiment of the present application;

[0021] Figure 3 It is a schematic diagram of the network architecture of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0026] Hu moment: It is an image feature that is invariant to translation, rotation, and scale.

[0027] ICP algorithm (Iterative Closest Point): It is a widely used point cloud registration algorithm, whose main purpose is to align two or more point cloud data sets.

[0028] According to an embodiment of the present application, an image alignment method is provided.

[0029] Figure 1 FIG. 1 is a flow chart of an image alignment method according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0030] Step S101, determining the cross-sectional contour information of the target image and the template image, wherein the cross-sectional contour information includes: template contour information and target contour information.

[0031] Among them, the target image can be an image taken of the real object, or a scanned image, or an industrial image that needs to be measured with high precision. The template image can be a known, standard reference image, can contain specific image features or patterns, or can be a logo, symbol, or icon, etc.; among them, the profile contour information can be the position of the contour line, the shape of the contour line, the number of contour lines, the size of the contour line, etc., which is not limited in this application.

[0032] Optionally, the target image in the present application corresponds to target contour information, and the template image corresponds to template contour information.

[0033] It should be noted that, before determining the contour information of the target image and the template image, the embodiment of the present application further includes: performing filtering processing on the initial image and the initial template respectively to obtain the target image and the template image.

[0034] Exemplarily, by performing Gaussian filtering on the initial image and the initial template and removing abnormal points, the target image and template image in the present application are obtained, and the image after filtering is visually clearer and more delicate, which is more conducive to subsequent image analysis and processing.

[0035] In some embodiments provided in the present application, the cross-sectional contour information of the target image and the template image can be determined by the following steps: calculating the depth histogram of the target image and the template image, wherein the depth histogram includes: a template depth interval and a target depth interval; extracting the interval contour information of the template depth interval and the target depth interval, and using the interval contour information as the cross-sectional contour information of the target image and the template image, wherein the cross-sectional contour information includes: template cross-sectional contour information and target cross-sectional contour information.

[0036] Exemplarily, the depth histogram of the template image is calculated. The template histogram is divided into 50 intervals according to the depth. The 10 intervals with the most points falling into the intervals are selected. The average depth of each interval is depth1, depth2, ..., depth9, depth10 respectively; taking depth1 as an example, the template profile information at the depth position of depth1 is extracted. The specific operation method is:

[0037] 1. The template image is A, the size of A is m rows and n columns. A is threshold segmented with depth1 as the threshold. The values ​​above the threshold are set to 255, and the values ​​below the threshold or empty data are set to 0. The binary image B is obtained. B is extended by 2 pixels and filled with 0 to obtain image C, with a size of m+4 rows and n+4 columns.

[0038] 2. Take any point on the edge of C as the seed point (x, y), x and y are the row and column coordinates of the seed point, I (x, y) is the pixel value, and I (x, y) is 0 at this time;

[0039] 3. Create a new image flag with the same size as C and assign all values ​​to 0;

[0040] 4. Checking the current seed point (x, y) includes the following steps:

[0041] a. Check whether it is within the range of image C, if not, return.

[0042] b. If flag(x, y) = 1, then return.

[0043] c. If I(x, y) = 255, then return,

[0044] d. Assign flag(x, y) = 1, assign I(x, y) = 255, enter 5,

[0045] 5. Change the four points above, below, left and right of the seed point I (x+1, y), I (x, y+1), I (x, y-1), and I (x-1, y) into seed points and enter 4.

[0046] 6. Execute 4 recursively until all points a, b, and c in 4 are returned. The algorithm ends and image D is obtained. The two pixel parts extending from the edge of D are cropped to obtain image E. Obviously, the size of E is m rows and n columns.

[0047] 7. Invert image E, that is, the value of 0 becomes 255, and the value of 255 becomes 0, and image F is obtained;

[0048] 8. Perform a dot addition operation on images F and B, and then extract the contour lines to obtain the outer contour Ldepth1 of the template image A at a depth of depth1 (that is, the template cross-sectional contour information in this application). Similarly, according to the above method for extracting the template cross-sectional contour information at a depth position of depth1, the outer contours Ldepth1, ..., Ldepth10 of all depths of depth1, depth2, ..., depth9, and depth10 can be extracted.

[0049] It should be noted that the principle of determining the target depth interval according to the target image and extracting the target profile profile information of the target depth interval is similar to the principle of determining the template depth interval according to the template image and extracting the template profile information of the template depth interval, which will not be described in detail here. The present application can effectively improve the accuracy of subsequent image alignment by determining the profile profile information.

[0050] Step S102, filtering the profile profile information according to a preset filtering method to obtain filtered profile profile information, wherein the preset filtering method includes at least one of the following: a line length filtering method, a similarity filtering method, and the filtered profile profile information includes at least one of the following: initial profile profile information, final profile profile information.

[0051] The line length screening method may be a contour line length screening method, and the similarity screening method includes: similarity calculation between the target image, the template image, and the profile contour information of the corresponding depth intervals.

[0052] In some embodiments provided in the present application, the preset filtering method is a line length filtering method, and the filtered profile contour information is the initial profile contour information. The profile contour information is filtered according to the preset filtering method to obtain the filtered profile contour information, which can be obtained through the following steps: extracting the contour line length of the profile contour information; if it is detected that the contour line length is greater than the preset contour line length, then taking the profile contour information corresponding to the contour line length as the initial profile contour information, wherein the initial profile contour information includes: initial target profile contour information and initial template profile contour information.

[0053] Exemplarily, the template profile contour information and the target profile contour information are respectively subjected to a first contour screening according to the contour line length to obtain the initial profile contour information of the present application. Specifically, the contour line lengths are sorted, and contours with contour line lengths lower than 30% of the maximum length (corresponding to the preset contour line length in the present application) are eliminated. Assuming that there are 8 contours left, the corresponding depth positions are renumbered as depth1, depth2, ..., depth7, depth8, and the profile contour information corresponding to the remaining 8 contour lengths is used as the initial profile contour information. The present application ensures the measurement accuracy of subsequent image alignment through the first screening of contour lines.

[0054] Optionally, in some embodiments provided in the present application, the preset screening method is a similarity screening method, and the filtered profile profile information is the final profile profile information. The profile profile information is screened according to the preset screening method to obtain the filtered profile profile information, which can be achieved through the following steps: determining the Hu moment of the initial profile profile information; determining the similarity factor of the initial profile profile information based on the Hu moment; if it is detected that the similarity factor is greater than the preset similarity factor, then the profile profile information corresponding to the similarity factor is used as the final profile profile information, wherein the final profile profile information includes: final target profile profile information and final template profile information.

[0055] Exemplarily, the Hu moment of the initial template profile information determined according to the template image is obtained by the following steps: calculating the Hu moment of the initial profile information corresponding to 8 profile lengths, assuming that the Hu moments are J1, J2, ... J7, J8, taking J1 as an example, J1 contains 7 values, J11, J12, ..., J17, and the scales of J11, J12, ..., J17 are unified into Js11, Js12, ..., Js17, and the calculation formula is:

[0056] ;

[0057] Among them, Js i represents the Hu moment after scale unification, i is a natural number, It is the Hu moment before scale unification. It should be noted that in the present application, sign() is a sign function, that is, the sign() function is a function that determines the positive or negative value of an input value.

[0058] The same logic as the above example, the Hu moment is unified according to the scale of the 8 contours corresponding to the depth of the target image: .

[0059] That is, the present application ensures that the subsequent matching results are not affected by the scale and improves the accuracy of the subsequent analysis by respectively obtaining the Hu moments of the target image and the template image with unified scales.

[0060] Specifically, the similarity factor of the initial profile information is determined based on the Hu moment, where the calculation formula of the similarity factor is:

[0061] ;

[0062] Among them, A is the template image, T is the target image, is the depth interval number, is the Hu moment of each contour in a single depth interval after the template image is scaled uniformly. It is the Hu moment of each contour in a single depth interval position after the target image is scaled uniformly. The value of q can be 1, 2, 3, 4, 5, 6, 7, or 8. It should be noted that in this application, similar() is a function that calculates the similarity factor between the template image and the target image under the constraint of the depth interval number.

[0063] That is, the present application ensures the accuracy of subsequent matching results by determining the similarity factor.

[0064] Exemplarily, if it is detected that the similarity factor is greater than the preset similarity factor, the profile information corresponding to the similarity factor can be obtained as the final profile information by:

[0065] Sort the similar (A, T, depthq) similarity factors of the above example from large to small, and perform a second contour screening. If a similarity factor threshold is set as the preset similarity factor in this application, if it is detected that there are three similarity factors greater than the preset similarity factor, then the contour pairs corresponding to the three depth intervals with the greatest similarity are selected as the final profile contour information, among which the final target profile contour information and the final template profile contour information in the final profile contour information are contour pairs of each other.

[0066] That is, the present application obtains the final profile information through secondary profile screening, which effectively ensures the accuracy of subsequent profile matching.

[0067] Step S103, matching the filtered profile information according to a preset matching method to obtain target matching information, wherein the preset matching method includes at least one of the following: a first matching method, a second matching method, and the target matching information includes at least one of the following: initial matching information, final matching information, wherein the matching accuracy of the first matching method is lower than the matching accuracy of the second matching method.

[0068] The first matching method may be a coarse matching method, and the second matching method may be a fine matching method.

[0069] Specifically, the preset matching method is the first matching method, the target matching information is the initial matching information, and the filtered profile contour information is matched according to the preset matching method to obtain the target matching information, which can be achieved through the following steps: calculating the contour moment of the final profile contour information, wherein the contour moment is used to describe the global characteristics of the final profile contour information; calculating the parameter difference of the final profile contour information based on the contour moment, wherein the parameter difference includes: center of mass position difference, rotation angle difference; determining the initial matching information of the final profile contour information based on the parameter difference, wherein the initial matching information includes: a first transformation matrix.

[0070] Exemplarily, according to the three contour pairs selected above, rough matching of graphics is performed. Taking the first contour pair as an example, the zero-order moments M of the two contours are calculated respectively. 00 , first-order moment M 10 , M 01 , the second-order moment M 20 , M 02 , M 11 , calculate the centroid x of the contour c and c , rotation direction θ, the calculation formula is:

[0071] ;

[0072] Calculate the centroid position difference and angle difference Δx, Δy, Δθ of the two contour pairs respectively.

[0073] The transformation matrix is ​​obtained as .

[0074] That is, the transformation matrix R obtained in this embodiment can be used as the initial matching information of this application, and the initial matching information obtained based on the rough matching of the graphics highlights the features, reduces the calculation amount of the subsequent matching, and thus improves the image alignment efficiency.

[0075] Optionally, in the embodiment provided in the present application, the preset matching method is the second matching method, the target matching information is the final matching information, and the filtered profile contour information is matched according to the preset matching method to obtain the target matching information, which can be obtained in the following way: input the first transformation matrix in the initial matching information into the target point cloud registration algorithm to obtain the second transformation matrix; determine the contour line length weight of the final profile contour information, and multiply and superimpose the contour line length weight and the second transformation matrix to obtain the final matching information.

[0076] Exemplarily, the ICP matching algorithm is executed with R as the initial value to obtain the precise matching transformation matrix of three groups of contour pairs (that is, the second transformation matrix in this application) Rs1, Rs2, and Rs3, and the contour line length weight p is calculated according to the contour line length weight p. i (In this example, i=1, 2, 3), and the final transformation matrix is ​​obtained (equivalent to the final matching information in this application) .

[0077] That is, the final matching information obtained by the present application based on precise graphic matching effectively highlights the features, greatly reduces the amount of subsequent matching calculations, and thus improves the efficiency of image alignment.

[0078] The above-mentioned contour line length weight is realized by the following steps: obtaining the contour line length of the final profile contour information; normalizing the contour line length to obtain the contour line length weight.

[0079] Exemplarily, the lengths of the contour lines of the three final profile information are obtained, and corresponding normalized weights are set for them. , i=1, 2, 3, where, The formula is:

[0080] ,

[0081] Among them, lo i is the length of the i-th contour,

[0082] That is, the final matching information obtained by the precise matching in the present application is combined with the image feature matching while also referring to the weight factor, which greatly reduces the amount of subsequent matching calculations while also improving the image alignment accuracy.

[0083] Step S104: mapping the target image into the template image based on the target matching information.

[0084] Specifically, according to the target matching information, each pixel of the target image will be mapped to the corresponding position of the template image according to the transformation function. The mapping method of the target matching information obtained based on the contour information of the present application can effectively improve the efficiency of image alignment.

[0085] In summary, the image alignment method provided by the embodiment of the present application determines the profile contour information of the target image and the template image, wherein the profile contour information includes: template contour information and target contour information; the profile contour information is filtered according to a preset filtering method to obtain the filtered profile contour information, wherein the preset filtering method includes at least one of the following: line length filtering method and similarity filtering method, and the filtered profile contour information includes at least one of the following: initial profile contour information and final profile contour information; the filtered profile contour information is matched according to a preset matching method to obtain target matching information, wherein the preset matching method includes at least one of the following: a first matching method and a second matching method, and the target matching information includes at least one of the following: initial matching information and final matching information, wherein the matching accuracy of the first matching method is lower than the matching accuracy of the second matching method; the target image and the template image are matched based on the target matching information, and the present application solves the problem of low image alignment efficiency in the related art by filtering the contour and subsequently matching the contour. Thus, the effect of improving the image alignment efficiency is achieved.

[0086] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0087] The embodiment of the present application also provides an image alignment device. It should be noted that the image alignment device of the embodiment of the present application can be used to execute the image alignment method provided in the embodiment of the present application. The image alignment device provided in the embodiment of the present application is introduced below.

[0088] Figure 2 is a schematic diagram of an image alignment device 200 according to an embodiment of the present application. Figure 2 As shown, the device includes: a determination unit 201, a screening unit 202, a matching unit 203, and a mapping unit 204.

[0089] Specifically, the determination unit 201 is used to determine the cross-sectional profile information of the target image and the template image, wherein the cross-sectional profile information includes: template profile information and target profile information;

[0090] A screening unit 202 is used to screen the profile information according to a preset screening method to obtain screened profile information, wherein the preset screening method includes at least one of the following: a line length screening method and a similarity screening method, and the screened profile information includes at least one of the following: initial profile information and final profile information;

[0091] The matching unit 203 is used to match the filtered profile information according to a preset matching method to obtain target matching information, wherein the preset matching method includes at least one of the following: a first matching method and a second matching method, and the target matching information includes at least one of the following: initial matching information and final matching information, wherein the matching accuracy of the first matching method is lower than the matching accuracy of the second matching method;

[0092] The mapping unit 204 is used to map the target image into the template image based on the target matching information.

[0093] In summary, the image alignment device provided by the embodiment of the present application determines the profile contour information of the target image and the template image through the determination unit 201, wherein the profile contour information includes: template contour information and target contour information; the screening unit 202 screens the profile contour information according to a preset screening method to obtain the screened profile contour information, wherein the preset screening method includes at least one of the following: line length screening method and similarity screening method, and the screened profile contour information includes at least one of the following: initial profile contour information and final profile contour information; the matching unit 203 matches the screened profile contour information according to a preset matching method to obtain target matching information, wherein the preset matching method includes at least one of the following: a first matching method and a second matching method, and the target matching information includes at least one of the following: initial matching information and final matching information, wherein the matching accuracy of the first matching method is lower than the matching accuracy of the second matching method; the mapping unit 204 maps the target image to the template image based on the target matching information. The problem of low image alignment efficiency in the related art is solved. By screening the contour and subsequently matching the contour, the effect of improving the image alignment efficiency is achieved. Furthermore, it includes: a first calculation module, used to calculate the depth histogram of the target image and the template image, wherein the depth histogram includes: a template depth interval and a target depth interval; a first extraction module, used to extract the interval contour information of the template depth interval and the target depth interval, and use the interval contour information as the profile contour information of the target image and the template image, wherein the profile contour information includes: template profile contour information and target profile contour information.

[0094] Optionally, in the image alignment device provided in the embodiment of the present application, the preset filtering method is the line length filtering method, and the filtered profile contour information is the initial profile contour information. The filtering unit includes: a second extraction module, used to extract the contour line length of the profile contour information; a first determination module, used to use the profile contour information corresponding to the contour line length as the initial profile contour information if it is detected that the contour line length is greater than the preset contour line length, wherein the initial profile contour information includes: initial target profile contour information and initial template profile contour information.

[0095] Optionally, in the image alignment device provided in the embodiment of the present application, the preset screening method is a similarity screening method, and the profile contour information after screening is the final profile contour information. The screening unit includes: a second determination module, used to determine the Hu moment of the initial profile contour information; a third determination module, used to determine the similarity factor of the initial profile contour information based on the Hu moment; a fourth determination module, used to use the profile contour information corresponding to the similarity factor as the final profile contour information if it is detected that the similarity factor is greater than the preset similarity factor, wherein the final profile contour information includes: final target profile contour information and final template profile contour information.

[0096] Optionally, in the image alignment device provided in the embodiment of the present application, the preset matching mode is the first matching mode, the target matching information is the initial matching information, and the matching unit includes: a second calculation module, used to calculate the contour moment of the final profile contour information, wherein the contour moment is used to describe the global characteristics of the final profile contour information; a third calculation module, used to calculate the parameter difference of the final profile contour information based on the contour moment, wherein the parameter difference includes: center of mass position difference, rotation angle difference; a fifth determination module, used to determine the initial matching information of the final profile contour information based on the parameter difference, wherein the initial matching information includes: a first transformation matrix.

[0097] Optionally, in the image alignment device provided in the embodiment of the present application, the preset matching method is the second matching method, the target matching information is the final matching information, and the matching unit includes: an input module, used to input the first transformation matrix in the initial matching information into the target point cloud registration algorithm to obtain the second transformation matrix; a sixth determination module, used to determine the contour line length weight of the final profile contour information, and multiply and superimpose the contour line length weight and the second transformation matrix to obtain the final matching information.

[0098] Optionally, in the image alignment device provided in the embodiment of the present application, the sixth determination module includes: an acquisition submodule, used to obtain the contour line length of the final profile contour information; and a processing submodule, used to normalize the contour line length to obtain the contour line length weight.

[0099] Optionally, in the image alignment device provided in the embodiment of the present application, the device also includes: a processing unit, which is used to filter the initial image and the initial template respectively before determining the contour information of the target image and the template image to obtain the target image and the template image.

[0100] The image alignment device 200 includes a processor and a memory. The determination unit 201, screening unit 202, matching unit 203, mapping unit 204, etc. are all stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions.

[0101] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and image alignment is performed by adjusting kernel parameters.

[0102] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0103] In the exemplary embodiment of the present application, a computer storage medium capable of implementing the above method is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible embodiments, various aspects of the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps described in the above "Exemplary Method" section of this specification according to various exemplary embodiments of the present application, such as the following steps may be executed: determining profile contour information of a target image and a template image, wherein the profile contour information includes: template contour information and target contour information; filtering the profile contour information according to a preset filtering method to obtain filtered profile contour information, wherein the preset filtering method includes at least one of the following: line length filtering method and similarity filtering method, and the filtered profile contour information includes at least one of the following: initial profile contour information and final profile contour information; matching the filtered profile contour information according to a preset matching method to obtain target matching information, wherein the preset matching method includes at least one of the following: a first matching method and a second matching method, and the target matching information includes at least one of the following: initial matching information and final matching information, wherein the matching accuracy of the first matching method is lower than the matching accuracy of the second matching method; mapping the target image to the template image based on the target matching information.

[0104] In an optional implementation: calculate depth histograms of the target image and the template image, wherein the depth histograms include: template depth intervals and target depth intervals; extract interval contour information of the template depth intervals and the target depth intervals, and use the interval contour information as profile contour information of the target image and the template image, wherein the profile contour information includes: template profile contour information and target profile contour information.

[0105] In an optional implementation: extract the contour line length of the profile contour information; if it is detected that the contour line length is greater than the preset contour line length, use the profile contour information corresponding to the contour line length as the initial profile contour information, wherein the initial profile contour information includes: initial target profile contour information, initial template profile contour information.

[0106] In an optional implementation: determine the Hu moment of initial profile contour information; determine the similarity factor of the initial profile contour information based on the Hu moment; if it is detected that the similarity factor is greater than a preset similarity factor, then use the profile contour information corresponding to the similarity factor as the final profile contour information, wherein the final profile contour information includes: final target profile contour information, final template profile contour information.

[0107] In an optional embodiment: the contour moments of the final profile contour information are calculated, wherein the contour moments are used to describe the global characteristics of the final profile contour information; the parameter differences of the final profile contour information are calculated based on the contour moments, wherein the parameter differences include: the center of mass position difference, the rotation angle difference; the initial matching information of the final profile contour information is determined based on the parameter differences, wherein the initial matching information includes: a first transformation matrix.

[0108] In an optional embodiment: the first transformation matrix in the initial matching information is input into the target point cloud registration algorithm to obtain the second transformation matrix; the contour line length weight of the final profile information is determined, and the contour line length weight and the second transformation matrix are multiplied and superimposed to obtain the final matching information.

[0109] In an optional implementation: the contour line length of the final profile information is obtained; and the contour line length is normalized to obtain a contour line length weight.

[0110] In an optional implementation manner: filtering is performed on the initial image and the initial template respectively to obtain the target image and the template image.

[0111] In an optional embodiment, the embodiment of the present application may also include a program product for implementing the above method, which may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0112] The program product may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0113] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0114] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0115] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0116] In addition, in an exemplary embodiment of the present application, an electronic device capable of implementing the above method is also provided.

[0117] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0118] Refer to the following Figure 3 The electronic device 300 according to this embodiment of the present application is described. Figure 3 The electronic device 300 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0119] like Figure 3 As shown, the electronic device 300 is in the form of a general computing device. The components of the electronic device 300 may include but are not limited to: at least one processing unit 310, at least one storage unit 320, a bus 330 connecting different system components (including the storage unit 320 and the processing unit 310), and a display unit 340.

[0120] The storage unit stores a program code, and the program code can be executed by the processing unit 310, so that the processing unit 310 executes the steps according to various exemplary embodiments of the present application described in the above "Exemplary Method" section of this specification. For example, the processing unit 310 can execute the following steps: determining the profile profile information of the target image and the template image, wherein the profile profile information includes: template profile information and target profile information; filtering the profile profile information according to a preset filtering method to obtain the filtered profile profile information, wherein the preset filtering method includes at least one of the following: line length filtering method and similarity filtering method, and the filtered profile profile information includes at least one of the following: initial profile profile information and final profile profile information; matching the filtered profile profile information according to a preset matching method to obtain target matching information, wherein the preset matching method includes at least one of the following: a first matching method and a second matching method, and the target matching information includes at least one of the following: initial matching information and final matching information, wherein the matching accuracy of the first matching method is lower than the matching accuracy of the second matching method; mapping the target image to the template image based on the target matching information.

[0121] In an optional implementation: calculate depth histograms of the target image and the template image, wherein the depth histogram includes: template depth intervals and target depth intervals; extract interval contour information of the template depth intervals and the target depth intervals, and use the interval contour information as profile contour information of the target image and the template image, wherein the profile contour information includes: template profile contour information and target profile contour information.

[0122] In an optional implementation: extract the contour line length of the profile contour information; if it is detected that the contour line length is greater than the preset contour line length, use the profile contour information corresponding to the contour line length as the initial profile contour information, wherein the initial profile contour information includes: initial target profile contour information, initial template profile contour information.

[0123] In an optional implementation: determine the Hu moment of initial profile contour information; determine the similarity factor of the initial profile contour information based on the Hu moment; if it is detected that the similarity factor is greater than a preset similarity factor, then use the profile contour information corresponding to the similarity factor as the final profile contour information, wherein the final profile contour information includes: final target profile contour information, final template profile contour information.

[0124] In an optional embodiment: the contour moments of the final profile contour information are calculated, wherein the contour moments are used to describe the global characteristics of the final profile contour information; the parameter differences of the final profile contour information are calculated based on the contour moments, wherein the parameter differences include: the center of mass position difference, the rotation angle difference; the initial matching information of the final profile contour information is determined based on the parameter differences, wherein the initial matching information includes: a first transformation matrix.

[0125] In an optional embodiment: the first transformation matrix in the initial matching information is input into the target point cloud registration algorithm to obtain the second transformation matrix; the contour line length weight of the final profile information is determined, and the contour line length weight and the second transformation matrix are multiplied and superimposed to obtain the final matching information.

[0126] In an optional implementation: the contour line length of the final profile information is obtained; and the contour line length is normalized to obtain a contour line length weight.

[0127] In an optional implementation manner: filtering is performed on the initial image and the initial template respectively to obtain the target image and the template image.

[0128] The storage unit 320 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 3201 and / or a cache memory unit 3202 , and may further include a read-only memory unit (ROM) 3203 .

[0129] The storage unit 320 may also include a program / utility 3204 having a set (at least one) of program modules 3205, such program modules 3205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0130] Bus 330 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0131] The electronic device 300 may also communicate with one or more external devices 400 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 300, and / or may communicate with any device that enables the electronic device 300 to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication may be performed through an input / output (I / O) interface 350. In addition, the electronic device 300 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 360. As shown, the network adapter 360 communicates with other modules of the electronic device 300 through a bus 330. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0132] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation method of the present application.

[0133] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0134] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application. The specification and embodiments are to be regarded as exemplary only, and the true scope and spirit of the present application are indicated by the claims.

Claims

1. An image alignment method, characterized in that: include: Determine the profile information of the target image and the template image, wherein the profile information includes: template profile information and target profile information; The cross-sectional profile information is screened according to a preset screening method to obtain screened cross-sectional profile information, wherein the preset screening method includes at least one of the following: a line length screening method and a similarity screening method, and the screened cross-sectional profile information includes at least one of the following: initial cross-sectional profile information and final cross-sectional profile information; Matching the filtered profile information according to a preset matching method to obtain target matching information, wherein the preset matching method includes at least one of the following: a first matching method and a second matching method, and the target matching information includes at least one of the following: initial matching information and final matching information, wherein the matching accuracy of the first matching method is lower than the matching accuracy of the second matching method; Mapping the target image into the template image based on the target matching information; The preset screening method is the line length screening method, the screened profile information is the initial profile information, and the profile information is screened according to the preset screening method to obtain the screened profile information, including: Extracting the contour line length of the profile information; If it is detected that the contour line length is greater than the preset contour line length, the profile profile information corresponding to the contour line length is used as the initial profile profile information, wherein the initial profile profile information includes: initial target profile profile information and initial template profile profile information; The preset screening method is the similarity screening method, the screened profile information is the final profile information, and the profile information is screened according to the preset screening method to obtain the screened profile information, including: Determine the Hu moment of the initial profile information; Determining a similarity factor of the initial profile information based on the Hu moment; If it is detected that the similarity factor is greater than the preset similarity factor, the profile information corresponding to the similarity factor is used as the final profile information, wherein the final profile information includes: final target profile information and final template profile information; The preset matching method is the first matching method, the target matching information is the initial matching information, and the filtered profile information is matched according to the preset matching method to obtain the target matching information, including: Calculating a contour moment of the final profile information, wherein the contour moment is used to describe a global feature of the final profile information; Calculating parameter differences of the final profile information based on the profile moment, wherein the parameter differences include: centroid position differences and rotation angle differences; Determine initial matching information of the final profile information according to the parameter difference, wherein the initial matching information includes: a first conversion matrix; The preset matching method is the second matching method, the target matching information is the final matching information, and the filtered profile information is matched according to the preset matching method to obtain the target matching information, including: Inputting the first transformation matrix in the initial matching information into a target point cloud registration algorithm to obtain a second transformation matrix; The contour line length weight of the final profile information is determined, and the contour line length weight and the second conversion matrix are multiplied and superimposed to obtain the final matching information.

2. The image alignment method according to claim 1, characterized in that: Determine the profile information of the target image and the template image, including: Calculating depth histograms of the target image and the template image, wherein the depth histogram includes: a template depth interval and a target depth interval; The interval contour information of the template depth interval and the target depth interval is extracted, and the interval contour information is used as the section contour information of the target image and the template image, wherein the section contour information includes: template section contour information and target section contour information.

3. The image alignment method according to claim 1, characterized in that: Determining the contour line length weight of the final profile information includes: Obtaining the contour line length of the final profile information; The contour line length is normalized to obtain the contour line length weight.

4. The image alignment method according to claim 1, characterized in that: Before determining the profile information of the target image and the template image, the method further includes: The initial image and the initial template are filtered to obtain the target image and the template image.

5. An image alignment device, characterized in that: include: A determination unit, used to determine the profile information of the target image and the template image, wherein the profile information includes: template profile information and target profile information; A screening unit, used to screen the profile information according to a preset screening method to obtain screened profile information, wherein the preset screening method includes at least one of the following: a line length screening method, a similarity screening method, and the screened profile information includes at least one of the following: initial profile information, final profile information; A matching unit, configured to match the filtered profile information according to a preset matching method to obtain target matching information, wherein the preset matching method includes at least one of the following: a first matching method and a second matching method, and the target matching information includes at least one of the following: initial matching information and final matching information, wherein the matching accuracy of the first matching method is lower than the matching accuracy of the second matching method; A mapping unit, configured to map the target image to the template image based on the target matching information; Wherein, the preset screening method is the line length screening method, the screened section profile information is the initial section profile information, and the screening unit includes: a second extraction module, used to extract the contour line length of the section profile information; a first determination module, used to use the section profile information corresponding to the contour line length as the initial section profile information if it is detected that the contour line length is greater than the preset contour line length, wherein the initial section profile information includes: initial target section profile information, initial template section profile information; Wherein, the preset screening method is the similarity screening method, the screened profile information is the final profile information, and the screening unit includes: a second determination module, used to determine the Hu moment of the initial profile information; a third determination module, used to determine the similarity factor of the initial profile information based on the Hu moment; a fourth determination module, used to use the profile information corresponding to the similarity factor as the final profile information if it is detected that the similarity factor is greater than the preset similarity factor, wherein the final profile information includes: final target profile information and final template profile information; Wherein, the preset matching mode is the first matching mode, the target matching information is the initial matching information, and the matching unit includes: a second calculation module, used to calculate the contour moment of the final profile profile information, wherein the contour moment is used to describe the global characteristics of the final profile profile information; a third calculation module, used to calculate the parameter difference of the final profile profile information based on the contour moment, wherein the parameter difference includes: the center of mass position difference and the rotation angle difference; a fifth determination module, used to determine the initial matching information of the final profile profile information according to the parameter difference, wherein the initial matching information includes: a first transformation matrix; Among them, the preset matching method is the second matching method, the target matching information is the final matching information, and the matching unit includes: an input module, used to input the first transformation matrix in the initial matching information into the target point cloud registration algorithm to obtain the second transformation matrix; a sixth determination module, used to determine the contour line length weight of the final profile contour information, and multiply and superimpose the contour line length weight and the second transformation matrix to obtain the final matching information.

6. A computer-readable storage medium, characterized in that: The storage medium includes a stored program, wherein the program executes the image alignment method according to any one of claims 1 to 4.

7. An electronic device, characterized in that: include: One or more processors, a memory, a display device, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the image alignment method described in any one of claims 1 to 4.

Images