Image processing method and image processing apparatus

Abstract

The present disclosure provides an image processing method and an image processing device. The method comprises: obtaining a template image; performing blur processing on the template image to generate a plurality of processed template images with different blur degrees; obtaining a to-be-matched image; calculating a normalized correlation coefficient of each processed template image and the to-be-matched image respectively to obtain a set of normalized correlation coefficient matrices, each normalized correlation coefficient matrix corresponding to a processed template image, and each value in each normalized correlation coefficient matrix corresponding to a pixel position; dividing the maximum value of each normalized correlation coefficient matrix into a corresponding maximum value group; and calculating a matching score of each maximum value group respectively, taking the maximum value in the matching scores as a final value, and taking the pixel position corresponding to the final value as a matching position. The image processing method has better robustness, can avoid the wrong matching of images, and improves the tolerance of the imaging clarity of the to-be-matched image.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to an image processing method and an image processing device. Background Art

[0002] Existing image matching technology is usually achieved by comparing the matching scores calculated at each pixel position of the template image on the image to be matched. The more similar the template image is to a certain position on the image to be matched, the higher the matching score. Therefore, it is necessary to ensure that the clarity of the template image used and the image to be matched are the same.

[0003] However, in real-world applications, the template image can be acquired with the camera's focus clearly adjusted and stable, while the image to be matched may exhibit varying degrees of blur due to various factors during the acquisition process. Blur typically includes motion blur and defocus blur. In this case, matching a clear template image with the blurred image to be matched can result in low matching scores or even incorrect matches. Summary of the Invention

[0004] In response to the problems in the prior art, the purpose of the present disclosure is to provide an image processing method and an image processing device, which can avoid the mismatching situation that may occur when performing image matching through a single template image by matching a series of template images with different blur levels with the image to be matched.

[0005] At least one embodiment of the present disclosure provides an image processing method, which includes: obtaining a template image; blurring the template image to generate multiple processed template images with different degrees of blur; obtaining an image to be matched; respectively calculating the normalized correlation coefficient between each processed template image and the image to be matched to obtain a set of normalized correlation coefficient matrices, wherein each normalized correlation coefficient matrix corresponds to a processed template image, and each value in each normalized correlation coefficient matrix corresponds to a pixel position; dividing the maximum value of each normalized correlation coefficient matrix into corresponding maximum value groups; and respectively calculating the matching score of each maximum value group, taking the maximum value in the matching score as the final value, and taking the pixel position corresponding to the final value as the matching position.

[0006] For example, in the image processing method provided by at least one embodiment of the present disclosure, obtaining a template image includes: obtaining a clear and stable first image, and intercepting the template image in the first image, wherein the template image includes a first specified characteristic pattern.

[0007] For example, in the image processing method provided in at least one embodiment of the present disclosure, a template image is blurred to generate multiple processed template images with different degrees of blur, including: using multiple blur kernels to blur the template image separately to obtain multiple processed template images, wherein the blur kernel is used to simulate the blur situation that occurs when the device acquires the image.

[0008] For example, in the image processing method provided by at least one embodiment of the present disclosure, the multiple blur kernels include m Gaussian blur kernels and n motion blur kernels, m+n is greater than or equal to 1, and both m and n are natural numbers.

[0009] For example, in the image processing method provided by at least one embodiment of the present disclosure, the image to be matched includes a first specified feature pattern.

[0010] For example, in the image processing method provided by at least one embodiment of the present disclosure, multiple blur kernels are used to blur the template image respectively, including: when the image to be matched includes blur caused by motion, selecting the motion blur kernel to blur the template image; when the image to be matched includes blur caused by defocus, selecting the Gaussian blur kernel to blur the template image; when the image to be matched includes blur caused by motion and blur caused by defocus, selecting the motion blur kernel and the Gaussian blur kernel to blur the template image.

[0011] For example, in the image processing method provided by at least one embodiment of the present disclosure, the maximum value of each normalized correlation coefficient matrix is ​​divided into corresponding maximum value groups, including: obtaining the pixel position corresponding to each maximum value; and dividing the corresponding maximum value into corresponding maximum value groups according to the distance between the pixel positions.

[0012] For example, in the image processing method provided by at least one embodiment of the present disclosure, corresponding maximum values ​​are divided into corresponding maximum value groups according to the distance between pixel positions, including: calculating the distance between the pixel positions corresponding to each two maximum values; and completing the division based on the distance, wherein the distance between the pixel positions corresponding to any two values ​​in each maximum value group is not greater than a preset threshold.

[0013] For example, in the image processing method provided by at least one embodiment of the present disclosure, the matching score of each maximum value group is calculated separately, including: calculating the product of the maximum value of the values ​​in each maximum value group and the number of values ​​in the maximum value group to obtain the matching score of each maximum value group.

[0014] At least one embodiment of the present disclosure further provides an image processing device, comprising: a processor; a memory storing executable instructions of the processor, wherein the processor is configured to execute the steps of the image processing method provided by at least one embodiment of the present disclosure by executing the executable instructions.

[0015] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0016] The image processing method and image processing device disclosed herein have the following beneficial effects:

[0017] The image processing method has better robustness, can avoid image mismatching, and improve the tolerance of the imaging clarity of the image to be matched. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Other features, objects and advantages of the present disclosure will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following accompanying drawings.

[0019] Figure 1 A schematic flowchart of an image processing method provided in at least one embodiment of the present disclosure;

[0020] Figure 2 A schematic flowchart of step S105 provided for at least one embodiment of the present disclosure;

[0021] Figure 3 is an exemplary schematic diagram of a first image;

[0022] Figure 4 is an exemplary schematic diagram of a template image;

[0023] Figure 5 An exemplary schematic diagram of multiple processed template images provided for at least one embodiment of the present disclosure;

[0024] Figure 6 is an exemplary schematic diagram of an image to be matched;

[0025] Figure 7 An exemplary schematic diagram of the result of image matching using the image processing method provided by at least one embodiment of the present disclosure;

[0026] Figure 8 A schematic block diagram of an image processing device provided for at least one embodiment of the present disclosure;

[0027] Figure 9 A schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0029] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0030] The flowcharts shown in the accompanying drawings are merely exemplary and do not necessarily include all steps. For example, some steps may be decomposed, while some steps may be combined or partially combined. Therefore, the actual execution order may change according to actual circumstances.

[0031] In order to solve the above technical problems, the present disclosure proposes a new technical solution. When the clarity of the image to be matched is significantly different from that of the template image, a series of template images with different degrees of blur are matched with the image to be matched, and a specific weighting function is used to process them to obtain the final matching result. This can avoid the mismatching that may occur when performing image matching through a single template image and improve the matching success rate.

[0032] Figure 1 A schematic flowchart of an image processing method provided in at least one embodiment of the present disclosure.

[0033] like Figure 1 As shown, the image processing method includes the following steps S101 to S106.

[0034] Step S101: Acquire a template image.

[0035] For example, in some embodiments of the present disclosure, step S101 may include: acquiring a clear and stable first image, and intercepting a template image from the first image.

[0036] For example, the focal length of the camera may be adjusted to capture the first image when the image is clear and stable.

[0037] For example, the template image includes a first designated characteristic pattern. The first designated characteristic pattern represents a non-repeating and unique pattern in the first image. The template image can be obtained by intercepting a sub-image near the first designated characteristic pattern in the first image.

[0038] It should be noted that, when the cut-out template pattern includes the first designated characteristic pattern, the smaller the size, the better.

[0039] Step S102: performing blur processing on the template image to generate a plurality of processed template images with different blurring degrees.

[0040] Step S103: Acquire the image to be matched.

[0041] For example, the image to be matched includes a first specified characteristic pattern.

[0042] Step S104: Calculate the normalized correlation coefficient between each processed template image and the image to be matched, and obtain a set of normalized correlation coefficient matrices.

[0043] Normalized Cross Correlation (NCC) is usually used to describe the similarity between two signals. In image matching, the normalized cross correlation coefficient is usually used to describe the similarity between different images.

[0044] For example, each normalized correlation coefficient matrix corresponds to the degree of similarity between a processed template image and the image to be matched, and each value in each normalized correlation coefficient matrix corresponds to a pixel position.

[0045] For example, the process of calculating the normalized correlation coefficient can reduce the calculation time and improve efficiency through multi-threading.

[0046] Step S105: Divide the maximum value of each normalized correlation coefficient matrix into corresponding maximum value groups.

[0047] Step S106: Calculate the matching score of each maximum value group respectively, take the maximum value among the matching scores as the final value, and take the pixel position corresponding to the final value as the matching position.

[0048] For example, in some embodiments of the present disclosure, step S102 may include: using multiple blur kernels to blur the template image respectively to obtain multiple processed template images.

[0049] The image to be matched may have varying degrees of blur due to various reasons during the image acquisition process, usually including motion blur and defocus blur. The blur kernel can be used to simulate the blur that occurs when the device acquires the image. The Gaussian blur kernel can be used to simulate defocus blur, and the motion blur kernel can be used to simulate motion blur.

[0050] The template image is blurred using multiple blur kernels to obtain multiple processed template images with different blur levels, so as to simulate various non-clear imaging situations that may appear in the captured image by the image acquisition device. Using multiple processed template images to match the image to be matched can avoid mismatching, increase the tolerance for the imaging clarity of the image to be matched, and improve robustness.

[0051] For example, in some embodiments of the present disclosure, the multiple blur kernels include m Gaussian blur kernels and n motion blur kernels, where m+n is greater than or equal to 1, and both m and n are natural numbers.

[0052] It should be noted that the parameters of the m+n blur kernels are all different, so that each processed template image represents a different blur situation.

[0053] Different blur kernels can be selected for actual blur situations to improve the robustness and universality of matching. For example, in some embodiments of the present disclosure, using multiple blur kernels to blur the template image separately may include: when the image to be matched includes blur caused by motion, selecting a motion blur kernel to blur the template image; when the image to be matched includes blur caused by defocus, selecting a Gaussian blur kernel to blur the template image; when the image to be matched includes blur caused by motion and defocus, selecting a motion blur kernel and a Gaussian blur kernel to blur the template image.

[0054] For example, the fuzzy processing can be calculated by the following formula (1):

[0055] t i ′=t*k i (1)

[0056] where t i ′ represents the template image after the i-th blur kernel processing, t represents the acquired template image, k i= represents the i-th blur kernel among multiple blur kernels, and * represents a convolution operation. Formula (1) exemplarily represents blurring a template image using any one of the multiple blur kernels, and does not limit the technical solution of the present disclosure. For example, a combination of a Gaussian blur kernel and a motion blur kernel among the multiple blur kernels may also be selected to blur the template image. For ease of description, in the subsequent embodiments of the present disclosure, i is also used to represent any one of the m+n blur kernels.

[0057] Figure 2 A schematic flowchart of step S105 is provided for at least one embodiment of the present disclosure.

[0058] like Figure 2 As shown, step S105 includes the following steps S201 to S202.

[0059] Step S201: Obtain the pixel position corresponding to each maximum value.

[0060] Step S202: Divide the corresponding maximum values ​​into corresponding maximum value groups according to the distances between pixel positions.

[0061] For example, in some embodiments of the present disclosure, step S202 may include: calculating the distance between pixel positions corresponding to every two maximum values; and completing the division based on the distance.

[0062] For example, the distance between pixel positions can be the geometric distance between two pixels, which can be obtained by the following formula (2):

[0063]

[0064] where d i′j′ represents the distance between pixel points i′ and j′, x i′ and y i′ Indicates the horizontal and vertical coordinates of pixel i′, x j′ and y j′ Indicates the horizontal and vertical coordinates of pixel point j′.

[0065] For example, the division is completed according to the distance between pixel positions / the size of the distance between pixel positions, and the distance between pixel positions corresponding to any two values ​​in each maximum value group is not greater than a preset threshold.

[0066] For example, the preset threshold may be set based on experience.

[0067] For example, for a set of normalized correlation coefficient matrices {C1, C2, ..., C m+n}, get the maximum value {c1, c2, ..., c m+n}, and record the pixel position {p1, p2, ..., p m+n}; where C i represents the i-th normalized correlation coefficient matrix, c i represents the maximum value of the i-th normalized correlation coefficient matrix, whose value range is [0, 1], p i Indicates the pixel position of the i-th maximum value, which can be expressed as p i =(x i ,y i ), where i∈[1,m+n]. Calculate the distance between each two pixel positions and replace the maximum value {c1, c2, ..., c m+n} is divided into q maximum value groups in, Represents the qth maximum value group, and contains w q Correlation coefficient; w1+w2+...+w q =m+n.

[0068] The pixel positions corresponding to any two values ​​in each maximum group should satisfy the following relationship (3):

[0069] d hi′j′ ≤thresh, h∈[1, q], i′, j′∈[1, w h ] (3)

[0070] h represents the hth group divided, d hi′j′ represents the distance between the i′th pixel position and the j′th pixel position in the hth group, w h It represents the number of elements in the hth group, and thresh represents the set grouping distance threshold, which can be set according to experience.

[0071] For example, in some embodiments of the present disclosure, step S106 may include: respectively calculating the product of the maximum value of the values ​​in each maximum value group and the number of values ​​in the maximum value group to obtain the matching score of each maximum value group.

[0072] For example, for the above q maximum value groups Calculate the maximum value of each maximum value group separately {c max1 , c max2 ,...,c maxq} multiplied by the number of values ​​in the maximum value group, we get the matching score of each maximum value group {s1, s2, ..., s q}, {s1, s2, ..., s q The maximum value s in maxAs the final value, the final value s max The corresponding pixel position p max As the matching position, where c maxq is the maximum value of the qth maximum value group, s max The value range is [0, m+n].

[0073] The image processing method provided by the embodiment of the present disclosure is described below through a specific embodiment.

[0074] First, a first image 300 is acquired. Figure 3 As shown. Figure 3 In the example, the first image 300 includes four independent patterns, three of which are square patterns and one is a cross pattern. Since the cross pattern is a non-repeating and unique pattern in the first image 300, the cross pattern can be used as the first designated characteristic pattern, and a sub-image near the first designated characteristic pattern can be intercepted as the template image 400. The intercepted template image 400 is used in the Figure 4 Shown in.

[0075] Then, 6 different Gaussian blur kernels are used to Figure 4 The template images 400 shown are blurred and the following are obtained: Figure 5 The 6 processed template images shown are from Figure 5 It can be seen that the blur level of each image is different.

[0076] Then, obtain the image 600 to be matched. The image 600 to be matched is as follows: Figure 6 As shown. Figure 6 In the example, the image to be matched 600 includes a first designated characteristic pattern (a cross pattern). Figure 6 It can be seen that compared with Figure 3 , Figure 6 The image to be matched is shown to have obvious out-of-focus blur.

[0077] Then, the normalized correlation coefficients of each processed template image and the image to be matched 600 are calculated to obtain a set of (6) normalized correlation coefficient matrices. Each normalized correlation coefficient matrix corresponds to a processed template image, and each value in each normalized correlation coefficient matrix corresponds to a pixel position.

[0078] Then, the maximum value of each normalized correlation coefficient matrix is ​​divided into the corresponding maximum value group.

[0079] Finally, calculate the matching score of each maximum value group, take the maximum value of the matching score as the final value, and take the pixel position corresponding to the final value as the matching position. The final matching result is as follows Figure 7As shown, the final value range is [0, 6], which is 5.99996, and the matching position is (749, 693). Figure 7 It can be seen that the matching position is accurate and the matching score is high, which can well complete the image matching task.

[0080] In summary, the image processing method provided by the present disclosure can match the image to be matched through a series of template images with different blur levels, and use a specific weighting function for processing to obtain the final matching result. It can avoid the mismatching situation that may occur when performing image matching through a single template image, and improve the matching success rate.

[0081] At least one embodiment of the present disclosure further provides an image processing device, comprising a processor and a memory. The memory stores computer-executable instructions that, when executed by the processor, can implement the image processing method provided in at least one embodiment of the present disclosure.

[0082] Figure 8 Schematic block diagram of an image processing device 800 provided in some embodiments of the present disclosure. Figure 8 As shown, the image processing device 800 includes a processor 810 and a memory 820. The memory 820 is used to store computer-executable instructions (e.g., one or more computer program modules). The processor 810 is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor 810, one or more steps in the image processing method described above can be performed. The memory 820 and the processor 810 can be interconnected via a bus system and / or other form of connection mechanism (not shown).

[0083] For example, the processor 810 may be a central processing unit (CPU), a graphics processing unit (GPU), or other processing units with data processing capabilities and / or program execution capabilities. For example, the central processing unit (CPU) may be an X86 or ARM architecture. The processor 810 may be a general-purpose processor or a dedicated processor, and may control other components in the image processing device 800 to perform desired functions.

[0084] For example, the memory 820 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, a flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor 810 may execute one or more computer program modules to implement various functions of the image processing device 800. The computer-readable storage medium may also store various applications and various data, as well as various data used and / or generated by the applications.

[0085] It should be noted that, in the embodiment of the present disclosure, the specific functions and technical effects of the image processing device 800 can be referred to the above description of the image processing method, which will not be repeated here.

[0086] Figure 9 This is a schematic block diagram of an electronic device provided in some embodiments of the present disclosure. The electronic device 900 is suitable for implementing the image processing method provided in the embodiments of the present disclosure. The electronic device 900 can be a terminal device or a computer system. It should be noted that Figure 9 The electronic device 900 shown is merely an example and does not limit the functions and scope of use of the embodiments of the present disclosure.

[0087] like Figure 9 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 910, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 920 or a program loaded from a storage device 980 into a random access memory (RAM) 930. Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 930. The processing device 910, the ROM 920, and the RAM 930 are connected to each other via a bus 940. An input / output (I / O) interface 950 is also connected to the bus 940.

[0088] Typically, the following devices may be connected to the I / O interface 950: an input device 960 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 970 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 980 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 990. The communication device 990 may allow the electronic device 900 to communicate with other electronic devices wirelessly or by wire to exchange data. Although Figure 9 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown, and the electronic device 900 may instead implement or possess more or fewer devices.

[0089] For example, according to an embodiment of the present disclosure, the above-mentioned image processing method can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for executing the above-mentioned image processing method. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 990, or installed from the storage device 980, or installed from the ROM 920. When the computer program is executed by the processing device 910, the functions defined in the image processing method provided by the embodiment of the present disclosure can be implemented.

[0090] There are a few points to note:

[0091] (1) The drawings of the embodiments of the present disclosure only relate to the structures involved in the embodiments of the present disclosure. Other structures may refer to conventional designs.

[0092] (2) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to form new embodiments.

[0093] The above description is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: include: Get the template image; performing blurring processing on the template image to generate a plurality of processed template images with different blurring degrees; Get the image to be matched; respectively calculating the normalized correlation coefficient between each of the processed template images and the image to be matched, to obtain a set of normalized correlation coefficient matrices, wherein each normalized correlation coefficient matrix corresponds to one of the processed template images, and each value in each normalized correlation coefficient matrix corresponds to a pixel position; Dividing the maximum value of each normalized correlation coefficient matrix into corresponding maximum value groups; and Calculate the matching score of each of the maximum value groups separately, take the maximum value among the matching scores as the final value, and take the pixel position corresponding to the final value as the matching position; wherein, calculating the matching score of each of the maximum value groups separately includes: calculating the product of the maximum value of the values ​​in each of the maximum value groups and the number of values ​​in the maximum value group, to obtain the matching score of each of the maximum value groups.

2. The image processing method according to claim 1, wherein: Acquiring the template image includes: A clear and stable first image is acquired, and the template image is intercepted from the first image, wherein the template image includes a first specified characteristic pattern.

3. The image processing method according to claim 1 or 2, characterized in that: The step of performing blurring processing on the template image to generate the plurality of processed template images with different blurring degrees comprises: The template image is blurred using multiple blur kernels to obtain the multiple processed template images, wherein the blur kernel is used to simulate the blurring situation that occurs when the device acquires the image.

4. The image processing method according to claim 3, wherein: The multiple blur kernels include m Gaussian blur kernels and n motion blur kernels, m+n is greater than or equal to 1, and both m and n are natural numbers.

5. The image processing method according to claim 2, wherein: The image to be matched includes the first designated characteristic pattern.

6. The image processing method according to claim 4, wherein: Using the multiple blur kernels to blur the template image respectively includes: In the case where the image to be matched includes blur caused by motion, the motion blur kernel is selected to perform blur processing on the template image. In the case where the image to be matched includes blur caused by defocus, the Gaussian blur kernel is selected to perform blur processing on the template image. In a case where the image to be matched includes blur caused by motion and blur caused by defocus, the motion blur kernel and the Gaussian blur kernel are selected to perform blur processing on the template image.

7. The image processing method according to claim 1, wherein: Dividing the maximum value of each normalized correlation coefficient matrix into corresponding maximum value groups, including: Get the pixel position corresponding to each maximum value; The corresponding maximum values ​​are divided into corresponding maximum value groups according to the distances between the pixel positions.

8. The image processing method according to claim 7, wherein: Dividing the corresponding maximum values ​​into corresponding maximum value groups according to the distances between the pixel positions, including: Calculate the distance between the pixel positions corresponding to each two maximum values; and The division is performed based on the distance, wherein the distance between pixel positions corresponding to any two values ​​in each maximum value group is not greater than a preset threshold.

9. An image processing device, characterized in that include: processor; a memory storing executable instructions for the processor; The processor is configured to perform the steps of the image processing method according to any one of claims 1 to 8 by executing the executable instructions.

Images