Image processing method, device, storage medium and program product
By obtaining the displacement magnitude and orientation angle feature values of image matching point pairs and using the median absolute deviation to filter thresholds, the proportion of mismatched point pairs is reduced, solving the problem of numerous mismatched point pairs in image stitching and achieving efficient and accurate image stitching.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HEFEI IFLYTEK TOYCLOUD TECH
- Filing Date
- 2026-03-19
- Publication Date
- 2026-06-19
AI Technical Summary
Existing technologies suffer from a high rate of mismatched points during image stitching, resulting in poor image stitching accuracy and an inability to achieve effective stitching.
By matching feature points of the images to be stitched, the displacement magnitude and orientation angle of the pixel position vectors of the matching point pairs are obtained. The median absolute deviation is used to calculate the screening threshold, and the correct matching point pairs are selected to reduce the proportion of mismatched point pairs.
It improves the accuracy and efficiency of image stitching, reduces complex modeling and consistency verification calculations, and enables fast and efficient image stitching.
Smart Images
Figure CN122243734A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, storage medium and program product. Background Technology
[0002] Image stitching refers to the process of seamlessly stitching together two or more images with overlapping areas into a panoramic image with a wider field of view and higher resolution through geometric transformations (such as homography transformation) and registration.
[0003] Image stitching requires calculating the coordinate mapping model between the two images to be stitched. This requires obtaining a set of matching point pairs between the two images. However, the set of matching point pairs obtained by directly matching the feature points of the two images contains a certain proportion of mismatched point pairs (outliers). The coordinate mapping model constructed by directly using the set of matching point pairs has poor accuracy and cannot achieve effective image stitching. Therefore, it is necessary to filter the matching point pairs to reduce the proportion of mismatched point pairs. Summary of the Invention
[0004] In view of the above problems, this application provides an image processing method, apparatus, storage medium, and program product to reduce the proportion of mismatched point pairs. The specific solution is as follows:
[0005] The first aspect of this application provides an image processing method, comprising:
[0006] Feature point matching is performed on the first and second images to be stitched together to obtain an initial set of matching point pairs;
[0007] Obtain feature values of at least one dimension for each matching point pair, wherein the feature values of at least one dimension for each matching point pair include: the displacement magnitude and / or orientation angle of the pixel position vector of the matching point pair;
[0008] The filtering threshold for that dimension is obtained based on the median absolute deviation of the feature values of the same dimension for each matching point pair.
[0009] Matching pairs that satisfy the absolute deviation of each dimension being less than or equal to the filtering threshold of that dimension are added as correct matching pairs to the target matching pair set.
[0010] In one possible implementation, matching pairs that satisfy the filtering threshold for each dimension, where the absolute deviation is less than or equal to that dimension, are added to the target matching pair set, including:
[0011] Add matching point pairs whose absolute deviation in the first dimension is less than or equal to the filtering threshold in the first dimension to the candidate matching point pair set; the feature value of the first dimension of each matching point pair is the displacement magnitude of the pixel position vector of the matching point pair.
[0012] In the candidate matching point pair set, matching point pairs whose absolute deviation in the second dimension is less than or equal to the filtering threshold in the second dimension are added to the target matching point pair set; the feature value of the second dimension of each matching point pair is the orientation angle of the pixel position vector of the matching point pair.
[0013] In one possible implementation, the screening threshold for that dimension is obtained based on the median absolute deviation of the feature values of the same dimension for each pair of matching points, including:
[0014] Obtain the stitching scene to which the first image and the second image belong; the stitching scene is either a pure image scene or a scene containing text;
[0015] Multiply the median absolute deviation of each dimension by the adjustment coefficient of that dimension in the stitching scenario to obtain the screening threshold for that dimension.
[0016] In one possible implementation, the adjustment coefficient for the same dimension differs depending on the stitching scenario.
[0017] In one possible implementation, the difference in adjustment coefficients for each dimension differs depending on the splicing scenario.
[0018] In one possible implementation, obtaining the stitching scene to which the first image and the second image belong includes:
[0019] Read the splicing scenario configured by the user;
[0020] Alternatively, character region recognition can be performed on the first image and the second image. If the proportion of character regions in the first image and the second image is greater than a preset proportion, it is determined to be a scene containing text; otherwise, it is determined to be a pure image scene.
[0021] One possible implementation also includes:
[0022] Construct a six-degree-of-freedom affine transformation model based on the set of target matching point pairs;
[0023] The first image is subjected to an affine transformation using the six-degree-of-freedom affine transformation model to obtain the third image;
[0024] Search for the optimal seam in the overlapping area of the third image and the second image;
[0025] The third image and the second image are stitched together based on the optimal stitching seam to obtain the stitched image.
[0026] A second aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the image processing method described in the first aspect or any implementation thereof.
[0027] A third aspect of this application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:
[0028] The memory is used to store computer programs;
[0029] The processor is used to execute the computer program so that the electronic device can implement the image processing method of the first aspect or any implementation thereof.
[0030] A fourth aspect of this application provides a computer storage medium carrying one or more computer programs that, when executed by an electronic device, enable the electronic device to perform the image processing method described in the first aspect or any implementation thereof.
[0031] By employing the above technical solutions, the image processing method, apparatus, storage medium, and program products provided in this application perform feature point matching on the first and second images to be stitched, obtain an initial set of matching point pairs, and then obtain feature values for at least one dimension of each matching point pair. The feature values for at least one dimension of each matching point pair include the displacement magnitude and / or orientation angle of the pixel position vector of that matching point pair. A filtering threshold for that dimension is obtained based on the median absolute deviation of the feature values for the same dimension of each matching point pair. Matching point pairs that satisfy the condition that the absolute deviation of each dimension is less than or equal to the filtering threshold for that dimension are added as correct matching point pairs to the target matching point pair set. When filtering correct matching point pairs (i.e., the target matching point pair set) from the initial set of matching point pairs, this application filters correct matching point pairs based on the median absolute deviation of the displacement magnitude and / or the median absolute deviation of the orientation angle of the pixel position vector of each matching point. This method is simple to calculate, can quickly eliminate mismatched point pairs, reduce the proportion of mismatched point pairs, and thus improve the image stitching effect. Attached Figure Description
[0032] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0033] Figure 1 A flowchart illustrating one implementation of the image processing method provided in this application;
[0034] Figure 2 A flowchart illustrating an implementation of obtaining the screening threshold for each dimension based on the median absolute deviation of each dimension, as provided in this application;
[0035] Figure 3 An example of performing feature point matching on the first and second images to be stitched together, as provided in this application, to obtain an initial set of matching point pairs;
[0036] Figure 4 An example of the set of target matching point pairs obtained through filtering provided in this application;
[0037] Figure 5 Example image of the stitched image provided in this application;
[0038] Figure 6 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation
[0039] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.
[0040] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.
[0041] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.
[0042] Currently, in order to filter out correct matching point pairs from the set of matched point pairs and reduce the proportion of mismatched point pairs, some solutions have proposed image stitching methods based on the RANSAC algorithm. The processing steps may include:
[0043] Extract an initial set of matching point pairs from images A and B to be stitched together; in each matching point pair in the initial set, one pixel belongs to image A and the other pixel belongs to image B.
[0044] Random sampling: Randomly select the minimum number of matching point pairs required to calculate the 8-DOF homography transformation matrix from the initial set of matching point pairs (minimum 4 matching point pairs are required).
[0045] Modeling: Using these 4 randomly selected matching point pairs, a temporary 8-DOF homography transformation matrix is calculated.
[0046] Consistency check: This temporary 8-DOF homography transformation matrix is used to "test" each matching point pair in the initial set of matching point pairs. Specifically, for each matching point pair, the coordinates of the point in image B are mapped to the coordinates in image A using this temporary 8-DOF homography transformation matrix. Then, the distance between the mapped position and the actual position of the matching point in image A is calculated. If this distance is less than a preset "tolerance threshold" (e.g., 5 pixels), the matching point pair is considered to support the current temporary 8-DOF homography transformation matrix and is an "interior point" (correctly matched point pair); otherwise, it is an "outterior point" (incorrectly matched point pair).
[0047] Iteration: Repeat the above "random sampling-modeling-consistency check" process many times (e.g., 2000 times). Each time, a temporary 8-DOF homography transformation matrix and a set of "interior points" supporting it will be obtained.
[0048] Selection: After the iteration is completed, select the 8-DOF homography transformation matrix generated by the iteration with the most "interior points". At the same time, all "interior points" that support this 8-DOF homography transformation matrix are the filtered, clean and reliable "correct matching points".
[0049] Reconstruction: Using the finally selected "correct matching points", an 8-DOF homography transformation matrix is constructed, and images A and B are stitched together based on this 8-DOF homography transformation matrix.
[0050] As can be seen, the above process requires multiple iterations of "random sampling-modeling-consistency check" to screen out the correct matching point pairs. This requires multiple executions of complex calculations such as modeling and consistency checks, resulting in low efficiency in screening matching point pairs and thus low efficiency in image stitching.
[0051] This application is proposed to quickly remove mismatched point pairs from the set of matched point pairs obtained through matching.
[0052] like Figure 1 The diagram shown is a flowchart of one implementation of the image processing method provided in this application, which may include:
[0053] Step S101: Perform feature point matching on the first and second images to be stitched together to obtain an initial set of matching point pairs.
[0054] Optionally, the AKAZE algorithm can be used to extract features from the first image to obtain a first set of feature points; the AKAZE algorithm can also be used to extract features from the second image to obtain a second set of feature points. While preserving edge details, the AKAZE algorithm can achieve more accurate feature localization than the traditional SIFT / SURF algorithm, providing high-quality raw data for subsequent affine transformation model building.
[0055] The first and second feature point sets are matched using a matcher to obtain an initial set of matched point pairs. The matching process can be found in existing solutions and will not be elaborated here.
[0056] In the initial set of matching point pairs, each matching point pair includes two feature points: a pixel in the first image (which can be denoted as the first pixel) and a pixel in the second image that matches the first pixel (which can be denoted as the second pixel). Each matching point pair can record the coordinates of at least the two matching pixels.
[0057] Step S102: Obtain feature values of at least one dimension for each matching point pair. The feature values of at least one dimension for each matching point pair include: the displacement magnitude and / or orientation angle of the pixel position vector of the matching point pair.
[0058] For ease of description and distinction, for the k-th (k=1, 2, 3, ..., n; n is the number of matching point pairs in the initial matching point pair set) matching point pair The displacement magnitude of the pixel position vector of the kth matching point pair can be denoted as the first dimension feature value of the kth matching point pair, and the orientation angle of the pixel position vector of the kth matching point pair can be denoted as the second dimension feature value of the kth matching point pair.
[0059] Suppose that the k-th matching point pair contains pixels in the first image. The coordinates are (x, y), and the pixel point in the second image. The coordinates are ( ).
[0060] Then the pixel position vector of the kth matching point pair for:
[0061] (1)
[0062] , (2)
[0063] The displacement magnitude of the pixel position vector of the k-th matching point pair for: The displacement modulus is essentially the distance between two pixels. Therefore, the set of n displacement moduli can be called the distance set L.
[0064] The direction angle of the pixel position vector of the k-th matching point pair for: The set of n direction angles can be called the angle set. .
[0065] Step S103: Obtain the screening threshold for that dimension based on the median absolute deviation of the feature values of the same dimension for each matching point pair.
[0066] The median absolute deviation for each dimension is the median of the absolute values of the differences between each feature value and the median for that dimension. The specific calculation process may include:
[0067] Calculate the median of the feature values in the same dimension for each matching point pair to obtain the feature median for each dimension.
[0068] Specifically, the median of the first-dimensional feature values of the n matching point pairs is calculated, which is equivalent to calculating the median of all distances in the distance set L. The resulting first-dimensional feature median (also known as the distance median) is denoted as: .
[0069] Calculate the median of the eigenvalues of the second dimension for n matching point pairs, i.e., calculate the angle set. The median of all angles is used to obtain the feature median of the second dimension (also called the angle median), denoted as: .
[0070] Calculate the median of the absolute deviations of the feature values and feature medians for each matching pair in the same dimension to obtain the median absolute deviations for each dimension.
[0071] Among them, the median absolute deviation of the first dimension for: .
[0072] absolute deviation of the median in the second dimension for: .
[0073] This represents the absolute deviation between the eigenvalues of the first dimension and the median of the eigenvalues, i.e., the absolute deviation between the displacement modulus and the median distance for each pair of matched points. This represents the determination deviation between the eigenvalues and the median of the second dimension, i.e., the absolute deviation between the orientation angle and the median angle for each pair of matched points.
[0074] The screening threshold for each dimension is obtained based on the median absolute deviation of each dimension.
[0075] Optionally, the median absolute deviation of each dimension can be multiplied by a preset adjustment coefficient to obtain the screening threshold for that dimension. The adjustment coefficients for different dimensions can be the same or different.
[0076] The adjustment coefficient for each dimension is greater than 1, meaning that the screening threshold for the first dimension is greater than the median absolute deviation of the first dimension; and the screening threshold for the second dimension is greater than the median decision deviation of the second dimension.
[0077] As an example, the range of adjustment coefficients for each dimension can be, but is not limited to, [1, 10].
[0078] Step S104: Add the matching point pairs that satisfy the absolute deviation of each dimension being less than or equal to the filtering threshold of that dimension as correct matching point pairs to the target matching point pair set.
[0079] Matching point pairs that meet the following conditions will be considered as correct matching point pairs and added to the target matching point pair set:
[0080] ,and .
[0081] in, The adjustment coefficient for the first dimension. This is the adjustment coefficient for the second dimension.
[0082] The target matching point pair set is used to stitch the first image and the second image together.
[0083] The image processing method provided in this application performs feature point matching on the first and second images to be stitched together to obtain an initial set of matching point pairs. When selecting correct matching point pairs (i.e., the target set of matching point pairs), the correct matching point pairs are selected based on the median absolute deviation of the displacement magnitude and / or the median absolute deviation of the orientation angle of the pixel position vector of each matching point. The calculation is simple and does not require complex operations such as modeling and consistency verification, nor does iterative calculation. It can quickly eliminate mismatched point pairs, reduce the proportion of mismatched point pairs, and thus improve the image stitching effect.
[0084] Once the feature value of one dimension of each matching point pair is obtained, the matching point pairs in the initial matching point pair set whose absolute deviation in that dimension is less than or equal to the filtering threshold of that dimension can be directly added to the target matching point pair set.
[0085] In an optional embodiment, given the feature values of each matching point pair in two dimensions, the above implementation of adding matching point pairs that satisfy the condition that the absolute deviation of each dimension is less than or equal to the filtering threshold of that dimension to the target matching point pair set can be any of the following three methods:
[0086] Method 1:
[0087] Add matching pairs from the initial set of matching pairs whose absolute deviation in the first dimension is less than or equal to the filtering threshold of the first dimension to the first candidate set of matching pairs; add matching pairs from the initial set of matching pairs whose absolute deviation in the second dimension is less than or equal to the filtering threshold of the second dimension to the second candidate set of matching pairs; and determine the intersection of the first and second candidate sets of matching pairs as the target set of matching pairs.
[0088] In other words, one approach is to simultaneously filter matching point pairs whose absolute deviation is less than or equal to the filtering threshold of each dimension; then, add the common matching point pairs from the candidate matching point pairs filtered by both dimensions to the target matching point pair set.
[0089] Method 2:
[0090] Add matching point pairs from the initial set whose absolute deviation in the first dimension is less than or equal to the filtering threshold of the first dimension to the candidate set of matching point pairs; the feature value of the first dimension of each matching point pair is the displacement magnitude of the pixel position vector of that matching point pair. Add matching point pairs from the candidate set whose absolute deviation in the second dimension is less than or equal to the filtering threshold of the second dimension to the target set of matching point pairs; the feature value of the second dimension of each matching point pair is the orientation angle of the pixel position vector of that matching point pair.
[0091] In other words, the second method first performs a coarse screening based on the distance between matching point pairs to obtain a set of candidate matching point pairs, and then performs a fine screening based on the direction angle of the matching point pairs to obtain a set of target matching point pairs.
[0092] Method 3:
[0093] Add matching pairs from the initial set of matching pairs whose absolute deviation in the second dimension is less than or equal to the filtering threshold of the second dimension to the candidate set of matching pairs; add matching pairs from the candidate set whose absolute deviation in the first dimension is less than or equal to the filtering threshold of the first dimension to the target set of matching pairs.
[0094] In other words, Method 3 first performs a coarse screening based on the direction angle of the matching point pairs to obtain a set of candidate matching point pairs, and then performs a fine screening based on the distance between the matching point pairs to obtain a set of target matching point pairs.
[0095] Methods 2 and 3 provide a matching point pair screening method that utilizes the dual statistical features (i.e., median absolute deviation) of the displacement magnitude and orientation angle of the pixel position vector of the matching point pair for "cascaded deterministic filtering", which is more efficient than Method 1.
[0096] The inventors of this application have discovered that when the first and second images to be stitched contain text, the filtering efficiency of method two is higher than that of other methods. Based on this, method two can be preferred to determine the target matching point pair set.
[0097] The process of determining the target matching point pair set described above in this application does not require random sampling like the RANSAC algorithm, and achieves fast and effective deterministic removal of outliers (i.e. mismatched point pairs).
[0098] In an optional embodiment, the flowchart for obtaining the screening threshold for each dimension based on the median absolute deviation of each dimension is shown below. Figure 2 As shown, it may include:
[0099] Step S201: Obtain the stitching scene to which the first image and the second image belong. The stitching scene is either a pure image scene (i.e., neither the first image nor the second image contains text or contains very little text) or a scene containing text (i.e., the first image and the second image contain a lot of text).
[0100] Optionally, the stitching scene can be configured by the user, that is, the user can specify the stitching scene based on the first and second images to be stitched.
[0101] Optionally, character region recognition can be performed on the first image and the second image respectively. If the proportion of character regions in the first image and the second image is greater than the preset proportion (i.e., the first image and the second image contain a lot of text), it is determined to be a scene containing text; otherwise (i.e., the first image and the second image do not contain text or contain very little text), it is determined to be a pure image scene.
[0102] The percentage of character regions in the first image and the second image refers to the ratio of the total area of the character regions in the first image and the second image to the total area of the first image and the second image.
[0103] Alternatively, the proportion of character regions in the first image and the second image refers to the larger of a first proportion of character regions in the first image and a second proportion of character regions in the second image. The first proportion is the ratio of the area of the character regions in the first image to the total area of the first image; the second proportion is the ratio of the area of the character regions in the second image to the total area of the second image.
[0104] Alternatively, the percentage of character regions in the first image and the second image refers to the average of the first percentage of character regions in the first image and the second percentage of character regions in the second image.
[0105] Alternatively, the proportion of character regions in the first image and the second image includes: a first proportion of character regions in the first image and a second proportion of character regions in the second image. Based on this, the proportion of character regions in the first image and the second image being greater than a preset proportion includes: the first proportion being greater than the preset proportion, and the second proportion being greater than the preset proportion.
[0106] Step S202: Multiply the absolute deviation of the median of each dimension by the adjustment coefficient of that dimension in the above splicing scenario to obtain the screening threshold of that dimension.
[0107] This application determines the screening thresholds for each dimension based on the stitching scenario, which improves the robustness of the image processing method, can adapt to diverse scenarios, and avoids the "one-size-fits-all" defect of fixed thresholds.
[0108] Optionally, the adjustment coefficient for the same dimension may vary depending on the splicing scenario.
[0109] As an example, the adjustment coefficient for each dimension in a pure image scenario is smaller than the adjustment coefficient for the same dimension in a scenario containing text. In other words, the adjustment coefficient for the first dimension in a pure image scenario is smaller than the adjustment coefficient for the first dimension in a scenario containing text; the adjustment coefficient for the second dimension in a pure image scenario is smaller than the adjustment coefficient for the second dimension in a scenario containing text.
[0110] Optionally, the difference in adjustment coefficients for each dimension may vary depending on the splicing scenario.
[0111] As an example, the difference in adjustment coefficients across dimensions in a pure image scenario is greater than the difference in adjustment coefficients across dimensions in a scenario containing text. In other words, the absolute difference between the adjustment coefficients of the first and second dimensions in a pure image scenario is greater than the absolute difference between the adjustment coefficients of the first and second dimensions in a scenario containing text.
[0112] In an optional embodiment, after obtaining the target matching point pair set, the first image and the second image can be stitched together based on the target matching point pair set, which may specifically include:
[0113] A six-degree-of-freedom affine transformation model is constructed based on the set of target matching point pairs. The six-degree-of-freedom affine transformation model (i.e., the six-degree-of-freedom affine transformation matrix) can be solved using the least squares method with the set of target matching point pairs. The inventors of this application have found that, compared to the 8-degree-of-freedom homography transformation matrix, the six-degree-of-freedom affine transformation model strongly constrains the parallelism of images, exhibiting extremely strong robustness in distant scenes, aerial photography, and approximately translational scenes, effectively preventing excessive stretching and distortion of images.
[0114] The first image is transformed using a six-degree-of-freedom affine transformation model to obtain the third image. The coordinates of each pixel in the third image are in the same coordinate system as the coordinates of each pixel in the second image.
[0115] The optimal stitching seam is searched in the overlapping region of the third and second images. An energy cost function for the overlapping region of the third and second images can be defined. Based on this energy cost function, dynamic programming is used to solve for the minimum energy path (i.e., the optimal stitching seam). The specific implementation can adopt existing solutions, which will not be detailed here.
[0116] The third and second images are stitched together based on the optimal stitching seam to obtain the stitched image. Existing solutions can be used for the specific implementation, which will not be elaborated here.
[0117] To better illustrate the effect of this application on filtering the initial set of matching points, two images are used as examples below.
[0118] like Figure 3 The image shown is an example of performing feature point matching on a first image and a second image to be stitched together, according to an embodiment of this application, to obtain an initial set of matching point pairs. In this example, the upper image is the first image, and the lower image is the second image. In a matching point pair, pixels located in the upper image are marked with blue dots, and pixels located in the lower image are marked with pink dots. The two pixels in the same matching point pair are connected by a red line.
[0119] like Figure 4 The image shown is an example of the target matching point pair set obtained by filtering according to an embodiment of this application. It can be seen that, based on the image processing method of this application, a large number of mismatched points are eliminated, while matching point pairs of text regions in the feature point set are retained.
[0120] like Figure 5 The image shown is an example of a stitched image provided in an embodiment of this application. That is, based on the image processing method of this application... Figure 3 The result of stitching the two images shown.
[0121] This application achieves efficient point-to-point purification and seamless visual stitching while ensuring the robustness of the model (affine transformation model) and preventing perspective distortion.
[0122] Corresponding to the method embodiments, this application also provides an electronic device. (See reference...) Figure 6 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of this application. The electronic device in these embodiments can be a terminal device (e.g., an in-vehicle infotainment system, a large-screen device, a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc.) or a server (which can be a single server, a server cluster, or a cloud server, etc.). Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.
[0123] like Figure 6 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 408 into a random access memory (RAM) 603. When the electronic device is powered on, the RAM 603 also stores various programs and data required for the operation of the electronic device. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0124] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, memory cards, hard drives, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.
[0125] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the image processing methods provided in this application.
[0126] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the image processing methods provided in this application.
[0127] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.
[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0129] In the above embodiments, the functionality can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented entirely or partially as a computer program product. Those skilled in the art can use different methods to implement the described functions for each specific solution, but such implementation should not be considered beyond the scope of this application.
[0130] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).
[0131] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0132] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. An image processing method, characterized in that, include: Feature point matching is performed on the first and second images to be stitched together to obtain an initial set of matching point pairs; Obtain feature values of at least one dimension for each matching point pair, wherein the feature values of at least one dimension for each matching point pair include: the displacement magnitude and / or orientation angle of the pixel position vector of the matching point pair; The filtering threshold for that dimension is obtained based on the median absolute deviation of the feature values of the same dimension for each matching point pair. Matching pairs that satisfy the absolute deviation of each dimension being less than or equal to the filtering threshold of that dimension are added as correct matching pairs to the target matching pair set.
2. The method according to claim 1, characterized in that, Add matching pairs that satisfy the filter threshold for each dimension, where the absolute deviation is less than or equal to that dimension, to the target matching pair set, including: Add matching point pairs whose absolute deviation in the first dimension is less than or equal to the filtering threshold in the first dimension to the candidate matching point pair set; the feature value of the first dimension of each matching point pair is the displacement magnitude of the pixel position vector of the matching point pair. In the candidate matching point pair set, matching point pairs whose absolute deviation in the second dimension is less than or equal to the filtering threshold in the second dimension are added to the target matching point pair set; the feature value of the second dimension of each matching point pair is the orientation angle of the pixel position vector of the matching point pair.
3. The method according to claim 1, characterized in that, The filtering threshold for that dimension is obtained based on the median absolute deviation of the feature values of each matching point pair in the same dimension, including: Obtain the stitching scene to which the first image and the second image belong; the stitching scene is either a pure image scene or a scene containing text; Multiply the median absolute deviation of each dimension by the adjustment coefficient of that dimension in the stitching scenario to obtain the screening threshold for that dimension.
4. The method according to claim 3, characterized in that, The adjustment coefficient for the same dimension varies depending on the splicing scenario.
5. The method according to claim 3, characterized in that, The difference in adjustment coefficients for each dimension varies depending on the splicing scenario.
6. The method according to claim 3, characterized in that, Obtaining the stitching scene to which the first image and the second image belong includes: Read the splicing scenario configured by the user; Alternatively, character region recognition can be performed on the first image and the second image. If the proportion of character regions in the first image and the second image is greater than a preset proportion, it is determined to be a scene containing text; otherwise, it is determined to be a pure image scene.
7. The method according to claim 1, characterized in that, Also includes: Construct a six-degree-of-freedom affine transformation model based on the set of target matching point pairs; The first image is subjected to an affine transformation using the six-degree-of-freedom affine transformation model to obtain the third image; Search for the optimal seam in the overlapping area of the third image and the second image; The third image and the second image are stitched together based on the optimal stitching seam to obtain the stitched image.
8. An electronic device, characterized in that, The electronic device includes at least one processor and a memory connected to the processor; wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the image processing method as described in any one of claims 1 to 7.
9. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the image processing method as described in any one of claims 1 to 7.
10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the image processing method as described in any one of claims 1 to 7.