Feature point matching method and device, terminal equipment and computer readable storage medium

Abstract

The application is suitable for the technical field of image processing, and provides a feature point matching method and device, terminal equipment and computer readable storage medium, the method comprises the following steps: acquiring a first to-be-processed image and a second to-be-processed image; extracting a plurality of first feature points from the first to-be-processed image; determining a second feature point corresponding to each first feature point from the second to-be-processed image according to a preset search strategy; and determining a plurality of target matching feature point pairs according to the plurality of first feature points and the plurality of second feature points. The above method effectively avoids the problem that feature points cannot be extracted in a weak texture scene, and can accurately and multiple match feature points in the weak texture scene, thereby avoiding the abnormal situation of scene correction.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a feature point matching method and device, a terminal device, and a computer readable storage medium. BACKGROUND

[0002] In image processing, obtaining a depth map requires line alignment correction of left and right disparity maps, and feature points of left and right views need to be matched during correction. Existing feature point matching algorithms such as the ORB algorithm (Oriented FAST and Rotated BRIEF algorithm), the scale-invariant feature transform algorithm (SIFT), and the speeded up robust features algorithm (SIFT) cannot accurately and abundantly match feature points in weak texture scenes during feature point matching, resulting in abnormal correction of some scenes. SUMMARY

[0003] Embodiments of the present application provide a feature point matching method and device, which can solve the problem that current feature point matching methods cannot accurately and abundantly match feature points in weak texture scenes, resulting in abnormal correction of some scenes.

[0004] In a first aspect, embodiments of the present application provide a feature point matching method, comprising:

[0005] obtaining a first to-be-processed image and a second to-be-processed image;

[0006] extracting a plurality of first feature points from the first to-be-processed image;

[0007] determining, according to a preset search strategy, a second feature point corresponding to each first feature point from the second to-be-processed image;

[0008] determining a plurality of target matching feature point pairs according to the plurality of first feature points and the plurality of second feature points.

[0009] In a second aspect, embodiments of the present application provide a feature point matching device, comprising:

[0010] a first obtaining unit configured to obtain a first to-be-processed image and a second to-be-processed image;

[0011] an extracting unit configured to extract a plurality of first feature points from the first to-be-processed image;

[0012] a first determining unit configured to determine, according to a preset search strategy, a second feature point corresponding to each first feature point from the second to-be-processed image;

[0013] The second determining unit is configured to determine a plurality of target matching feature point pairs according to the plurality of first feature points and the plurality of second feature points.

[0014] In a third aspect, an embodiment of the present application provides a terminal device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the feature point matching method according to the first aspect when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the feature point matching method according to the first aspect.

[0016] In the embodiment of the present application, the first to-be-processed image and the second to-be-processed image are acquired, a plurality of first feature points are extracted from the first to-be-processed image, a second feature point corresponding to each first feature point is determined from the second to-be-processed image according to a preset search strategy, and a plurality of target matching feature point pairs are determined according to the plurality of first feature points and the plurality of second feature points. According to the above method, the plurality of first feature points are extracted from the first to-be-processed image according to the preset uniform distribution extraction strategy, the problem that feature points cannot be extracted in a weak texture scene is effectively avoided, feature points can be accurately and abundantly matched in the weak texture scene, and the scene correction exception is avoided. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is a schematic flowchart of a feature point matching method provided by the first embodiment of the present application;

[0019] Figure 2 is a schematic flowchart of S101 refinement in the feature point matching method provided by the first embodiment of the present application;

[0020] Figure 3 is a schematic flowchart of S102 refinement in the feature point matching method provided by the first embodiment of the present application;

[0021] Figure 4 is a schematic flowchart of S103 refinement in the feature point matching method provided by the first embodiment of the present application;

[0022] Figure 5is a schematic flowchart of S104 in a feature point matching method provided by a first embodiment of the present application;

[0023] Figure 6 is a schematic diagram of a feature point matching apparatus provided by a second embodiment of the present application;

[0024] Figure 7 is a schematic diagram of a terminal device provided by a third embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, for the purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0026] It should be understood that the term "comprises" when used in this specification and the appended claims indicates the presence of the described features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0027] It should also be understood that the term "and / or" when used in this specification and the appended claims, such as in the phrases "A and / or B" and "A and / or B and / or C", means any combination of one or more of the associated listed items and can be interpreted as "one or more of A and B" or "one or more of A, B, and C".

[0028] As used in this specification and the appended claims, the term "if' can be construed to mean "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be construed to mean "upon determining" or "in response to determining" or "upon [the described condition or event] being detected" or "in response to [the described condition or event] being detected", depending on the context.

[0029] In addition, the terms "first", "second", "third", etc. are used herein only to distinguish one element from another, and do not imply or suggest a relative importance of the elements so designated.

[0030] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0031] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating a feature point matching method provided in the first embodiment of this application. In this embodiment, the execution subject of the feature point matching method is a device with feature point matching functionality, such as a desktop computer, server, etc. Figure 1 The feature point matching methods shown may include:

[0032] S101: Obtain the first image to be processed and the second image to be processed.

[0033] In this embodiment, the device acquires a first image to be processed and a second image to be processed. When calculating the depth map, it is necessary to align and correct the first and second initial images acquired by the two camera modules. Generally, a binocular camera is used to acquire two disparity maps, which serve as the first and second initial images.

[0034] The first image to be processed and the second image to be processed are determined by acquiring two disparity maps (left and right) using a binocular camera. In the first embodiment, the device can directly receive the first image to be processed and the second image to be processed sent by other devices. In the second embodiment, the device can acquire the first initial image and the second initial image using its built-in binocular camera, and then process the first initial image and the second initial image respectively to obtain the first image to be processed and the second image to be processed.

[0035] The second embodiment will be described in detail below. S101 may include S1011 to S1013, such as... Figure 2 As shown, S1011 to S1013 are as follows:

[0036] S1011: Obtain the first initial image and the second initial image.

[0037] The device can acquire a first initial image and a second initial image using the built-in binocular camera, or it can receive the first initial image and the second initial image acquired by an external binocular camera.

[0038] S1012: Perform grayscale processing on the first initial image to obtain the first image to be processed.

[0039] The device performs grayscale processing on the first initial image to obtain the first image to be processed.

[0040] Furthermore, to enhance feature point matching, the device can perform detail enhancement processing after grayscale processing. Specifically, the device can perform grayscale processing on the first initial image to obtain a first grayscale image. Then, detail enhancement processing is performed on the first grayscale image to obtain a first image to be processed. The core of image detail enhancement is to represent the original image as the sum of the base layer and the detail layer, and then enhance the detail layer separately to obtain the enhanced image. In this embodiment, the algorithm for detail enhancement processing is not limited.

[0041] In one implementation, a local histogram equalization algorithm can be used to enhance the details of the first grayscale image to obtain the first image to be processed.

[0042] Specifically, the device pre-stores a preset window radius, establishes a sliding window based on the preset window radius, traverses the first grayscale image using the sliding window, performs dynamic local histogram equalization on the selected area each time the sliding window slides, and calculates the local histogram of each selected area.

[0043] Specifically, the formula for calculating the local histogram (hist(i)) of the selected region is as follows:

[0044]

[0045] Where L is the maximum gray level of the selected area, count is the number of pixels in the selected area, and w and h are the width and height of the selected area.

[0046] Furthermore, when the probability of low-level grayscale values ​​is too high, it can lead to image distortion. To avoid local distortion, hist(i) can be used as the initial local histogram, and a threshold cropping process can be applied to it. Specifically, after obtaining the initial local histogram hist(i), the peak values ​​of hist(i) are cropped. A cropping threshold T is set in the device, and the local histogram Hist(i) of the selected region is calculated as follows:

[0047]

[0048] Where L is the maximum gray level of the selected area, hist(i) is the initial local histogram, and α is the clipping value.

[0049] After obtaining the local histogram Hist(i) of the selected region, the device performs mapping calculations based on the preset mapping formula and the local histograms of each selected region to obtain the first image to be processed.

[0050] The specific mapping formula is as follows:

[0051]

[0052] Where gray(x,y) is the gray value of the first grayscale image grayL at coordinates (x,y), Hist is the local histogram of the first grayscale image grayL, L is the maximum gray level, and gray_(x,y) is the gray value of the first grayscale image grayL at coordinates (x,y) after mapping processing. The first grayscale image grayL is processed according to the above formula to obtain the enhanced first image to be processed.

[0053] In addition, to reduce computation, the sliding window moves at a step size equal to the preset window radius when traversing the entire graph.

[0054] S1013: Perform grayscale processing on the second initial image to obtain the second image to be processed.

[0055] Steps S1012 and S1013 can be performed in parallel, or step S1013 can be executed first and then step S1012. No restrictions are imposed here.

[0056] In this embodiment, the device performs grayscale processing on the second initial image to obtain a second image to be processed. Specifically, the second initial image is grayscale processed to obtain a second grayscale image. The second grayscale image is then subjected to detail enhancement processing to obtain the second image to be processed.

[0057] For details on obtaining the second image to be processed from the second initial image in this embodiment, please refer to the relevant description in S1012 regarding obtaining the first image to be processed from the first initial image, which will not be repeated here.

[0058] S102: Extract multiple first feature points from the first image to be processed.

[0059] To avoid the high concentration of matched feature points in a single area, the device stores a preset uniform distribution extraction strategy. The device extracts multiple first feature points from the first image to be processed according to this strategy. In other words, the first feature points extracted from the first image to be processed by the device using the preset uniform distribution extraction strategy are uniformly distributed. This preset uniform distribution extraction strategy effectively optimizes the existing feature matching methods' inability to extract feature points in areas with weak texture.

[0060] Specifically, S102 can include S1021 to S1022, such as Figure 3 As shown, S1021 to S1022 are as follows:

[0061] S1021: Obtain the preset extraction interval according to the preset uniform distribution extraction strategy.

[0062] The device has a preset extraction interval, which can be set according to the size of the image being processed and the number of points to be extracted.

[0063] S1022: Extract a plurality of uniformly distributed first feature points from the first image to be processed according to a preset extraction interval; wherein the interval between adjacent first feature points is the preset extraction interval.

[0064] The device extracts multiple uniformly distributed first feature points from the first image to be processed according to a preset extraction interval; wherein the interval between adjacent first feature points is the preset extraction interval.

[0065] For example, if the preset extraction interval is set to 15 and the first feature point is Pt1, the specific formula for extracting the first feature point can be:

[0066] Pt1={(x,y),x mod 15=1, y mod 15=1}

[0067] Here, mod represents the remainder.

[0068] S103: Determine the second feature point corresponding to each first feature point from the second image to be processed according to the preset search strategy.

[0069] The device stores a preset search strategy. The device determines the second feature point corresponding to each first feature point from the second image to be processed according to the preset search strategy. In this embodiment, there is no restriction on the preset search strategy.

[0070] Specifically, to improve search accuracy, a multi-scale search approach can be adopted. S103 can include S1031 to S1032, such as... Figure 4 As shown, S1031 to S1032 are as follows:

[0071] S1031: Using the coordinates of each first feature point as the center, determine the search area corresponding to each first feature point in the second image to be processed according to the center and the preset search radius.

[0072] The device stores a preset search radius. Using the coordinates of the first feature point as the center, the device can determine the search area corresponding to the first feature point in the second image to be processed, based on the center and the preset search radius. This search area is relatively large, allowing the device to continue searching within it to determine the second feature point corresponding to the first feature point.

[0073] S1032: According to the preset search strategy, determine the second feature point corresponding to each first feature point from the search area corresponding to each first feature point.

[0074] The device determines a second feature point corresponding to each first feature point from the search area corresponding to each first feature point according to a preset search strategy. In this embodiment, the preset search strategy is not limited.

[0075] In one implementation, since the distance between the two camera modules is relatively small, a multi-scale sparse optical flow tracking method can be used to determine the second feature point corresponding to each first feature point. Specifically, the loss value corresponding to each pixel in the search region corresponding to each first feature point is calculated according to the sparse optical flow tracking algorithm; the pixel with the smallest loss value is determined as the second feature point corresponding to each first feature point. The specific calculation methods for the loss value and the second feature point are as follows:

[0076]

[0077] Where R is the preset search radius, Pt2[i](u,v) is the second feature point corresponding to the first feature point, grayL(x,y) is the point (x,y) in the first grayscale image, grayR(u+Δx,v+Δy) is the point (u+Δx,v+Δy) in the second grayscale image, i is the label of the first feature point to be matched, S[i] is the matching flag between the first feature point with label i and the optimal second feature point matched, and c t Let C be the loss threshold and C be the loss value. If the minimum loss value is greater than the loss threshold, the match is considered inaccurate.

[0078] S104: Determine several target matching feature point pairs based on multiple first feature points and multiple second feature points.

[0079] The device records the first feature point and its corresponding second feature point as a target matching feature point pair, and the device determines several target matching feature point pairs based on all the first feature points and their corresponding second feature points.

[0080] Among them, several target matching feature point pairs are used to calculate the rotation and vertical translation matrices between the left and right disparity maps. The translation matrices are used to correct the left and right disparity maps.

[0081] In one implementation, to further improve the accuracy of feature point matching, the matched feature point pairs can be filtered. S104 may include S1041 to S1044, such as... Figure 5 As shown, S1041 to S1044 are as follows:

[0082] S1041: Determine several initial matching feature point pairs based on multiple first feature points and multiple second feature points.

[0083] The device records multiple first feature points and their corresponding second feature points as an initial matching feature point pair, and obtains several initial matching feature point pairs.

[0084] S1042: Calculate the Euclidean distance, the first horizontal distance, and the second vertical distance between the first and second feature points in each initial matched feature point pair.

[0085] The device calculates the Euclidean distance, the first horizontal distance, and the second vertical distance between the first and second feature points in each initial matched feature point pair. The specific calculation formula is as follows:

[0086]

[0087] Where n is the total number of matching point pairs, (Pt1, Pt2) is the initial matching feature point pair, Pt1 is the first feature point, Pt2 is the second feature point, d is the Euclidean distance between the first and second feature points, dx is the first distance in the horizontal direction, and dy is the second distance in the vertical direction.

[0088] S1043: Calculate the matching score for each initial matching feature point pair based on the Euclidean distance, the first distance, and the second distance.

[0089] The device calculates the matching score of the initial matching feature point pairs based on Euclidean distance, first distance, and second distance. Specifically, it can iterate through all initial matching feature point pairs and score each initial matching feature point by calculating the correlation between the first and second distances between each pair.

[0090] The specific scoring formula is as follows:

[0091]

[0092] Where i = 1, 2...n, thresh is the empirical threshold, S[j] is the matching flag of the j-th feature point, n is the total number of matching point pairs, d is the Euclidean distance between the first and second feature points, dx is the first distance in the horizontal direction, and dy is the second distance in the vertical direction.

[0093] S1044: Initial matching feature point pairs with matching scores greater than or equal to a preset score threshold are determined as target matching feature point pairs.

[0094] The device has a preset score threshold. It identifies initial matching feature point pairs whose matching score Z is greater than or equal to the preset score threshold as target matching feature point pairs. The judgment logic formula is as follows:

[0095]

[0096] Where α is a constant value, is the threshold ratio coefficient, outliers indicates that the current initial matching feature point pair is an outlier and needs to be deleted, and inliers indicates that the current initial matching feature point pair is a normal value and can be marked as the target matching feature point pair.

[0097] The target matching feature point pairs obtained according to the method in this embodiment can be used for subsequent alignment correction calculations. Through the target matching feature point pairs, the rotation and vertical translation matrix of the right view relative to the left view is calculated. This matrix is ​​then used to correct the right view, thereby achieving row alignment correction of the left and right views.

[0098] In this embodiment, a first image to be processed and a second image to be processed are obtained; multiple first feature points are extracted from the first image to be processed; second feature points corresponding to each first feature point are determined from the second image to be processed according to a preset search strategy; and several target matching feature point pairs are determined based on the multiple first feature points and the multiple second feature points. The above method extracts multiple first feature points from the first image to be processed according to a preset uniform distribution extraction strategy, effectively avoiding the problem of feature point extraction in weak texture scenes. Even in weak texture scenes, it can accurately and extensively match feature points, avoiding abnormal scene correction.

[0099] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0100] Please see Figure 6 , Figure 6 This is a schematic diagram of the feature point matching device provided in the second embodiment of this application. The included units are used for performing... Figures 1 to 5 The steps in the corresponding embodiments. Please refer to the details. Figures 1 to 5The relevant descriptions in the corresponding embodiments are shown below. For ease of explanation, only the parts relevant to this embodiment are shown. See also... Figure 6 The feature point matching device 6 includes:

[0101] The first acquisition unit 610 is used to acquire a first image to be processed and a second image to be processed.

[0102] Extraction unit 620 is used to extract multiple first feature points from the first image to be processed;

[0103] The first determining unit 630 is used to determine the second feature point corresponding to each first feature point from the second image to be processed according to a preset search strategy;

[0104] The second determining unit 640 is used to determine several target matching feature point pairs based on multiple first feature points and multiple second feature points.

[0105] Furthermore, the first acquisition unit 610 includes:

[0106] The second acquisition unit is used to acquire the first initial image and the second initial image;

[0107] The first processing unit is configured to perform grayscale processing on the first initial image to obtain a first image to be processed, and to perform grayscale processing on the second initial image to obtain a second image to be processed.

[0108] Further, the first processing unit includes:

[0109] The second processing unit is used to perform grayscale processing on the first initial image to obtain a first grayscale image;

[0110] The third processing unit is used to perform detail enhancement processing on the first grayscale image to obtain the first image to be processed.

[0111] Furthermore, the third processing unit is specifically used for:

[0112] A sliding window is created based on a preset window radius;

[0113] The first grayscale image is traversed using a sliding window. Dynamic local histogram equalization is performed on the selected area each time the sliding window is slid, and the local histogram of each selected area is calculated.

[0114] The mapping calculation is performed based on the preset mapping formula and the local histogram of each selected region to obtain the first image to be processed.

[0115] Furthermore, the first processing unit is specifically used for:

[0116] The second initial image is converted to grayscale to obtain a second grayscale image;

[0117] The second grayscale image is subjected to detail enhancement processing to obtain the second image to be processed.

[0118] Furthermore, the extraction unit 620 is specifically used for:

[0119] The preset extraction interval is obtained according to the preset uniform distribution extraction strategy;

[0120] Multiple uniformly distributed first feature points are extracted from the first image to be processed according to a preset extraction interval; wherein the interval between adjacent first feature points is the preset extraction interval.

[0121] Furthermore, the first determining unit 630 includes:

[0122] The third determining unit is used to determine the search area corresponding to each first feature point in the second image to be processed, with the coordinates of each first feature point as the center and according to the center and a preset search radius.

[0123] The fourth determining unit is used to determine the second feature point corresponding to each first feature point from the search area corresponding to each first feature point according to a preset search strategy.

[0124] Furthermore, the fourth determining unit is specifically used for:

[0125] The loss value corresponding to each pixel in the search region corresponding to each first feature point is calculated based on the sparse optical flow tracing algorithm.

[0126] The pixel with the smallest loss value is determined as the second feature point corresponding to each first feature point.

[0127] Furthermore, the second determining unit 640 is specifically used for:

[0128] Several initial matching feature point pairs are determined based on multiple first feature points and multiple second feature points;

[0129] Calculate the Euclidean distance, the first horizontal distance, and the second vertical distance between the first and second feature points in each initial matched feature point pair;

[0130] The matching score for each initial matching feature point pair is calculated based on the Euclidean distance, the first distance, and the second distance.

[0131] Initial matching feature point pairs whose matching scores are greater than or equal to a preset score threshold are identified as target matching feature point pairs.

[0132] Furthermore, the first image to be processed and the second image to be processed are two disparity maps acquired by the binocular camera module; several target matching feature point pairs are used to calculate the rotation and vertical translation matrix between the two disparity maps, and the translation matrix is ​​used to correct the two disparity maps.

[0133] Figure 7 This is a schematic diagram of the terminal device provided in the third embodiment of this application. Figure 7 As shown, the terminal device 7 in this embodiment includes a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70, such as a feature point matching program. When the processor 70 executes the computer program 72, it implements the steps in the various feature point matching method embodiments described above, for example... Figure 1 Steps 101 to 104 are shown. Alternatively, when processor 70 executes computer program 72, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 6 The functions of modules 610 to 640 are shown.

[0134] For example, computer program 72 can be divided into one or more modules / units, one or more of which are stored in memory 71 and executed by processor 70 to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of computer program 72 in terminal device 7. For example, computer program 72 can be divided into a first acquisition unit, an extraction unit, a first determination unit, and a second determination unit, with the specific functions of each unit as follows:

[0135] The first acquisition unit is used to acquire the first image to be processed and the second image to be processed.

[0136] An extraction unit is used to extract multiple first feature points from a first image to be processed;

[0137] The first determining unit is used to determine the second feature point corresponding to each first feature point from the second image to be processed according to a preset search strategy;

[0138] The second determining unit is used to determine several target matching feature point pairs based on multiple first feature points and multiple second feature points.

[0139] The terminal device may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that... Figure 7 This is merely an example of terminal device 7 and does not constitute a limitation on terminal device 7. It may include more or fewer components than shown, or combine certain components, or different components. For example, the feature point matching device may also include input / output devices, network access devices, buses, etc.

[0140] The processor 70 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0141] The memory 71 can be an internal storage unit of the terminal device 7, such as a hard disk or RAM of the terminal device 7. The memory 71 can also be an external storage device of the terminal device 7, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal device 7. Furthermore, the terminal device 7 can include both internal storage units and external storage devices. The memory 71 is used to store computer programs and other programs and data required by the terminal device. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0142] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0143] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0144] This application also provides a timing device for a virtual timer, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on at least one processor. When the processor executes the computer program, it implements the steps in any of the above method embodiments.

[0145] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps described in the various method embodiments above.

[0146] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0147] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0148] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0149] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0150] In the embodiments provided in this application, it should be understood that the disclosed apparatus / device and method can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A feature point matching method for weakly textured scenes, characterized in that, include: Obtain the first image to be processed and the second image to be processed; Extract multiple first feature points from the first image to be processed; According to a preset search strategy, the second feature point corresponding to each first feature point is determined from the second image to be processed; Several target matching feature point pairs are determined based on the plurality of first feature points and the plurality of second feature points; The step of extracting multiple first feature points from the first image to be processed includes: obtaining a preset extraction interval according to a preset uniform distribution extraction strategy; and extracting multiple uniformly distributed first feature points from the first image to be processed according to the preset extraction interval; wherein the interval between adjacent first feature points is the preset extraction interval. The step of determining the second feature point corresponding to each first feature point from the second image to be processed according to the preset search strategy includes: taking the coordinates of each first feature point as the center, determining the search area corresponding to each first feature point in the second image to be processed according to the center and the preset search radius; and determining the second feature point corresponding to each first feature point from the search area corresponding to each first feature point according to the preset search strategy.

2. The method as described in claim 1, characterized in that, The acquisition of the first image to be processed and the second image to be processed includes: Obtain the first initial image and the second initial image; The first initial image is converted to grayscale to obtain a first image to be processed, and the second initial image is converted to grayscale to obtain a second image to be processed.

3. The method as described in claim 2, characterized in that, The step of converting the first initial image to grayscale to obtain the first image to be processed includes: The first initial image is converted to grayscale to obtain a first grayscale image; The first grayscale image is subjected to detail enhancement processing to obtain the first image to be processed.

4. The method as described in claim 3, characterized in that, The step of performing detail enhancement processing on the first grayscale image to obtain the first image to be processed includes: A sliding window is created based on a preset window radius; The first grayscale image is traversed using the sliding window. Dynamic local histogram equalization is performed on the selected area each time the sliding window is slid, and the local histogram of each selected area is calculated. The first image to be processed is obtained by performing mapping calculations based on the preset mapping formula and the local histograms of each selected region.

5. The method according to any one of claims 1-4, characterized in that, The step of determining the second feature point corresponding to each first feature point from the search region corresponding to each first feature point according to a preset search strategy includes: The loss value corresponding to each pixel in the search region corresponding to each first feature point is calculated according to the sparse optical flow tracking algorithm. The pixel with the smallest loss value is determined as the second feature point corresponding to each first feature point.

6. The method as described in claim 5, characterized in that, The step of determining several target matching feature point pairs based on the plurality of first feature points and the plurality of second feature points includes: Several initial matching feature point pairs are determined based on the plurality of first feature points and the plurality of second feature points; Calculate the Euclidean distance, the first horizontal distance, and the second vertical distance between the first and second feature points in each initial matched feature point pair; The matching score of each initial matching feature point pair is calculated based on the Euclidean distance, the first distance, and the second distance; The initial matching feature point pairs whose matching scores are greater than or equal to a preset score threshold are determined as target matching feature point pairs.

7. The method as described in claim 1, characterized in that, The first image to be processed and the second image to be processed are two disparity maps acquired by a binocular camera module; the plurality of target matching feature point pairs are used to calculate the rotation and vertical translation matrix between the two disparity maps, and the translation matrix is ​​used to correct the two disparity maps.

8. A feature point matching device for weakly textured scenes, characterized in that, include: The first acquisition unit is used to acquire the first image to be processed and the second image to be processed. An extraction unit is used to extract multiple first feature points from the first image to be processed; The first determining unit is used to determine the second feature point corresponding to each first feature point from the second image to be processed according to a preset search strategy; The second determining unit is used to determine several target matching feature point pairs based on the plurality of first feature points and the plurality of second feature points; The extraction unit is specifically configured to: obtain a preset extraction interval according to a preset uniform distribution extraction strategy; extract a plurality of uniformly distributed first feature points from the first image to be processed according to the preset extraction interval; wherein the interval between adjacent first feature points is the preset extraction interval; The first determining unit includes: The third determining unit is used to determine the search area corresponding to each first feature point in the second image to be processed, with the coordinates of each first feature point as the center of a circle, and according to the center of the circle and a preset search radius. The fourth determining unit is used to determine a second feature point corresponding to each first feature point from the search area corresponding to each first feature point according to a preset search strategy.

9. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images