Target matching method, device, equipment and storage medium

Abstract

The present application discloses a target matching method and apparatus, device and storage medium. The target matching method comprises: obtaining a first image and a second image, wherein the first image and the second image are obtained by photographing a target to be matched by different shooting components; obtaining the disparity between the target to be matched in the first image and the second image based on a first distance between the target to be matched and at least one shooting component; and determining the matching feature point pairs of the target to be matched in the first image and the second image according to the disparity. The above scheme can improve the accuracy of target matching between different images.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a target matching method and apparatus, device and storage medium. Background Art

[0002] Object matching in images is a basic problem in computer vision. The accuracy of object matching will affect the operations after object matching. The common method of object matching in images is to use the two closest feature points in the two images to be matched as matching point pairs. Taking face matching as an example, the main thing is to determine the offset of the face in the two images. If the vertical coordinates of the face in the two images are consistent and the horizontal difference is the smallest, then it is considered a matching face. The problem with this method is that when there are multiple faces in the image, the smallest position difference between the two images is used as the matching condition during the object matching process, that is, the difference in the coordinates of the face in the two images is the smallest, but the two faces with the smallest position difference in the two images may not be the faces that should be matched, which makes it easier to have the problem of wrong matching. Summary of the invention

[0003] The present application at least provides a target matching method and apparatus, device and storage medium.

[0004] The present application provides a target matching method, including: acquiring a first image and a second image, wherein the first image and the second image are obtained by photographing a target to be matched by different shooting components; obtaining a disparity of the target to be matched between the first image and the second image based on a first distance between the target to be matched and at least one shooting component; and determining, according to the disparity, matching feature point pairs about the target to be matched in the first image and the second image.

[0005] Therefore, the parallax of the target to be matched between the first image and the second image is obtained by the first distance between the target to be matched and at least one shooting component, and then the matching feature point pair of the target to be matched is determined according to the parallax to achieve target matching between the first image and the second image. Compared with the method of directly using the nearest feature points in the two images as the matching point pairs, the present application refers to the distance between the target and the camera component to obtain a more accurate parallax, and then uses the parallax to obtain the matching point pairs, thereby improving the accuracy of the matching feature point pairs of the target to be matched between different images, that is, improving the accuracy of target matching between different images.

[0006] Wherein, according to the disparity, the matching feature point pairs about the target to be matched in the first image and the second image are determined, including: based on the disparity, determining the reference position corresponding to the first feature point in the second image, wherein the first feature point is obtained by performing target feature point detection on the first image; based on the reference position, selecting the second feature point matching the first feature point in the second image, wherein the second feature point is obtained by performing target feature point detection on the second image.

[0007] Therefore, the reference position corresponding to the first feature point in the second image is determined by parallax, and the second feature point is determined based on the reference position, which reduces the area in the second image that matches the first feature point, thereby quickly finding the matching point pair and improving the matching accuracy.

[0008] Among them, according to the disparity, determining the matching feature point pairs about the target to be matched in the first image and the second image, also includes: determining the antipodal line corresponding to the first feature point in the second image; based on the reference position, selecting the second feature point matching the first feature point in the second image, including: determining a candidate area including the reference position in the second image, and in the candidate area, selecting the second feature point whose positional relationship with the antipodal line meets the preset requirements as the second feature point matching the first feature point.

[0009] Therefore, the second feature point that matches the first feature point is determined by combining the two dimensions of disparity and epipolar geometry constraint, which improves the accuracy of target matching compared to using a single dimension.

[0010] Wherein, determining the epipolar line corresponding to the first feature point in the second image includes: obtaining the epipolar line using the coordinates of the first feature point and the basic matrix; and / or, a preset requirement is that the distance between the second feature point and the epipolar line in the candidate area is minimum.

[0011] Therefore, according to the principle of epipolar geometry constraints, the second feature point corresponding to the first feature point should be on the epipolar line, but errors are inevitable in the target matching process, resulting in the second feature point not being on the epipolar line. Therefore, selecting the feature point with the smallest distance to the epipolar line in the candidate area as the second feature point corresponding to the first feature point can improve the success rate of target matching.

[0012] Among them, the first image or the second image is a reference image, the shooting component corresponding to the reference image is a reference shooting component, and the first distance is the distance between the target to be matched and the reference shooting component; the method also includes the following steps to obtain the first distance: determining a first size of a target area corresponding to the target to be matched in the reference image; and obtaining the first distance by using a ratio between the first size and a preset size.

[0013] Therefore, by using the ratio between the first size of the target area corresponding to the target to be matched in the reference image and the preset size to determine the first distance, the technical effect of determining the distance between the target to be matched and the shooting component using a single image is achieved, breaking the traditional cognition that at least two images need to be combined to obtain the distance between the target to be matched and the shooting.

[0014] Among them, the first size includes a first width and a first height of the target area, and the preset size includes a preset width and a preset height; using the ratio between the first size and the preset size to obtain the first distance, including: obtaining a first ratio between the first width and the preset width, and a second ratio between the first height and the preset height; obtaining the second distance between the target to be matched and the reference shooting component based on the first ratio, and obtaining the third distance between the target to be matched and the reference shooting component based on the second ratio; obtaining the first distance based on the second distance and the third distance.

[0015] Therefore, by performing a weighted summation on the second distance and the third distance, the results in the width and height directions can be comprehensively considered, thereby improving the accuracy of the first distance compared to using the second distance or the third distance obtained in a single direction as the first distance.

[0016] Among them, obtaining the second distance between the target to be matched and the reference shooting component based on the first ratio includes: multiplying the first ratio by the first focal length of the reference shooting component in the width direction to obtain the second distance; obtaining the third distance between the target to be matched and the reference shooting component based on the second ratio includes: multiplying the second ratio by the second focal length of the reference shooting component in the height direction to obtain the third distance; obtaining the first distance based on the second distance and the third distance includes: performing weighted summation of the second distance and the third distance to obtain the first distance.

[0017] Therefore, because the resolution of the reference shooting component in the width or height direction may be different, the second distance and the third distance obtained by multiplying the first ratio by the first focal length of the reference shooting component in the width direction and multiplying the second ratio by the first focal length of the reference shooting component in the height direction are more accurate.

[0018] Among them, determining the first size of the target area corresponding to the target to be matched in the reference image includes: obtaining the first size based on the coordinates of the first feature point obtained by detecting the target feature points of the reference image; or obtaining the first size based on the size of the area obtained by detecting the target area of ​​the reference image.

[0019] Therefore, the size of the target area is obtained according to the coordinates of the first feature point or according to the size of the area obtained by monitoring the target area of ​​the first image. The process is simple and the computing power required by the device is low.

[0020] Among them, the first distance is the distance between the target to be matched and the reference shooting component, and the reference shooting component is the shooting component of the first image or the second image; based on the first distance between the target to be matched and at least one shooting component, the disparity of the target to be matched between the first image and the second image is obtained, including: based on the first distance, the focal length of the reference shooting component, and the baseline between the shooting components of the first image and the second image, the disparity is obtained.

[0021] Therefore, the parallax between the matching point pairs is obtained by using the first distance, the focal length of the reference shooting component, and the baseline between the shooting components of the first image and the second image. The whole process is simple and requires low computing power, thereby saving processing resources of the execution device.

[0022] The present application provides a target matching device, including: an image acquisition module, used to acquire a first image and a second image, wherein the first image and the second image are obtained by photographing a target to be matched by different shooting components; a disparity acquisition module, used to obtain the disparity of the target to be matched between the first image and the second image based on a first distance between the target to be matched and at least one shooting component; and a matching module, used to determine, based on the disparity, a pair of matching feature points about the target to be matched in the first image and the second image.

[0023] The present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the target matching method in the first aspect above.

[0024] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the target matching method in the first aspect when the program instructions are executed by a processor.

[0025] The above scheme obtains the disparity of the target to be matched between the first image and the second image through the first distance between the target to be matched and at least one shooting component, and then determines the matching feature point pair of the target to be matched according to the disparity to achieve target matching between the first image and the second image. Compared with the method of directly taking the nearest feature points in the two images as the matching point pairs, the present application refers to the distance between the target and the camera component to obtain a more accurate disparity, and then uses the disparity to obtain the matching point pairs, thereby improving the accuracy of the matching feature point pairs of the target to be matched between different images, that is, improving the accuracy of target matching between different images.

[0026] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0028] Figure 1 This is a flow chart of an embodiment of the target matching method of the present application. Figure 1 ;

[0029] Figure 2 This is a flow chart of an embodiment of the target matching method of the present application. Figure 2 ;

[0030] Figure 3 It is a structural schematic diagram of an embodiment of a target matching device of the present application;

[0031] Figure 4 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0032] Figure 5 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0033] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0034] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0035] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0036] The present application provides some object matching methods and devices. The object matching is to match the corresponding feature points belonging to the same object in the first image and the second image, that is, to achieve the matching of the object and then find the same object in the first image and the second image. The object can be any object, such as a face, a building, a vehicle, etc.

[0037] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the target matching method of the present application. Figure 1.

[0038] Specifically, the target matching method may include the following steps:

[0039] Step S11: Acquire a first image and a second image, wherein the first image and the second image are acquired by photographing the target to be matched by different photographing components.

[0040] Among them, the shooting components for shooting the first image and the second image can be placed arbitrarily, as long as the imaging planes of the first image and the second image are parallel, for example, the two shooting components can be placed side by side in the horizontal direction, or in the vertical direction, etc. The two shooting components can form a binocular camera system. The embodiment of the present disclosure takes the two shooting components placed side by side in the horizontal direction as an example. Among them, the device that performs target matching and the shooting component can be an integrated device or independent devices. The integrated device means that it can be controlled by the same processor, and the independent device means that it is controlled by different processors. Among them, the first image and the second image can be images that have not been processed or images that have been processed. The image processing method can be to adjust the brightness, resolution, etc. Further, the modalities of the first image and the second image can be the same or different. For example, the first image and the second image can be visible light images or infrared light images at the same time, or one of them is a visible light image and the other is an infrared light image. The form of the first image and the second image is not specifically specified here.

[0041] The first image and the second image both include at least one target, and one of the targets is used as a target to be matched, so as to match feature points of the target to be matched in the first image and the second image, and then search for the target to be matched in the first image and the second image to achieve target matching. The disclosed embodiment takes the target to be matched as a face as an example.

[0042] Step S12: based on a first distance between the target to be matched and at least one photographing component, obtaining a parallax of the target to be matched between the first image and the second image.

[0043] The first distance between the target to be matched and at least one shooting component may be the first distance to one of the shooting components or the first distance between the two shooting components. The first distance to the two shooting components may specifically be the distance to the midpoint of the baseline of the two shooting components. Because the distances to the shooting components are different, the parallax between the matching feature point pairs of the target to be matched in the captured image is also different. Specifically, the farther the distance between the target to be matched and the shooting component, the smaller the parallax. Similarly, the closer the distance between the target to be matched and the shooting component, the larger the parallax. Therefore, the size of the parallax can be determined according to the first distance between the target to be matched and the shooting component.

[0044] Step S13: determining matching feature point pairs in the first image and the second image regarding the target to be matched according to the disparity.

[0045] The method of determining the matching feature point pair of the target to be matched in the first image and the second image according to the disparity may be to determine a feature point in one of the images, and then determine the matching point corresponding to the feature point in the other image according to the disparity, thereby obtaining the matching feature point pair of the target to be matched. If the target to be matched is a human face, the feature points of the target to be matched in the first image and the second image may be obtained by performing feature detection on the area containing the human face in the image.

[0046] The above scheme obtains the disparity of the target to be matched between the first image and the second image through the first distance between the target to be matched and at least one shooting component, and then determines the matching feature point pair of the target to be matched according to the disparity to achieve target matching between the first image and the second image. Compared with the method of directly taking the nearest feature points in the two images as the matching point pairs, the present application refers to the distance between the target and the camera component to obtain a more accurate disparity, and then uses the disparity to obtain the matching point pairs, thereby improving the accuracy of the matching feature point pairs of the target to be matched between different images, that is, improving the accuracy of target matching between different images.

[0047] In some disclosed embodiments, the first image or the second image is a reference image, the shooting component corresponding to the reference image is a reference shooting component, and the first distance is the distance between the target to be matched and the reference shooting component. The first distance may be obtained by:

[0048] First, determine the first size of the target area corresponding to the target to be matched in the reference image. Optionally, the first size includes a first width and a first height of the target area. The target area may be an area corresponding to a specific part of the target to be matched in the reference image. For example, if the target to be matched is a human face, the target area may be an area corresponding to eyebrows, eyes, nose, and mouth in the reference image. There are many ways to specifically determine the first size of the target area corresponding to the target to be matched in the reference image. The first way is to obtain the first size based on the coordinates of the first feature point obtained by detecting the target feature points of the reference image. The embodiment of the present disclosure takes the reference image as the first image as an example. For example, by inputting the first image and the second image into a first target detection deep neural network, the feature point coordinate information (X l1 ,Y l1 ),…(X ln ,Y ln ) and the feature point coordinate information (X r1 ,Y r1 ),…(X rn,Y rn ). The first width can be the difference between the maximum value and the minimum value of all width coordinates, which can be expressed as the first width w=max(X l1, …X ln )-min(X l1, …X ln ), the first height h=max(Y l1, …Y ln )-min(Y l1, …Y ln ). By acquiring the feature point coordinate information of another image at the same time as acquiring the feature point coordinates of the reference image, the target matching process is simplified. The second is to obtain the first size based on the size of the area obtained by detecting the target area of ​​the reference image. Among them, the area obtained by detecting the target area can be the area contained in the target detection frame, and the area contained in the target detection frame is a specific part of the target to be matched. Optionally, the width of the target detection frame is used as the first width, and the height of the target detection frame is used as the first height. For example, taking the target to be matched as a face as an example, the target detection frame is a face detection frame, wherein the area contained in the face detection frame is the area jointly formed by the eyebrows, eyes, nose and mouth. For example, the reference image is input into the second target detection deep neural network, and the face detection frame is output, wherein the width of the face detection frame is used as the first width, and the height of the face detection frame is used as the first height. By obtaining the size of the target area according to the first feature point coordinates or according to the size of the area obtained by monitoring the target area of ​​the first image, the process is simple and the computing power required by the device is low.

[0049] Secondly, the first distance is obtained by using the ratio between the first size and the preset size. The preset size includes a preset width and a preset height. The ratio between the first size and the preset size may be the ratio between the widths or the heights, or the ratio between the widths may be further combined with the ratio between the heights. The first distance may be obtained by performing a perspective transformation using the first size. By determining the first distance using the ratio between the first size of the target area corresponding to the target in the reference image and the preset size of the target to be matched, the technical effect of determining the distance between the target to be matched and the shooting component using a single image is achieved, breaking the traditional perception that at least two images need to be combined to obtain the distance between the target to be matched and the shooting component.

[0050] Specifically, the first distance can be obtained by using the ratio between the first size and the preset size, by obtaining a first ratio between the first width and the preset width, and a second ratio between the first height and the preset height. The second distance between the target to be matched and the reference shooting component is obtained based on the first ratio, and the third distance between the target to be matched and the reference shooting component is obtained based on the second ratio. The preset size here is the real size of other targets with the same attribute as the target to be matched. For example, when the target to be matched is a human face, the preset size is the real size of other preset human faces. The first size and the preset size are both sizes corresponding to preset parts of the target to be matched. For example, the preset size corresponds to the real size of an area composed of eyebrows, eyes, nose and mouth, and the first size also corresponds to the size of a target area composed of eyebrows, eyes, nose and mouth in the reference image.

[0051] The second distance may be obtained by multiplying the first ratio by the first focal length of the reference camera assembly in width to obtain the second distance. Specifically, the formula for obtaining the second distance D2 is D2=W1f x / w, where W1 is the preset width, w is the first width, and f x is the first focal length of the reference camera assembly in the width direction, where different camera assemblies have different focal lengths in the width direction. x It is obtained by calibrating two reference shooting components, mainly by converting the actual focal length of the reference shooting component to obtain the first focal length in the width direction and the second focal length in the height direction. The definition of width and height here is consistent with the definition method of width and height of the reference image. The way to obtain the third distance can be to multiply the second ratio by the second focal length of the reference shooting component in the height direction to obtain the third distance. Specifically, the formula for obtaining the third distance D3 can be D3 = H1f y / h, where H1 refers to the preset altitude, f y It refers to the second focal length of the reference shooting component in the height direction, wherein different shooting components have different focal lengths in the width direction, and h refers to the first height.

[0052] Finally, based on the second distance and the third distance, the first distance is obtained. Specifically, the second distance and the third distance are weighted and summed to obtain the first distance. Among them, the formula of weighted summation can be D1=λD2+(1-λ)D3, wherein D1 refers to the first distance, D2 refers to the second distance, D3 refers to the third distance, λ is a preset parameter, and 0≤λ≤1, for example, λ is taken as 0.5. In the disclosed embodiment, λ is a fixed parameter, and the value of λ is mainly related to the resolution of the reference image. If the resolution of the reference shooting component is higher, λ is closer to 0.5. However, in other disclosed embodiments, the value of λ can also be adjusted dynamically. Generally, if the absolute value of the difference between the second distance and the third distance obtained is less than the first difference, λ is taken as 0.5; if the absolute value of the difference between the second distance and the third distance is greater than the first difference, λ is taken as greater than 0.5 or less than 0.5, wherein if the second distance is greater than the third distance, λ is taken as greater than 0.5, and if the second distance is less than the third distance, λ is taken as less than 0.5. And the greater the absolute value of the difference between the two, the further away the value of λ is from 0.5.

[0053] By performing a weighted summation of the second distance and the third distance, the results in the width and height directions can be comprehensively considered, and the accuracy of the first distance can be improved compared to using the second distance or the third distance obtained in a single direction as the first distance.

[0054] Furthermore, because the resolution of the reference shooting component in the width or height direction may be different, the second distance and the third distance obtained by multiplying the first ratio by the first focal length of the reference shooting component in the width direction and multiplying the second ratio by the first focal length of the reference shooting component in the height direction are more accurate.

[0055] In some disclosed embodiments, the first distance is the distance between the target to be matched and the reference shooting component, and the reference shooting is the shooting component of the first image or the second image. Wherein, based on the first distance between the target to be matched and the shooting component, the parallax of the target to be matched between the first image and the second image is obtained, including:

[0056] Based on the first distance, the focal length of the reference shooting component, and the baseline between the shooting components of the first image and the second image, the parallax is obtained. Specifically, the formula for obtaining the parallax Δd is as follows: Δd=f x B / D1, where Δd is the parallax, f xis the focal length of the reference shooting component in the width direction, D1 is the first distance, and B is the baseline length. The length of the baseline can be obtained by calibrating the two shooting components. The parallax between the matching point pairs is obtained by the first distance, the focal length of the reference shooting component, and the baseline between the shooting components of the first image and the second image. The whole process is simple and requires low computing power, thereby saving processing resources of the execution device.

[0057] The step of determining the matching feature point pair of the target to be matched in the first image and the second image according to the parallax includes: determining the reference position corresponding to the first feature point in the second image based on the parallax. The first feature point is obtained by detecting the target feature point in the first image. Assume that the coordinates of the first feature point in the first image are (x ln ,y ln ), where the parallax is Δd, then the reference position corresponding to the first feature point in the second image should be (x ln +Δd,y ln +Δd). That is to say, the second feature point corresponding to the first feature point in the second image should be near the reference position. Then, based on the reference position, a second feature point matching the first feature point is selected in the second image. The second feature point is obtained by detecting the target feature point of the second image. Among them, the selection method may be to select the feature point in the second image that is closest to the reference position as the second feature point matching the first feature point. The distance here may be the Euclidean distance. The reference position corresponding to the first feature point in the second image is determined by parallax, and then the second feature point is determined according to the reference position, which reduces the area in the second image that matches the first feature point, thereby quickly finding the matching point pair and improving the matching accuracy. On the other hand, precisely because the area in the second image that matches the first feature point is reduced, the matching speed is greatly improved, and the power consumption of the device that executes the target matching method is reduced.

[0058] In some disclosed embodiments, the step of determining the pair of matching feature points in the first image and the second image about the target to be matched based on the parallax also includes determining the epipolar line corresponding to the first feature point in the second image. The epipolar line corresponding to the first feature point in the second image may be determined by using the coordinates of the first feature point and the basic matrix to obtain the epipolar line. The basic matrix is ​​obtained by calibrating the two shooting components. The specific process of calibrating the two camera components is before step S11. The formula for obtaining the epipolar line l may be: l = (x ln ,y ln , 1)*F, where (x ln ,y ln , 1) is the homogeneous coordinates of the first feature point, and F is the basic matrix.

[0059] Among them, based on the reference position, the process of selecting the second feature point matching the first feature point in the second image can specifically be to determine a candidate area including the reference position in the second image, and select the second feature point whose positional relationship with the epipolar line meets the preset requirements in the candidate area. Among them, the candidate area can be an area of ​​a preset candidate size centered on the reference position. The determination of the preset candidate size can be set based on experience and is not specifically specified here. By comprehensively determining the second feature point matching the first feature point in two dimensions of disparity and epipolar geometric constraints, the accuracy of target matching is improved compared to using a single dimension. In addition, by selecting a feature point whose positional relationship with the epipolar line meets the preset requirements as the second feature point matching the first feature point, it is equivalent to reducing the feature matching area from two dimensions to one dimension, thereby reducing the influence of the placement position between the two shooting components on the target matching. And by performing target matching in two dimensions of disparity and epipolar geometric constraints, even if there are multiple faces in the first image and the second image, it is not easy to have an erroneous match.

[0060] The preset requirement may be that the distance between the second feature point and the epipolar line in the candidate area is the smallest. That is, the feature point with the smallest distance to the epipolar line in the candidate area is selected as the second feature point that matches the first feature point. The distance here may be the Euclidean distance. According to the principle of epipolar geometry constraints, the second feature point corresponding to the first feature point should be on the epipolar line, but errors are inevitable in the target matching process, resulting in the second feature point not being on the epipolar line. Therefore, selecting the feature point with the smallest distance to the epipolar line in the candidate area as the second feature point corresponding to the first feature point can improve the success rate of target matching.

[0061] In the above scheme, even when there are multiple interfering targets in the first image and the second image, by obtaining the distance between the target to be matched and the shooting component and obtaining the parallax, the feature point pair about the target to be matched is determined according to the parallax, thereby reducing the occurrence of erroneous matching of the feature points of the target to be matched with other interfering targets, thereby improving the accuracy of the extraction results of the matching feature points of the target to be matched when there are many interfering targets.

[0062] To better understand the technical solution proposed in the embodiment of the present disclosure, please refer to the following example.

[0063] See also Figure 2 , Figure 2 This is a flow chart of an embodiment of the target matching method of the present application. Figure 2 .like Figure 2 As shown, the target matching method proposed in the embodiment of the present disclosure can be divided into the following steps:

[0064] Step S11: Acquire a first image and a second image, wherein the first image and the second image are acquired by photographing the target to be matched by different photographing components.

[0065] The specific method of obtaining the first image and the second image can be found above and will not be repeated here.

[0066] Step S121: obtaining a first distance between the target to be matched and the reference shooting component.

[0067] The reference shooting component may be the shooting component corresponding to the first image, or the shooting component corresponding to the second image. The specific method of obtaining the first distance between the target to be matched and the reference shooting component is as described above, and will not be repeated here.

[0068] Step S122: Based on the first distance, obtaining the disparity of the target to be matched between the first image and the second image.

[0069] The method of obtaining the disparity is as described above and will not be described again here.

[0070] Step S131: determining a reference position corresponding to a first feature point in the second image based on the parallax, wherein the first feature point is obtained by performing target feature point detection on the first image.

[0071] The method of determining the reference position corresponding to the first feature point in the second image based on the parallax is as described above and will not be repeated here.

[0072] Step S132: determining the epipolar line corresponding to the first feature point in the second image.

[0073] Specifically, the method for determining the epipolar line corresponding to the first feature point in the second image is as described above, and will not be repeated here. In the example given in the embodiment of the present disclosure, step S131 and step S132 are steps performed simultaneously. Of course, in other embodiments, one of them can be executed first, and then the other. The execution order of step S131 and step S132 is not specifically specified here.

[0074] Step S133: determining a candidate region including the reference position in the second image, and selecting, in the candidate region, a second feature point whose positional relationship with the epipolar line meets a preset requirement as the second feature point matching the first feature point.

[0075] Among them, the preset requirements are as mentioned above, and the process of selecting the second feature point matching the first feature point in the candidate area is as mentioned above, which will not be repeated here.

[0076] The above scheme improves the accuracy of target matching by comprehensively determining the second feature point that matches the first feature point in terms of the two dimensions of parallax and epipolar geometric constraints, compared to using a single dimension. In addition, by selecting a feature point whose positional relationship with the epipolar line meets the preset requirements as the second feature point that matches the first feature point, it is equivalent to reducing the feature matching area from two dimensions to one dimension, thereby reducing the influence of the placement position between the two shooting components on the target matching. In addition, by performing target matching in the two dimensions of parallax and epipolar geometric constraints, it is less likely to cause false matching even when there are multiple faces in the first image and the second image.

[0077] After obtaining the matching feature point pairs of the target to be matched through the solution proposed in the embodiment of the present disclosure, the extracted matching feature point pairs of the target to be matched can be further utilized, for example, using the matching feature point pairs to perform depth acquisition, liveness detection, body temperature measurement and other operations. For example, in a multi-person temperature measurement scenario, it is necessary to locate the same face captured by different cameras in a binocular camera system, and then obtain the position of the face in the thermal imager based on the face coordinates, and finally perform temperature extraction. Therefore, face matching in a multi-person scenario is a prerequisite for rapid body temperature measurement of multiple people. Without the solution provided in the embodiment of the present disclosure, a high-precision and high-robustness face matching result can be obtained to assist in body temperature measurement in multi-person scenarios.

[0078] The above scheme obtains the disparity of the target to be matched between the first image and the second image through the first distance between the target to be matched and at least one shooting component, and then determines the matching feature point pair of the target to be matched according to the disparity to achieve target matching between the first image and the second image. Compared with the method of directly taking the nearest feature points in the two images as the matching point pairs, the present application refers to the distance between the target and the camera component to obtain a more accurate disparity, and then uses the disparity to obtain the matching point pairs, thereby improving the accuracy of the matching feature point pairs of the target to be matched between different images, that is, improving the accuracy of target matching between different images.

[0079] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0080] The target matching method may be executed by a target matching device, for example, the target matching method may be executed by a terminal device or a server or other processing device, wherein the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, and an autonomous vehicle with requirements for visual positioning, three-dimensional reconstruction, image registration, etc., a robot with positioning and mapping requirements, a medical imaging system with registration requirements, glasses, helmets, and other products for augmented reality or virtual reality, etc. In some possible implementations, the target matching method may be implemented by a processor calling computer-readable instructions stored in a memory.

[0081] See also Figure 3 , Figure 3 It is a structural schematic diagram of an embodiment of the target matching device of the present application. The target matching device 30 includes an image acquisition module 31, a disparity acquisition module 32, and a matching module 33. The image acquisition module 31 is used to acquire a first image and a second image, wherein the first image and the second image are obtained by photographing the target to be matched by different shooting components; the disparity acquisition module 32 is used to obtain the disparity between the first image and the second image of the target to be matched based on the first distance between the target to be matched and at least one shooting component; the matching module 33 is used to determine the matching feature point pairs of the target to be matched in the first image and the second image according to the disparity.

[0082] The above scheme obtains the disparity of the target to be matched between the first image and the second image through the first distance between the target to be matched and at least one shooting component, and then determines the matching feature point pair of the target to be matched according to the disparity to achieve target matching between the first image and the second image. Compared with the method of directly taking the nearest feature points in the two images as the matching point pairs, the present application refers to the distance between the target and the camera component to obtain a more accurate disparity, and then uses the disparity to obtain the matching point pairs, thereby improving the accuracy of the matching feature point pairs of the target to be matched between different images, that is, improving the accuracy of target matching between different images.

[0083] In some disclosed embodiments, the matching module 33 determines matching feature point pairs regarding the target to be matched in the first image and the second image based on the disparity, including: determining a reference position corresponding to the first feature point in the second image based on the disparity, wherein the first feature point is obtained by performing target feature point detection on the first image; and selecting a second feature point matching the first feature point in the second image based on the reference position, wherein the second feature point is obtained by performing target feature point detection on the second image.

[0084] The above scheme determines the reference position corresponding to the first feature point in the second image through parallax, and then determines the second feature point based on the reference position, which reduces the area in the second image that matches the first feature point, thereby quickly finding the matching point pair and improving the matching accuracy.

[0085] In some disclosed embodiments, the matching module 33 determines, based on the disparity, matching feature point pairs regarding the target to be matched in the first image and the second image, and also includes: determining the epipolar line corresponding to the first feature point in the second image; based on the reference position, selecting a second feature point in the second image that matches the first feature point, including: determining a candidate area including the reference position in the second image, and in the candidate area, selecting a second feature point whose positional relationship with the epipolar line meets preset requirements as the second feature point that matches the first feature point.

[0086] The above solution combines the two dimensions of parallax and epipolar geometry constraint to determine the second feature point that matches the first feature point, thereby improving the accuracy of target matching compared to using a single dimension.

[0087] In some disclosed embodiments, the matching module 33 determines the epipolar line corresponding to the first feature point in the second image, including: using the coordinates of the first feature point and the basic matrix to obtain the epipolar line; and / or, the preset requirement is that the distance between the second feature point and the epipolar line in the candidate area is minimized.

[0088] In the above scheme, according to the principle of epipolar geometry constraint, the second feature point corresponding to the first feature point should be on the epipolar line. However, errors are inevitable in the target matching process, resulting in the second feature point not being on the epipolar line. Therefore, the feature point with the smallest distance to the epipolar line in the candidate area is selected as the second feature point corresponding to the first feature point, which can improve the success rate of target matching.

[0089] In some disclosed embodiments, the first image or the second image is a reference image, the shooting component corresponding to the reference image is a reference shooting component, and the first distance is the distance between the target to be matched and the reference shooting component; the disparity acquisition module 32 is also used to perform the following steps to obtain the first distance: determine a first size of a target area corresponding to the target to be matched in the reference image; and obtain the first distance using a ratio between the first size and a preset size.

[0090] The above scheme determines the first distance by using the ratio between the first size of the target area corresponding to the target in the reference image and the preset size, thereby achieving the technical effect of determining the distance between the target to be matched and the shooting component using a single image, breaking the traditional cognition that at least two images need to be combined to obtain the distance between the target to be matched and the shooting.

[0091] In some disclosed embodiments, the first size includes a first width and a first height of the target area, and the preset size includes a preset width and a preset height; the disparity acquisition module 32 uses the ratio between the first size and the preset size to obtain a first distance, including: obtaining a first ratio between the first width and the preset width, and a second ratio between the first height and the preset height; obtaining a second distance between the target to be matched and the reference shooting component based on the first ratio, and obtaining a third distance between the target to be matched and the reference shooting component based on the second ratio; obtaining the first distance based on the second distance and the third distance.

[0092] The above scheme can comprehensively consider the results in the width and height directions by performing weighted summation of the second distance and the third distance, and can improve the accuracy of the first distance compared to using the second distance or the third distance obtained in a single direction as the first distance.

[0093] In some disclosed embodiments, the disparity acquisition module 32 obtains a second distance between the target to be matched and the reference shooting component based on the first ratio, including: multiplying the first ratio by the first focal length of the reference shooting component in the width direction to obtain the second distance; obtains a third distance between the target to be matched and the reference shooting component based on the second ratio, including: multiplying the second ratio by the second focal length of the reference shooting component in the height direction to obtain the third distance; obtains the first distance based on the second distance and the third distance, including: performing weighted summation of the second distance and the third distance to obtain the first distance.

[0094] In the above scheme, because the resolution of the reference shooting component in the width or height direction may be different, the second distance and the third distance obtained by multiplying the first ratio by the first focal length of the reference shooting component in the width direction and multiplying the second ratio by the first focal length of the reference shooting component in the height direction are more accurate.

[0095] In some disclosed embodiments, the disparity acquisition module 32 determines a first size of a target area corresponding to the target to be matched in the reference image, including: obtaining the first size based on the coordinates of a first feature point obtained by detecting target feature points on the reference image; or obtaining the first size based on the size of an area obtained by detecting the target area on the reference image.

[0096] In the above scheme, the size of the target area is obtained according to the coordinates of the first feature point or according to the size of the area obtained by monitoring the target area of ​​the first image. The process is simple and the computing power required by the device is low.

[0097] In some disclosed embodiments, the first distance is the distance between the target to be matched and a reference capturing component, and the reference capturing component is a capturing component of the first image or the second image; the disparity acquisition module 32 obtains the disparity of the target to be matched between the first image and the second image based on the first distance between the target to be matched and at least one capturing component, including: obtaining the disparity based on the first distance, the focal length of the reference capturing component, and the baseline between the capturing components of the first image and the second image.

[0098] The above scheme obtains the disparity between the matching point pairs through the first distance, the focal length of the reference shooting component, and the baseline between the shooting components of the first image and the second image. The whole process is simple and requires low computing power, thereby saving processing resources of the execution device.

[0099] The above scheme obtains the disparity of the target to be matched between the first image and the second image through the first distance between the target to be matched and at least one shooting component, and then determines the matching feature point pair of the target to be matched according to the disparity to achieve target matching between the first image and the second image. Compared with the method of directly taking the nearest feature points in the two images as the matching point pairs, the present application refers to the distance between the target and the camera component to obtain a more accurate disparity, and then uses the disparity to obtain the matching point pairs, thereby improving the accuracy of the matching feature point pairs of the target to be matched between different images, that is, improving the accuracy of target matching between different images.

[0100] See also Figure 4 , Figure 4 4 is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 40 includes a memory 41 and a processor 42, and the processor 42 is used to execute program instructions stored in the memory 41 to implement the steps in the above target matching method embodiment. In a specific implementation scenario, the electronic device 40 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 40 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0101] Specifically, the processor 42 is used to control itself and the memory 41 to implement the steps in the above-mentioned target matching method embodiment. The processor 42 can also be called a CPU (Central Processing Unit). The processor 42 may be an integrated circuit chip with signal processing capabilities. The processor 42 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 42 can be implemented by an integrated circuit chip.

[0102] The above scheme obtains the disparity of the target to be matched between the first image and the second image through the first distance between the target to be matched and at least one shooting component, and then determines the matching feature point pair of the target to be matched according to the disparity to achieve target matching between the first image and the second image. Compared with the method of directly taking the nearest feature points in the two images as the matching point pairs, the present application refers to the distance between the target and the camera component to obtain a more accurate disparity, and then uses the disparity to obtain the matching point pairs, thereby improving the accuracy of the matching feature point pairs of the target to be matched between different images, that is, improving the accuracy of target matching between different images.

[0103] See also Figure 5 , Figure 5 The computer-readable storage medium 50 stores program instructions 501 that can be executed by a processor, and the program instructions 501 are used to implement the steps in the above target matching method embodiment.

[0104] The above scheme obtains the disparity of the target to be matched between the first image and the second image through the first distance between the target to be matched and at least one shooting component, and then determines the matching feature point pair of the target to be matched according to the disparity to achieve target matching between the first image and the second image. Compared with the method of directly taking the nearest feature points in the two images as the matching point pairs, the present application refers to the distance between the target and the camera component to obtain a more accurate disparity, and then uses the disparity to obtain the matching point pairs, thereby improving the accuracy of the matching feature point pairs of the target to be matched between different images, that is, improving the accuracy of target matching between different images.

[0105] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0106] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0107] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0108] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0109] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. A target matching method, characterized in that: include: Acquire a first image and a second image, wherein the first image and the second image are obtained by photographing the matching target by different photographing components; Based on a first distance between the target to be matched and at least one of the photographing components, obtaining a disparity of the target to be matched between the first image and the second image, wherein when the first distance between the target to be matched and at least one of the photographing components is different, the disparity of the target to be matched between the first image and the second image is different; Determine, according to the disparity, a pair of matching feature points in the first image and the second image regarding the target to be matched; The first image or the second image is a reference image, the shooting component corresponding to the reference image is a reference shooting component, and the first distance is the distance between the target to be matched and the reference shooting component; the method further comprises the following steps to obtain the first distance: determining a first size of a target area corresponding to the target to be matched in the reference image; obtaining the first distance by using a ratio between the first size and a preset size, wherein the first size includes a first width and a first height of the target area, and the preset size includes a preset width and a preset height; The obtaining of the first distance by using the ratio between the first size and the preset size includes: obtaining a first ratio between the first width and the preset width, and a second ratio between the first height and the preset height; obtaining a second distance between the target to be matched and the reference shooting component based on the first ratio, and obtaining a third distance between the target to be matched and the reference shooting component based on the second ratio; obtaining a first product between the second distance and a first weight, and a second product between the third distance and a second weight, and adding the first product to the second product to obtain the first distance, wherein the sum of the first weight and the second weight is 1; In which, in response to the absolute value of the difference between the second distance and the third distance being greater than the first difference, and the second distance being greater than the third distance, it is determined that the first weight is greater than 0.5; in response to the absolute value of the difference between the second distance and the third distance being greater than the first difference, and the second distance being less than the third distance, it is determined that the first weight is less than 0.5; in which, the greater the absolute value of the difference between the second distance and the third distance, the further the first weight is from 0.

5.

2. The method according to claim 1, characterized in that The step of determining, according to the disparity, a pair of matching feature points in the first image and the second image regarding the target to be matched comprises: Based on the disparity, determining a reference position corresponding to a first feature point in the second image, wherein the first feature point is obtained by performing target feature point detection on the first image; Based on the reference position, a second feature point matching the first feature point is selected in the second image, wherein the second feature point is obtained by performing target feature point detection on the second image.

3. The method according to claim 2, characterized in that The determining, according to the disparity, a pair of matching feature points in the first image and the second image about the target to be matched further includes: determining an epipolar line corresponding to the first feature point in the second image; The selecting, based on the reference position, a second feature point matching the first feature point in the second image comprises: A candidate region including the reference position is determined in the second image, and a second feature point whose positional relationship with the epipolar line meets a preset requirement is selected in the candidate region as a second feature point matching the first feature point.

4. The method according to claim 3, characterized in that The determining the epipolar line corresponding to the first feature point in the second image includes: Obtaining the epipolar line using the coordinates of the first feature point and a fundamental matrix; And / or, the preset requirement is that the distance between the second feature point and the epipolar line in the candidate area is minimum.

5. The method according to any one of claims 1 to 4, characterized in that: The obtaining a second distance between the target to be matched and the reference shooting component based on the first ratio includes: Multiplying the first ratio by a first focal length of the reference shooting component in a width direction to obtain the second distance; The obtaining a third distance between the target to be matched and the reference shooting component based on the second ratio includes: The second ratio is multiplied by the second focal length of the reference shooting component in the height direction to obtain a third distance.

6. The method according to any one of claims 1 to 4, characterized in that: The determining a first size of a target area corresponding to the target to be matched in the reference image includes: Obtaining the first size based on coordinates of a first feature point obtained by performing target feature point detection on the reference image; Alternatively, the first size is obtained based on the size of the area obtained by performing target area detection on the reference image.

7. The method according to any one of claims 1 to 4, characterized in that: The first distance is the distance between the target to be matched and a reference shooting component, and the reference shooting component is a shooting component of the first image or the second image; The obtaining, based on a first distance between the target to be matched and at least one of the photographing components, a disparity of the target to be matched between the first image and the second image comprises: The disparity is obtained based on the first distance, the focal length of the reference photographing component, and a baseline between photographing components of the first image and the second image.

8. A target matching device, characterized in that: include: An image acquisition module, used to acquire a first image and a second image, wherein the first image and the second image are obtained by photographing the matching target by different photographing components; a disparity acquisition module, configured to obtain a disparity of the target to be matched between the first image and the second image based on a first distance between the target to be matched and at least one of the photographing components, wherein when the first distance between the target to be matched and at least one of the photographing components is different, the disparity of the target to be matched between the first image and the second image is different; A matching module, configured to determine, according to the disparity, a pair of matching feature points in the first image and the second image regarding the target to be matched; The first image or the second image is a reference image, the shooting component corresponding to the reference image is a reference shooting component, and the first distance is the distance between the target to be matched and the reference shooting component; the disparity acquisition module is further used to perform the following steps to obtain the first distance: determine a first size of a target area corresponding to the target to be matched in the reference image; obtain the first distance by using a ratio between the first size and a preset size, the first size includes a first width and a first height of the target area, and the preset size includes a preset width and a preset height; The obtaining of the first distance by using the ratio between the first size and the preset size includes: obtaining a first ratio between the first width and the preset width, and a second ratio between the first height and the preset height; obtaining a second distance between the target to be matched and the reference shooting component based on the first ratio, and obtaining a third distance between the target to be matched and the reference shooting component based on the second ratio; obtaining a first product between the second distance and a first weight, and a second product between the third distance and a second weight, and adding the first product to the second product to obtain the first distance, wherein the sum of the first weight and the second weight is 1; In which, in response to the absolute value of the difference between the second distance and the third distance being greater than the first difference, and the second distance being greater than the third distance, it is determined that the first weight is greater than 0.5; in response to the absolute value of the difference between the second distance and the third distance being greater than the first difference, and the second distance being less than the third distance, it is determined that the first weight is less than 0.5; in which, the greater the absolute value of the difference between the second distance and the third distance, the further the first weight is from 0.

5.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images