Image mapping method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses an image mapping method and device, electronic equipment and a storage medium, and belongs to the technical field of image processing, and the method comprises the steps: determining a first target feature point from a plurality of first feature points of a first image based on a to-be-mapped region of the first image, the distance between the first target feature point and the central position of the to-be-mapped area is smaller than a threshold value; based on the first target feature point, determining a second target feature point from a plurality of second feature points of a second image, the second target feature point corresponding to the first target feature point; and mapping the to-be-mapped region to the second image based on the plurality of first target feature points and the plurality of second target feature points. According to the invention, the image mapping accuracy can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of image processing, including but not limited to an image mapping method and apparatus, an electronic device, and a storage medium. Background Art

[0002] With the rapid development of computer vision and image processing technologies, image mapping technology has also made great progress. In this way, two images can be mapped into one image, or a partial area in one image can be mapped into another image.

[0003] In related technologies, generally, all feature points in two images can be obtained by extracting features from the two images respectively. Then, the feature points in the two images are matched, and the random sample consensus algorithm is used to eliminate the mismatched point pairs. Then, based on the matched point pairs between the two images, the two images are registered and mapped to obtain the mapped image.

[0004] However, in the scenario of local mapping, this solution may cause the problem that the mapped image is offset. Therefore, the accuracy of image mapping in the solutions of related technologies is poor. Summary of the Invention

[0005] In view of this, the image mapping method, apparatus, electronic device, and storage medium provided by the embodiments of the present application can improve the accuracy of image mapping. The image mapping method, apparatus, electronic device, and storage medium provided by the embodiments of the present application are implemented as follows:

[0006] In a first aspect of the embodiments of the present application, an image mapping method is provided. The method includes:

[0007] Based on the area to be mapped in the first image, a first target feature point is determined from multiple first feature points in the first image, and the distance between the first target feature point and the center position of the area to be mapped is less than a threshold;

[0008] Based on the first target feature point, a second target feature point is determined from multiple second feature points in the second image, and the second target feature point corresponds to the first target feature point;

[0009] Based on multiple first target feature points and multiple second target feature points, the area to be mapped is mapped to the second image.

[0010] Optionally, the determining a first target feature point from multiple first feature points in the first image based on the area to be mapped in the first image includes:

[0011] Determine an associated region that matches the to-be-mapped region, where the central position of the associated region is the same as the central position of the to-be-mapped region, the area of the associated region is larger than the area of the to-be-mapped region, and the distance between any point on the edge position of the associated region and the central position of the associated region is less than or equal to the threshold;

[0012] Determine the first target feature points among the multiple first feature points that are located in the associated region.

[0013] Optionally, the determining the associated region that matches the to-be-mapped region includes:

[0014] Generate coordinate information of a second position point according to the coordinate information of the first position point of the to-be-mapped region, the central position of the to-be-mapped region, and a preset region expansion parameter, where the preset region expansion parameter is used to indicate the size relationship between the associated region and the to-be-mapped region;

[0015] Construct the associated region based on the coordinate information of the multiple second target points.

[0016] Optionally, the determining the associated region that matches the to-be-mapped region includes:

[0017] Taking the central position of the to-be-mapped region as the center point and the threshold as the radius, obtain the associated region.

[0018] Optionally, the determining the second target feature points from the multiple second feature points of the second image based on the first target feature points includes:

[0019] Based on the correspondence between the first feature points and the second feature points, take the second feature points corresponding to the first target feature points as the second target feature points.

[0020] Optionally, the mapping the to-be-mapped region to the second image based on the multiple first target feature points and the multiple second target feature points includes:

[0021] Determine a mapping relationship based on the multiple first target feature points and the multiple second target feature points;

[0022] Map the to-be-mapped region to the second image according to the mapping relationship.

[0023] Optionally, the determining the mapping relationship based on the multiple first target feature points and the multiple second target feature points includes:

[0024] Determine the first coordinates of the multiple first target feature points and the second coordinates of the multiple second target feature points;

[0025] Calculate each of the first coordinates and each of the second coordinates to obtain a mapping matrix, which is used to indicate the geometric transformation relationship between the first image and the second image.

[0026] Optionally, mapping the to-be-mapped region to the second image according to the mapping relationship includes:

[0027] Determine first coordinate information corresponding to each first pixel point in the to-be-mapped region;

[0028] Multiply each of the first coordinate information by the mapping matrix respectively to obtain second coordinate information corresponding to each of the first coordinate information, where the second coordinate information is used to indicate a target region corresponding to the to-be-mapped region in the second image;

[0029] Adjust the pixel values of each second pixel point in the target region based on the pixel values of each first pixel point, each of the first coordinate information, and each of the second coordinate information.

[0030] In a second aspect of the embodiments of the present application, an image mapping device is further provided. The device includes:

[0031] A first determination module, configured to determine a first target feature point from multiple first feature points of the first image based on the to-be-mapped region of the first image, where the distance between the first target feature point and the center position of the to-be-mapped region is less than a threshold;

[0032] A second determination module, configured to determine a second target feature point from multiple second feature points of the second image based on the first target feature point, where the second target feature point corresponds to the first target feature point;

[0033] A mapping module, configured to map the to-be-mapped region to the second image based on multiple first target feature points and multiple second target feature points.

[0034] The electronic device provided by the embodiments of the present application includes a memory and a processor. The memory stores a computer program that can run on the processor, and when the processor executes the program, the method described in the embodiments of the present application is implemented.

[0035] The computer-readable storage medium provided by the embodiments of the present application stores a computer program, and when the computer program is executed by a processor, the method provided by the embodiments of the present application is implemented.

[0036] The image mapping method, device, electronic device, and computer-readable storage medium provided by the embodiments of the present application determine first target feature points from multiple first feature points of the first image based on the area to be mapped of the first image. Based on the first target feature points, second target feature points are determined from multiple second feature points of the second image. Based on the multiple first target feature points and the multiple second target feature points, the area to be mapped is mapped to the second image.

[0037] Among them, since each first target feature point is a first feature point with a relatively short distance from the center position of the area to be mapped, and each second target feature point corresponds to each first target feature point, and because the distance between each first target feature point and the center position of the area to be mapped is less than the threshold, it can be ensured as much as possible that each second target feature point can also be located near the target area corresponding to each area to be mapped in the second image. In this way, the first target feature points in the first image that can indicate the surroundings of the area to be mapped, and the second target feature points corresponding to each first target feature point in the second image can be determined. It can be seen that even if the distribution of the first feature points in the first image and the second feature points in the second image is uneven, the embodiments of the present application use the first target feature points with a distance less than the threshold from the center position of the area to be mapped, and the second target feature points corresponding to each first target feature point as the reference points for mapping the area to be mapped in the first image to the second image. In this way, the influence of feature points that are far away can be excluded when performing local mapping on the image, and thus the problem of mapping result deviation caused by uneven distribution of feature points in the image can be effectively solved.

[0038] In this way, the effect of improving the accuracy of image mapping can be achieved, so as to at least partially solve the technical problems proposed in the background art. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0040] Figure 1 It is a flowchart of the first image mapping method provided by the embodiments of the present application;

[0041] Figure 2 It is a schematic diagram of the first image and the second image provided by the embodiments of the present application;

[0042] Figure 3 It is a flowchart of the second image mapping method provided by the embodiments of the present application;

[0043] Figure 4 Flow chart of the third image mapping method provided by the embodiments of the present application;

[0044] Figure 5 Schematic diagram of the associated area provided by the embodiments of the present application;

[0045] Figure 6 Flow chart of the fourth image mapping method provided by the embodiments of the present application;

[0046] Figure 7 Flow chart of the fifth image mapping method provided by the embodiments of the present application;

[0047] Figure 8 Flow chart of the sixth image mapping method provided by the embodiments of the present application;

[0048] Figure 9 Flow chart of the seventh image mapping method provided by the embodiments of the present application;

[0049] Figure 10 Schematic structural diagram of an image mapping device provided by the embodiments of the present application. Detailed implementation manners

[0050] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the following will further describe the specific technical solutions of the present application in detail with reference to the accompanying drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not intended to limit the scope of the present application.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0052] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0053] It should be noted that the terms "first / second / third" involved in the embodiments of the present application are used to distinguish similar or different objects, and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0054] In the related art, generally, feature extraction can be performed on two images to obtain all the feature points in the two images respectively. Then, the feature points in the two images are matched, and the random sample consensus algorithm is used to eliminate the mismatched point pairs. Then, based on the matched point pairs between the two images, the two images are registered and mapped to obtain the mapped image.

[0055] However, in the scenario of local mapping, this solution may cause the problem that the mapped image is offset. Therefore, the accuracy of image mapping in the related art solution is poor.

[0056] To this end, the embodiments of the present application provide an image mapping method. Based on the area to be mapped of the first image, the first target feature points are determined from multiple first feature points of the first image; based on the first target feature points, the second target feature points are determined from multiple second feature points of the second image; based on the multiple first target feature points and the multiple second target feature points, the area to be mapped is mapped to the second image. Wherein, the distance between the first target feature point and the center position of the area to be mapped is less than a threshold, and the second target feature point corresponds to the first target feature point. In this way, the accuracy of image mapping can be improved.

[0057] The embodiments of the present application are described by taking the image mapping method applied in an electronic device as an example. However, it does not mean that the embodiments of the present application can only be applied to image mapping in an electronic device.

[0058] Optionally, the electronic device may include any processor with functions such as processing, control, recognition, and operation. The functions implemented by the image mapping method can be realized by the processor in the electronic device calling program codes. Of course, the program codes can be stored in a computer storage medium. It can be seen that the electronic device at least includes a processor and a storage medium.

[0059] Moreover, the electronic device can be a server, or a wearable device (such as a smart watch, a smart bracelet, smart glasses, etc.), a smart phone, a tablet computer, a notebook computer, a vehicle-mounted terminal, a PC (Personal Computer, personal computer), and any other possible terminal devices. In addition, the electronic device can also be a device composed of a server and a terminal device.

[0060] The image mapping method provided by the embodiments of the present application will be explained in detail below.

[0061] Figure 1 It is a schematic flowchart of an image mapping method provided by the present application. Refer to Figure 1 The embodiments of the present application provide an image mapping method, and the method includes:

[0062] Step 110: Determine first target feature points from multiple first feature points of the first image based on the area to be mapped of the first image.

[0063] Optionally, the first image can be any image that needs to be mapped, that is, the first image can refer to the source image in the image mapping process. The first image can be a Red-Green-Blue (RGB) image, a grayscale image, a binary image, or any other possible type of image. Additionally, the area to be mapped can be any possible area in the first image, and the area to be mapped can specifically be set or selected by a person skilled in the art according to actual needs, or can be determined by performing recognition processing on the first image based on a corresponding image recognition algorithm. The embodiments of the present application do not make any limitations in this regard.

[0064] Optionally, the first feature points of the first image can be simply understood as points at key positions in the first image with significant geometric or brightness features. Specifically, the first feature points can refer to points where the grayscale value changes drastically in the first image. The first feature points can be used as reference points for matching the first image, and can specifically be used to calculate the spatial correspondence relationship between different images, so as to accurately align and / or map the first image with other images.

[0065] Exemplarily, when applying the image mapping method provided in the embodiments of the present application to the field of homework grading, the first image can be obtained by an image acquisition device installed on a terminal device by photographing schoolwork materials such as test papers or exercise books. In this case, the area to be mapped can refer to the answering area in schoolwork materials such as test papers or exercise books. For example, referring to Figure 2 , Figure 2 (a) in shows a schematic diagram of a first image. It can be seen that a plurality of questions and the corresponding answering areas for each question are shown in the first image T1, and there are answering results in 4 answering areas (specifically, areas Q1, Q2, Q3, and Q4 in the first image T1). Then, these 4 answering areas can be used as the areas to be mapped in the first image T1 respectively.

[0066] For another example, continuing to refer to Figure 2 (a) in, each small circle P1 around the questions and answering areas in the first image T1 is a first feature point in the first image T1. Additionally, to more conveniently and clearly introduce the distribution of the first feature points in the first image T1, Figure 2 a partial enlarged schematic diagram of the first image T1 is also provided. Specifically, referring to Figure 2 (b) in, it can be seen that each first feature point (that is, each small circle) is respectively distributed at positions where the grayscale value changes drastically in the first image T1.

[0067] In this embodiment, the distance between the first target feature point and the center position of the to-be-mapped area is less than a threshold. Generally, the distance between the first target feature point and the center position of the to-be-mapped area may refer to the pixel distance. Additionally, the center position of the to-be-mapped area may refer to the geometric center of the to-be-mapped area; for example, if the to-be-mapped area is a circular area, then the center position of the to-be-mapped area may refer to the center of the circle of the to-be-mapped area; if the to-be-mapped area is a rectangular area, then the center position of the to-be-mapped area may refer to the intersection of the diagonals of the to-be-mapped area. The embodiments of the present application do not limit this.

[0068] Optionally, the threshold may be set by relevant technicians according to actual needs. Generally, the threshold may be set to be relatively small, so that the distances between the respective first target feature points and the to-be-mapped areas are relatively close. Moreover, in actual applications, the first target feature point is not necessarily determined directly based on the threshold. Specifically, the first target feature point may be determined by any possible means, as long as it is ensured that the distances between the finally determined respective first target feature points and the center position of the to-be-mapped area are less than the threshold. That is to say, the fact that the distance between the first target feature point and the center position of the to-be-mapped area is less than the threshold is actually the final result, rather than a means of screening the first target feature point from the first feature points. The embodiments of the present application do not limit this.

[0069] It should be noted that since each first feature point is a point at a key position in the first image with significant geometric or luminance features (specifically, a point where the gray value changes drastically), and each first target feature point is a first feature point with a relatively close distance to the center position of the to-be-mapped area, in this way, the feature points that can indicate the surroundings of the to-be-mapped area can be determined, which is convenient for subsequent alignment and / or mapping of the first image based on each first target feature point.

[0070] Step 120: Based on the first target feature point, determine a second target feature point from multiple second feature points of the second image.

[0071] Optionally, the second image may be any type of image. Specifically, the second image may be an RGB image, a grayscale image, a binary image, or any other possible type of image. Generally, the second image may refer to the reference image in the image mapping process. That is to say, in the image mapping method provided by the embodiments of the present application, specifically, each to-be-mapped area in the first image is mapped to the second image.

[0072] Optionally, the second feature points of the second image can be simply understood as the points at key positions in the second image with significant geometric or brightness features, and the second feature points can specifically refer to the points where the gray value changes drastically in the second image. The second feature points can be used as reference points for matching the first image, and can specifically be used to calculate the spatial correspondence between different images, so as to accurately align and / or map the second image with the first image.

[0073] In this embodiment, the first image and the second image generally may have the same feature points, that is, each first feature point may correspond one-to-one to each second feature point. Moreover, since the second target feature points are determined based on the first target feature points, that is to say, the second target feature points may correspond to the first target feature points.

[0074] Exemplarily, continue to refer to Figure 2 , Figure 2 Figure (c) in [reference] shows a schematic diagram of a second image. The second image T2 can be obtained by scanning schoolwork materials such as test papers or exercise books, or can refer to the original document when printing the schoolwork materials. The embodiments of the present application do not make any limitations in this regard. Specifically, each question and the corresponding answer area in the second image T2 are the same as those in the first image T1. Moreover, each small circle P2 located around the questions and answer areas in the second image T2 is the second feature point in the second image T2. It can be seen that each second feature point in the second image T2 corresponds to each first feature point in the first image T1.

[0075] For another example, there may also be corresponding target areas in the second image T2 (specifically, areas Q5, Q6, Q7, and Q8 in the second image T2). When mapping the first image T1 to the second image T2, specifically, each area to be mapped (areas Q1, Q2, Q3, and Q4) in the first image T1 can be mapped to the target areas (areas Q5, Q6, Q7, and Q8) in the second image T2 respectively. It can be seen that each area to be mapped in the first image corresponds one-to-one to each target area in the second image.

[0076] It should be noted that since each second feature point is a point at a key position in the second image with significant geometric or brightness features (specifically, a point where the gray value changes drastically), and each second feature point can be used as a reference point for matching the second image. Since each second target feature point corresponds to each first target feature point, and since the distance between each first target feature point and the center position of the area to be mapped is less than the threshold, it can be ensured as much as possible that each second target feature point can also be located near the target area corresponding to each area to be mapped in the second image.

[0077] Step 130: Based on a plurality of first target feature points and a plurality of second target feature points, map the to-be-mapped region to the second image.

[0078] In this embodiment, during the process of mapping the to-be-mapped region to the second image, it can be performed in any possible manner. For example, it can be mapped by a homography transformation method such as direct linear transformation (DLT) or RANSAC algorithm, or it can be mapped by piecewise affine warping or any other possible method. The embodiments of the present application do not make any limitations in this regard.

[0079] Moreover, specifically mapping the to-be-mapped region to the second image may refer to adjusting the pixel values of the target region in the second image to be the same as those of the corresponding to-be-mapped region; or, it may also refer to cropping, copying, or moving the to-be-mapped region to the corresponding target region in the second image. The embodiments of the present application do not make any limitations in this regard.

[0080] It should be noted that in the solutions of the related art, all the feature points in two images are used for matching, and then based on all the matching point pairs between the two images, the two images are registered and mapped to obtain the mapped image. Since there may be an uneven distribution of feature points in these two images, then, if it is necessary to map the region with fewer feature points in the image, the final mapping result will be affected by the distribution of feature points, resulting in the mapping result shifting towards the region with dense feature points. For example, Figure 2 In the first image and the second image shown, a large number of feature points are distributed on the left side of the image, and only a small number of feature points are distributed on the right side of the image. When mapping the regions Q1, Q2, Q3, and Q4 in the first image T1 to the regions Q5, Q6, Q7, and Q8 in the second image T2, if all the first feature points of the first image T1 and all the second feature points of the second image T2 are used, the final mapping result will be biased towards the left side.

[0081] It should be noted that, however, in this embodiment, during the process of mapping the first image and the second image, not all the first feature points in the first image and all the second feature points in the second image are used as the benchmark points for matching, but only the first target feature points whose distances from the center position of the to-be-mapped region are less than the threshold, and the corresponding second target feature points are used as the benchmark points for matching the first image and the second image.

[0082] That is, even if the first feature points in the first image and the second feature points in the second image are unevenly distributed, only the first target feature points around each area to be mapped and the corresponding second target feature points are used as the benchmark points for matching the first image and the second image. In this way, the influence of feature points far away from the area to be mapped can be excluded when performing local mapping on the image, and thus the deviation problem of the mapping result caused by the uneven distribution of feature points in the image can be effectively solved.

[0083] In the embodiment of the present application, based on the area to be mapped of the first image, the first target feature points are determined from multiple first feature points of the first image. Based on the first target feature points, the second target feature points are determined from multiple second feature points of the second image. Based on the multiple first target feature points and the multiple second target feature points, the area to be mapped is mapped to the second image.

[0084] Among them, since each first target feature point is a first feature point with a relatively short distance from the central position of the area to be mapped, and each second target feature point corresponds to each first target feature point, and because the distance between each first target feature point and the central position of the area to be mapped is less than the threshold, it can be ensured as much as possible that each second target feature point can also be located near the target area corresponding to each area to be mapped in the second image. In this way, the first target feature points that can indicate around the area to be mapped in the first image and the second target feature points corresponding to each first target feature point in the second image can be determined. It can be seen that even if the first feature points in the first image and the second feature points in the second image are unevenly distributed, in the embodiment of the present application, the first target feature points with a distance less than the threshold from the central position of the area to be mapped and the second target feature points corresponding to each first target feature point are used as the benchmark points for mapping the area to be mapped in the first image to the second image. In this way, the influence of feature points far away can be excluded when performing local mapping on the image, and thus the deviation problem of the mapping result caused by the uneven distribution of feature points in the image can be effectively solved.

[0085] In this way, the effect of improving the accuracy of image mapping can be achieved.

[0086] In a possible implementation manner, see Figure 3 , based on the area to be mapped of the first image, determining the first target feature points from multiple first feature points of the first image includes:

[0087] Step 1101: Determine the associated area that matches the area to be mapped.

[0088] In this embodiment, the area of the associated region is greater than the area of the region to be mapped, and the distance between any point on the edge position of the associated region and the center position of the associated region is less than or equal to the threshold. That is to say, the associated region can contain the region to be mapped.

[0089] Optionally, the associated region can be any possible shape such as a rectangle, a circle, a triangle, etc., and the embodiments of the present application do not limit this. Moreover, the associated region can be determined in any possible way, and the embodiments of the present application do not limit this.

[0090] Step 1102: Determine the first feature points located in the associated region among the multiple first feature points as the first target feature points.

[0091] In this embodiment, the center position of the associated region can also be the same as the center position of the region to be mapped. In this way, the problem that the distribution of the determined first target feature points relative to the region to be mapped is uneven due to the center offset between the associated region and the region to be mapped can be reduced.

[0092] It should be noted that since the distance between any point on the edge position of the associated region and the center position of the associated region is less than or equal to the threshold, and the center position of the associated region can also be the same as the center position of the region to be mapped, it can be ensured that the distance between the first feature points located in the associated region and the center position of the region to be mapped is less than the threshold. In this way, the influence of feature points that are far away from the region to be mapped can be excluded when mapping the image subsequently.

[0093] The following gives an example of the specific method for determining the associated region. In a possible implementation, see Figure 4 , determining the associated region matching the region to be mapped includes:

[0094] Step 1103: Generate the coordinate information of the second position points according to the coordinate information of the first position points of the region to be mapped, the center position of the region to be mapped, and the preset region expansion parameter.

[0095] Optionally, the first position point can be a point for indicating the size of the region to be mapped. For example, the first position point can be each point on the edge position of the region to be mapped, or can also be each vertex of the region to be mapped. The embodiments of the present application do not limit this.

[0096] In this embodiment, each second position point can correspond to each first position point respectively. Correspondingly, the second position point can be a point for indicating the size of the associated region. For example, the second position point can be each point on the edge position of the associated region, or can also be each vertex of the associated region.

[0097] Optionally, the preset region expansion parameter is used to indicate the size relationship between the associated region and the region to be mapped. Generally, the preset region expansion parameter can be set based on the above threshold. For example, if the region to be mapped is circular and the radius of the region to be mapped is 10, and the threshold is 20, then the preset region expansion parameter can be 2; if the region to be mapped is rectangular and the diagonal length of the region to be mapped is 8, and the threshold is 40, then the preset region expansion parameter can be 5. It can be specifically set according to actual needs, and the embodiments of the present application do not limit this.

[0098] Exemplarily, when generating the coordinate information of the second position point according to the coordinate information of the first position point of the region to be mapped, the center position of the region to be mapped, and the preset region expansion parameter, specifically, it can first be determined whether the center position of the region to be mapped is the coordinate origin; if so, the coordinate information of each first position point can be directly multiplied by the preset region expansion parameter to obtain the coordinate information of each second position point. For example, if the region to be mapped is rectangular and the center position of the region to be mapped is the coordinate origin, the preset region expansion parameter is 2, and the first position points of the region to be mapped include four vertices with coordinates (1, 1), (1, -1), (-1, 1), and (-1, -1). It can be seen that the region to be mapped is a square with a side length of 2, and the diagonal length is In this way, the coordinate information of each second position point can be obtained as (2, 2), (2, -2), (-2, 2), and (-2, -2). It can be seen that the associated region is a square with a side length of 4, and the diagonal length is

[0099] If the center position of the region to be mapped is not the coordinate origin, then it is necessary to determine the quadrant in which the first position point of the region to be mapped is located, and then process the coordinate information of the first position point according to the preset region expansion parameter to obtain the coordinate information of the second position point. For another example, if the region to be mapped is rectangular and the center position of the region to be mapped is (1.5, 2.5), and the first position points of the region to be mapped include four vertices with coordinates (1, 2), (2, 2), (1, 3), and (2, 3). It can be seen that the region to be mapped is a square with a side length of 1, and the diagonal length is Assuming that the preset region expansion parameter is 2, then the associated region should be a square with a side length of 2 and a diagonal length of and having the same center position as the region to be mapped. Therefore, the coordinate information of each second position point can be determined as (0.5, 1.5), (2.5, 1.5), (0.5, 3.5), and (2.5, 3.5).

[0100] It should be noted that the above content is only several possible examples for implementing step 1103, and does not mean that in the image mapping method provided in the embodiments of the present application, the coordinate information of each second position point can only be determined in the manners listed above. The embodiments of the present application do not make any limitations in this regard.

[0101] Step 1104: Based on the coordinate information of the multiple second target points, construct the association area.

[0102] Generally, the area surrounded by the multiple second target points can be used as the association area, and the embodiments of the present application do not make any limitations in this regard.

[0103] Exemplarily, referring to Figure 5 , Figure 5 in, (a) shows a schematic diagram of an association area. As can be seen from Figure 5 (a), an association area G2 is constructed around the area Q2 to be mapped. The central positions of the association area G2 and the area Q2 to be mapped are the same. The area of the association area G2 is larger than the area of the area Q2 to be mapped. Moreover, the distance between any point on the edge position of the association area G2 and the central position of the association area G2 is less than or equal to half of the diagonal of the association area G2. Moreover, an association area G4 is constructed around the area Q4 to be mapped. The central positions of the association area G4 and the area Q4 to be mapped are the same. The area of the association area G4 is larger than the area of the area Q4 to be mapped. Moreover, the distance between any point on the edge position of the association area G4 and the central position of the association area G4 is less than or equal to half of the diagonal of the association area G4. That is, the preset area expansion parameter corresponding to the association area can be set according to the above threshold.

[0104] In addition, in Figure 5 (a), each small circle M1 located within the association area G2 corresponds to the first target feature point of the area Q2 to be mapped, and each small circle located within the association area G4 corresponds to the first target feature point of the area Q4 to be mapped.

[0105] In a possible implementation manner, referring to Figure 6 , determining the association area matching the area to be mapped includes:

[0106] Step 1105: Using the central position of the area to be mapped as the center point and the threshold as the radius, obtain the association area.

[0107] Exemplarily, referring to Figure 5 , Figure 5 in, (b) shows a schematic diagram of an association area. As can be seen from Figure 5As shown in (b) of [Figure 0], an associated region G3 is constructed around the region Q3 to be mapped. The center positions of the associated region G3 and the region Q3 to be mapped are the same. The area of the associated region G3 is larger than that of the region Q3 to be mapped. Moreover, the distance between any point on the edge position of the associated region G3 and the center position of the associated region G3 is equal to the radius of the associated region G2.

[0108] It should be noted that in this way, a circular associated region with the same center as the region to be mapped and a radius equal to the threshold can be obtained. Therefore, the distance between any point on the edge position of the associated region and the center position of the associated region is equal to the threshold. In this way, it can be ensured that the distance between the first feature point located in the associated region and the center position of the region to be mapped is less than the threshold.

[0109] It can be understood that the two methods provided in steps 1103 - 1105 are only for illustrative purposes of the process of determining the associated region, and do not mean that the embodiments of the present application can only determine the associated region in such a way. In addition, the embodiments of the present application do not have to determine the first target feature point only through the associated region. For example, the distance between each first feature point and the center position of the region to be mapped can be calculated respectively, and then all the first feature points with a distance less than or equal to the above threshold can be used as the first target feature points. In this way, it is no longer necessary to screen out the first target feature points from each first feature point by determining the associated region. It can be seen that the embodiments of the present application can determine each first target feature point through a variety of different methods, which has a certain degree of flexibility and practicality.

[0110] In a possible implementation manner, refer to Figure 7 , based on the first target feature point, determining a second target feature point from multiple second feature points of the second image, including:

[0111] Step 1201: Based on the correspondence between the first feature point and the second feature point, take the second feature point corresponding to the first target feature point as the second target feature point.

[0112] In this embodiment, the correspondence can be information for aligning the first image and the second image to the same coordinate system. Specifically, the correspondence can be determined by any possible method such as Scale - Invariant Feature Transform (SIFT), Speeded - Up Robust Features (SURF), Oriented FAST and Rotated BRIEF, Harris corner detection, RANSAC, etc. The embodiments of the present application do not limit this.

[0113] It can be understood that since each first target feature point is a special first feature point, when determining the corresponding relationship between each first feature point and each second feature point, it is possible to determine which second feature points correspond to each first target feature point, and thus determine each second target feature point.

[0114] In a possible implementation manner, referring to Figure 8 , mapping the to-be-mapped region to the second image based on multiple first target feature points and multiple second target feature points includes:

[0115] Step 1301: Determine a mapping relationship based on multiple first target feature points and multiple second target feature points.

[0116] Therefore, this mapping relationship can be used to indicate the conversion relationship between the to-be-mapped region (and / or the associated region corresponding to the to-be-mapped region) and the target region in the second image.

[0117] Step 1302: Map the to-be-mapped region to the second image according to this mapping relationship.

[0118] It should be noted that since the first target feature point is a feature point in the first image whose distance from the center position of the to-be-mapped region is less than a threshold, and the second target feature point corresponds to the first target feature point in the second image. That is to say, this mapping relationship can not only align the to-be-mapped region in the first image and the target region in the second image to the same coordinate system, but also exclude the influence of feature points that are far away from the to-be-mapped region on the mapping, and thus can effectively solve the problem of mapping result deviation caused by uneven distribution of feature points in the image.

[0119] In a possible implementation manner, referring to Figure 9 , determining a mapping relationship based on multiple first target feature points and multiple second target feature points includes:

[0120] Step 1303: Determine the first coordinates of multiple first target feature points and the second coordinates of multiple second target feature points.

[0121] Optionally, the first coordinate of the first target feature point refers to the coordinate value of the first target feature point in the local coordinate system established based on the first image. The second coordinate of the second target feature point refers to the coordinate value of the second target feature point in the local coordinate system established based on the second image.

[0122] Step 1304: Calculate each first coordinate and each second coordinate to obtain a mapping matrix.

[0123] Optionally, the mapping matrix is used to indicate the geometric transformation relationship between the first image and the second image, that is, the mapping matrix can describe, through mathematical modeling, the way to transform the pixel positions of the first image into the coordinate system of the second image. Additionally, the mapping matrix can be a homography matrix or any other possible matrix, and the embodiments of the present application do not limit this.

[0124] Exemplarily, assume that the first coordinate of the first target feature point is X = [x, y] T , and the second coordinate of the second target feature point is X' = [x', y'] T . Denote the mapping matrix as H, then X' = HX. Specifically, assume that H is an 8-parameter mapping matrix. It can be known that a mapping matrix H can be determined by four pairs of feature points obtained by matching four first target feature points and four second target feature points. Then the mapping matrix H can be obtained as shown in the following formula:

[0125] It should be noted that since only the first target feature points and the corresponding second target feature points closer to the area to be mapped are used when calculating the mapping matrix, and other feature points farther away are not used, the mapping matrix can exclude the influence of feature point pairs far from the area to be mapped on the mapping, and thus can effectively solve the problem of deviation in the mapping result caused by uneven distribution of feature points in the image.

[0126] In a possible implementation manner, mapping the area to be mapped to the second image according to the mapping relationship includes:

[0127] Determine the first coordinate information corresponding to each first pixel point in the area to be mapped.

[0128] Optionally, each first pixel point may specifically include all pixel points in the area to be mapped, and the first coordinate information may refer to the coordinate value of the first pixel point in the local coordinate system established based on the first image.

[0129] Multiply each first coordinate information by the mapping matrix respectively to obtain the second coordinate information corresponding to each first coordinate information.

[0130] Optionally, the second coordinate information may refer to the coordinate value of the local coordinate system established based on the first image. Specifically, the second coordinate information is used to indicate the target area corresponding to the area to be mapped in the second image. That is to say, based on the second coordinate information, the target area corresponding to the area to be mapped in the second image can be determined. Specifically, the area surrounded by each second coordinate information can be used as the target area, and the embodiments of the present application do not limit this.

[0131] It should be noted that since the mapping matrix can indicate the geometric transformation relationship between the first image and the second image, when each piece of first coordinate information is multiplied by the mapping matrix respectively, the coordinate information corresponding to each first pixel point in the first image in the second image can be accurately calculated for subsequent image mapping.

[0132] Based on the pixel values of each first pixel point, each piece of first coordinate information, and each piece of second coordinate information, adjust the pixel values of each second pixel point in the target area.

[0133] Specifically, the second coordinate information corresponding to a first pixel point can be matched according to the first coordinate information of any first pixel point, then a second pixel point corresponding to this second coordinate information in the second image is determined, and then the pixel value of this second pixel point is adjusted to the pixel value of this first pixel point. And the above steps are repeated multiple times until the adjustment stops when the pixel values of all second pixel points in the target area are adjusted. In this way, the area to be mapped can be mapped to the second image.

[0134] It should be noted that since the mapping matrix is calculated based on the first target feature points and the corresponding second target feature points around each area to be mapped. In this way, even if the first feature points in the first image and the second feature points in the second image are unevenly distributed, the influence of feature points far away from the area to be mapped can be excluded during local image mapping, and thus the problem of mapping result deviation caused by uneven distribution of feature points in the image can be effectively solved.

[0135] In a possible implementation manner, the method may further include:

[0136] Feature extraction is respectively performed on the first image and the second image through a preset extraction algorithm to obtain a plurality of first feature points and a plurality of second feature points.

[0137] Wherein, the first feature point corresponds to a first marker information and a first coordinate information, and the second feature point corresponds to a second marker information and a second coordinate information.

[0138] Optionally, the first marker information and the second marker information may refer to feature descriptors, and a feature descriptor is a set of numerical vectors that can be used to describe the local information of feature points in an image, such as texture, gradient, color distribution, etc. Therefore, the first marker information and the second marker information can be used to distinguish different feature points.

[0139] Optionally, the preset extraction algorithm may be the Scale-Invariant Feature Transform (SIFT) algorithm, the Oriented FAST and Rotated BRIEF (ORB) algorithm, the Speeded-Up Robust Features (SURF) algorithm, the Binary Robust Independent Elementary Features (BRIEF) algorithm, or any other possible feature extraction algorithm. The embodiments of the present application do not limit this.

[0140] Generally, the same preset extraction algorithm can be selected for feature extraction of the first image and the second image. In this way, the problem that the first feature points and the second feature points may be different due to different extraction algorithms can be avoided, so as to improve the accuracy of subsequent image mapping.

[0141] In a possible implementation manner, after obtaining a plurality of first feature points and a plurality of second feature points by respectively performing feature extraction on the first image and the second image through a preset extraction algorithm, the method further includes:

[0142] Matching the first marking information corresponding to each first feature point with the second marking information corresponding to each second feature point respectively to obtain the corresponding relationship between the first feature point and the second feature point.

[0143] Generally, if the first marking information corresponding to a first feature point is the same as or highly similar to the second marking information corresponding to a second feature point, it can be determined that this first feature point and this second feature point correspond to each other.

[0144] It should be noted that any possible method such as Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), Oriented FAST and Rotated BRIEF, Harris corner detection, RANSAC, etc. can be used to determine the corresponding relationship. The embodiments of the present application do not limit this. In this way, it is convenient to subsequently use the second feature point corresponding to the first target feature point as the second target feature point based on the corresponding relationship.

[0145] It should be understood that although the steps in the above flowcharts are displayed in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0146] Based on the foregoing embodiments, an embodiment of the present application provides an image mapping device. The device includes each module included therein, as well as each unit included in each module, and can be implemented by a processor; of course, it can also be implemented by specific logic circuits. During implementation, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0147] Figure 10 is a schematic structural diagram of an image mapping device provided by an embodiment of the present application. Refer to Figure 10 , the device includes:

[0148] A first determination module 201, configured to determine a first target feature point from multiple first feature points of the first image based on the area to be mapped of the first image.

[0149] Optionally, the distance between the first target feature point and the center position of the area to be mapped is less than a threshold.

[0150] A second determination module 202, configured to determine a second target feature point from multiple second feature points of the second image based on the first target feature point.

[0151] Optionally, the second target feature point corresponds to the first target feature point.

[0152] A mapping module 203, configured to map the area to be mapped to the second image based on multiple first target feature points and multiple second target feature points.

[0153] The description of the above device embodiment is similar to the description of the above method embodiment, and has beneficial effects similar to those of the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0154] It should be noted that in the embodiment of the present application Figure 10 The division of the modules of the image mapping device shown is schematic, and is only a logical function division. There may be other division methods during actual implementation. In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or can exist separately physically, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware, or in the form of a software functional unit, or in the form of a combination of software and hardware.

[0155] It should be noted that in the embodiments of the present application, if the above method is implemented in the form of software function modules and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing an electronic device to execute all or part of the methods described in the various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), magnetic disks, or optical discs that can store program codes. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0156] The embodiments of the present application provide an electronic device, which can be a server. Its internal structure diagram may include a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the electronic device is used to store data. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0157] The embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the method provided in the above embodiments are implemented.

[0158] The embodiments of the present application provide a computer program product containing instructions. When it runs on a computer, it causes the computer to execute the steps in the method provided in the above method embodiments.

[0159] Those skilled in the art can understand that the structure of the electronic device listed in the above embodiments is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0160] In one embodiment, the image mapping device provided by the present application can be implemented in the form of a computer program, and the computer program can run on the above electronic device. Each program module constituting the above device can be stored in the memory of the electronic device. The computer program composed of each program module causes the processor to execute the steps in the methods described in the various embodiments of the present application in this specification.

[0161] It should be noted that the descriptions of the above storage medium and device embodiments are similar to those of the above method embodiments and have similar beneficial effects to the method embodiments. For the technical details not disclosed in the storage medium, storage medium and device embodiments of the present application, please refer to the descriptions of the method embodiments of the present application for understanding.

[0162] It should be understood that the "one embodiment" or "an embodiment" or "some embodiments" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, the appearances of "in one embodiment" or "in an embodiment" or "in some embodiments" throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the magnitudes of the serial numbers of the above processes do not mean the order of execution is prior or subsequent, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The serial numbers of the embodiments of the present application above are only for description and do not represent the advantages or disadvantages of the embodiments. The descriptions of the above embodiments tend to emphasize the differences between the embodiments, and their similarities or similarities can be referred to each other. For the sake of brevity, they will not be elaborated herein.

[0163] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, object A and / or object B can represent: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0164] It should be noted that in this article, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, the element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0165] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed with each other can be through some interfaces. The indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0166] The modules described above as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules; they can be located in one place or distributed to multiple network units; some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0167] In addition, in each embodiment of the present application, all the functional modules can be integrated in a processing unit, or each module can be a separate unit alone, or two or more modules can be integrated in a unit; the above-mentioned integrated modules can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0168] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: removable storage devices, read-only memory (ROM), magnetic disks, or optical disks, etc., which can store program codes.

[0169] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling an electronic device to execute all or part of the methods described in each embodiment of the present application. And the foregoing storage medium includes: removable storage devices, ROM, magnetic disks, or optical disks, etc., which can store program codes.

[0170] The methods disclosed in several method embodiments provided by the present application can be combined arbitrarily without conflict to obtain new method embodiments.

[0171] The features disclosed in several product embodiments provided by this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0172] The features disclosed in several method or device embodiments provided by this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0173] As mentioned above, it is only the implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by this application, and all of them should be covered by the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claimed rights.

[0174] The above are only the preferred embodiments of this application and are not used to limit this application. For those skilled in the art, this application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.

Claims

1. An image mapping method, characterized in that, The method includes: Based on the area to be mapped in the first image, determining a first target feature point from a plurality of first feature points in the first image, wherein the distance between the first target feature point and the center position of the area to be mapped is less than a threshold; Based on the first target feature point, determining a second target feature point from a plurality of second feature points in the second image, wherein the second target feature point corresponds to the first target feature point; Based on a plurality of the first target feature points and a plurality of the second target feature points, mapping the area to be mapped to the second image.

2. The image mapping method according to claim 1, wherein The determining a first target feature point from a plurality of first feature points in the first image based on the area to be mapped in the first image includes: Determining an associated area that matches the area to be mapped, wherein the center position of the associated area is the same as the center position of the area to be mapped, the area of the associated area is larger than the area of the area to be mapped, and the distance between any point on the edge position of the associated area and the center position of the associated area is less than or equal to the threshold; Determining the first feature points located in the associated area among the plurality of first feature points as the first target feature points.

3. The image mapping method according to claim 2, wherein The determining an associated area that matches the area to be mapped includes: Generating coordinate information of a second position point according to the coordinate information of a first position point of the area to be mapped, the center position of the area to be mapped, and a preset area expansion parameter, wherein the preset area expansion parameter is used to indicate the size relationship between the associated area and the area to be mapped; Constructing the associated area based on the coordinate information of the plurality of second target points.

4. The image mapping method according to claim 2, characterized in that, The determining an associated area that matches the area to be mapped includes: Taking the center position of the area to be mapped as the center point and the threshold as the radius to obtain the associated area.

5. The image mapping method according to claim 2, wherein The determining a second target feature point from a plurality of second feature points in the second image based on the first target feature point includes: Based on the correspondence relationship between the first feature point and the second feature point, taking the second feature point corresponding to the first target feature point as the second target feature point.

6. The image mapping method according to any one of claims 1-5, characterized in that, The mapping the area to be mapped to the second image based on a plurality of the first target feature points and a plurality of the second target feature points includes: Determining a mapping relationship based on a plurality of the first target feature points and a plurality of the second target feature points; Mapping the area to be mapped to the second image according to the mapping relationship.

7. The image mapping method according to claim 6, characterized in that, The determining a mapping relationship based on a plurality of the first target feature points and a plurality of the second target feature points includes: Determining first coordinates of a plurality of the first target feature points and second coordinates of a plurality of the second target feature points; Calculating each of the first coordinates and each of the second coordinates to obtain a mapping matrix, wherein the mapping matrix is used to indicate the geometric transformation relationship between the first image and the second image.

8. The image mapping method according to claim 7, wherein The mapping the area to be mapped to the second image according to the mapping relationship includes: Determining first coordinate information corresponding to each first pixel point in the area to be mapped; Multiply each of the first coordinate information by the mapping matrix to obtain second coordinate information corresponding to each of the first coordinate information, where the second coordinate information is used to indicate a target area corresponding to the area to be mapped in the second image; Based on the pixel values of each of the first pixel points, each of the first coordinate information, and each of the second coordinate information, adjust the pixel values of each of the second pixel points in the target area.

9. An image mapping device, characterized in that, The apparatus includes: A first determination module, configured to determine a first target feature point from multiple first feature points of the first image based on the area to be mapped in the first image, where the distance between the first target feature point and the center position of the area to be mapped is less than a threshold; A second determination module, configured to determine a second target feature point from multiple second feature points of the second image based on the first target feature point, where the second target feature point corresponds to the first target feature point; A mapping module, configured to map the area to be mapped to the second image based on multiple first target feature points and multiple second target feature points.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory stores a computer program that can run on the processor, and when the processor executes the program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method according to any one of claims 1 to 8.

Images