Image processing method and electronic equipment

Abstract

The invention provides an image processing method and electronic equipment. The image processing method comprises the following steps: acquiring a first image and a second image of an object; determining a first vector of a first feature point of the object in the first image and a second vector of a second feature point of the object in the second image; based on the first vector and the second vector, obtaining a group of feature matching relations of the first image and the second image at a certain position; obtaining an optimal pose transformation matrix of the object according to the multiple groups of feature matching relationships at the multiple positions; and obtaining an object image of the object according to the first image, the second image and the optimal pose transformation matrix. And each group of feature matching relations can be quickly determined according to the first vector and the second vector, so that the optimal pose transformation matrix of the object can be quickly obtained according to multiple groups of feature matching relations at multiple positions. And finally, obtaining an object image of the object according to the first image, the second image and the optimal pose transformation matrix, so that the object image of the object better accords with a real object.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular, relates to an image processing method and electronic equipment. Background Art

[0002] When an electronic device captures an image of an object, the object presented in the image may not match the real object due to factors such as the structural setting, installation position and shooting delay of the electronic device.

[0003] At present, various methods are used to improve the situation where the objects presented in the image do not match the real objects. For example, when using the traditional feature point matching algorithm, this method will make feature point judgment errors when calculating feature points and point clouds on the grid, thereby causing other errors and failing to improve the situation where the objects presented in the image do not match the real objects. For another example, a deep learning method can be used, which requires huge amounts of data and is time-consuming to calculate. For another example, geometric consistency can be used to verify the matching relationship between the feature points of the point cloud and the image in the point cloud image. This method has difficulty finding accurate and sufficiently corresponding feature points for point clouds with high discreteness. Summary of the invention

[0004] The embodiments of the present application provide an image processing method and an electronic device, so that the electronic device can present an object more in line with the actual situation.

[0005] A first aspect of an embodiment of the present application provides an image processing method, including: acquiring a first image and a second image of an object; determining a first vector of a first feature point of the object in the first image, and a second vector of a second feature point of the object in the second image; based on the first vector and the second vector, obtaining a set of feature matching relationships between the first image and the second image at a certain position; obtaining an optimal posture transformation matrix of the object based on multiple sets of feature matching relationships located at multiple positions; and obtaining an object image of the object based on the first image, the second image and the optimal posture transformation matrix.

[0006] In the above technical solution, each set of feature matching relationships can be determined faster according to the first vector and the second vector, so as to more quickly obtain the optimal posture transformation matrix of the object according to multiple sets of feature matching relationships located at multiple positions. Finally, the object image of the object is obtained according to the first image, the second image and the optimal posture transformation matrix, so that the object image of the object is more consistent with the real object.

[0007] In some embodiments of the first aspect, determining a first vector of a first feature point of an object in a first image and a second vector of a second feature point of an object in a second image includes: extracting a first edge segment of the object in the first image and a second edge segment of the object in the second image in a pixel coordinate system; obtaining a first direction and a first change intensity according to the grayscale of pixels on both sides of the first edge segment, and obtaining a second direction and a second change intensity according to the grayscale of pixels on both sides of the second edge segment; obtaining a first vector of the first feature point of the first edge segment according to a first perpendicular line of the first edge segment, and the first direction and the first change intensity; obtaining a second vector of the second feature point of the second edge segment according to a second perpendicular line of the second edge segment, and the second direction and the second change intensity.

[0008] In some embodiments of the first aspect, a first direction and a first change intensity are obtained according to the grayscale of pixels on both sides of a first edge segment, and a second direction and a second change intensity are obtained according to the grayscale of pixels on both sides of a second edge segment, including: determining a first grayscale mean of pixels on the first side of the first edge segment and a second grayscale mean of pixels on the second side; obtaining the first direction and the first change intensity according to the first grayscale mean and the second grayscale mean; determining a third grayscale mean of pixels on the third side of the second edge segment and a fourth grayscale mean of pixels on the fourth side; obtaining the second direction and the second change intensity according to the third grayscale mean and the fourth grayscale mean.

[0009] In some embodiments of the first aspect, obtaining a first direction according to the first grayscale mean and the second grayscale mean includes: comparing the sizes of the first grayscale mean and the second grayscale mean, and obtaining a first direction pointing from a first side of the smaller first grayscale mean to a second side of the larger second grayscale mean; obtaining a second direction according to the third grayscale mean and the fourth grayscale mean includes: comparing the sizes of the third grayscale mean and the fourth grayscale mean, and obtaining a second direction pointing from a third side of the smaller third grayscale mean to a fourth side of the larger third grayscale mean.

[0010] In some embodiments of the first aspect, obtaining a first change intensity according to a first grayscale mean and a second grayscale mean includes: obtaining the first change intensity according to a difference between the first grayscale mean and the second grayscale mean; obtaining a second change intensity according to a third grayscale mean and a fourth grayscale mean, including: obtaining the second change intensity according to a difference between the third grayscale mean and the fourth grayscale mean.

[0011] In some embodiments of the first aspect, based on the first vector and the second vector, a set of feature matching relationships between the first image and the second image at a certain position is obtained, including: in the image coordinate system, multiple angles are obtained according to the first vector of each first feature point and the second vectors of multiple second feature points; a matching point is determined among the multiple second feature points according to the multiple angles and angle thresholds; a matching relationship is obtained according to the first vector of each first feature point and the second vector of the matching point; and a set of feature matching relationships between the first image and the second image at a certain position is obtained according to all the matching relationships.

[0012] In some embodiments of the first aspect, a matching point is determined among multiple second feature points based on multiple angles and angle thresholds, including: based on the first vector and the second vector, determining the distance between each first feature point and multiple second feature points; sorting the multiple distances from near to far, and comparing the multiple angles and angle thresholds in turn according to the sorting results; if the multiple angles are greater than or equal to the angle threshold, abandoning the first feature points corresponding to the multiple angles; if the angle is greater than or equal to the angle threshold, determining the second feature point corresponding to the angle as the matching point.

[0013] In some embodiments of the first aspect, a distance between the first feature point and the second feature point is within a preset range.

[0014] In some embodiments of the first aspect, an optimal posture transformation matrix of an object is obtained based on multiple sets of feature matching relationships located at multiple positions, including: moving a first image and a second image multiple times; determining multiple sets of feature matching relationships between the first image and the second image located at multiple positions; obtaining multiple matching degrees based on the multiple sets of feature matching relationships; and determining that the posture transformation matrix corresponding to the maximum matching degree among the multiple matching degrees is the optimal posture transformation matrix.

[0015] The second aspect of the present application provides an electronic device, comprising: a processor, adapted to execute a computer program; and a computer-readable storage medium, in which the computer program is stored. When the computer program is executed by the processor, the above-mentioned image processing method is executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of an application scenario of an electronic device according to an embodiment of the present application.

[0017] Figure 2 Schematic diagram of a process of an image processing method according to an embodiment of the present application.

[0018] Figure 3 A schematic diagram of an application scenario of the image processing method according to an embodiment of the present application.

[0019] Figure 4A schematic diagram of another application scenario of the image processing method according to an embodiment of the present application.

[0020] Figure 5 A schematic diagram of yet another application scenario of the image processing method of an embodiment of the present application.

[0021] Figure 6 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In the present application, the term "plurality" refers to two or more. In addition, it should be understood that in the description of the present application, the terms "first", "second", etc. are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0023] The following is a brief description of the relevant technologies:

[0024] When an electronic device captures an image of an object, the object presented in the image may not match the real object due to factors such as the structural setting, installation position and shooting delay of the electronic device.

[0025] At present, after obtaining the three-dimensional point cloud image of the point cloud projected onto the object (the point cloud in the point cloud image is the projection point), as well as the image of the object without the point cloud projection, various methods are combined to improve the situation where the object presented in the image does not match the real object. For example, using the traditional feature point matching algorithm, this method will make feature point determination errors when calculating feature points and point clouds on the grid, thereby causing other errors and failing to improve the situation where the object presented in the image does not match the real object. For another example, a deep learning method can be used, which requires huge amounts of data and is time-consuming to calculate. For another example, geometric consistency can be used to verify the matching relationship between the feature points of the point cloud and the image in the point cloud image. This method has difficulty finding accurate and sufficiently corresponding feature points for point clouds with high discreteness.

[0026] In view of this, an embodiment of the present application provides an image processing method and an electronic device, so that the electronic device can present an object more in line with the actual situation.

[0027] See also Figure 1 , Figure 1 The electronic device 10 is an application scenario diagram, in which the electronic device 10 includes a color image acquisition unit 11 and a laser acquisition unit 12. The color image acquisition unit 11 is used to acquire a true color image of an object, and the color image acquisition unit 11 may be a camera, etc. The laser acquisition unit is used to acquire a three-dimensional image of an object, and the laser acquisition unit may be a laser scanner.

[0028] Please combine Figure 1 and Figure 2, an image processing method provided in an embodiment of the present application comprises the following steps:

[0029] Step S101: Acquire a first image and a second image of an object.

[0030] The first image refers to a true color image captured by the color image acquisition unit 11. The true color image includes the position information and color information of the object in space. The second image refers to a three-dimensional image captured by the laser acquisition unit 12. The laser acquisition unit 12 emits a laser to the object, receives the laser reflected or scattered back by the object, and converts the laser into three-dimensional point cloud data (3D point cloud) of the object on the three-dimensional image. In this way, the point cloud on the second image can indicate the position and direction of the object in three-dimensional space.

[0031] Step S102: determining a first vector of a first feature point of an object in a first image, and a second vector of a second feature point of an object in a second image.

[0032] An object in an image can be composed of multiple feature points. The feature points of an object in an image can be extracted through a feature extraction algorithm, and each extracted feature point has a corresponding vector. A first feature point refers to a feature point that reflects an object in a first image, and each first feature point has a corresponding first vector. A second feature point refers to a feature point that reflects an object in a second image, and each second feature point has a corresponding second vector.

[0033] Step S103: Based on the first vector and the second vector, a set of feature matching relationships between the first image and the second image at a certain position is obtained.

[0034] The color image acquisition unit 11 and the laser acquisition unit 12 capture the same object, so the object in the first image and the object in the second image can overlap in theory, that is, the first feature point can find a second feature point that can overlap in theory. The matching relationship between the first feature point and the second feature point can refer to the existence of a theoretical overlapping relationship between the first feature point and the second feature point.

[0035] The first feature point has a first vector, and the second feature point has a second vector. The first vector and the second vector can be used to determine whether there is a matching relationship between the first specific point and the second specific point. There is one matching relationship between a first specific point and a second specific point, and a set of feature matching relationships includes all matching relationships. For example, there are N matching relationships between the N first features of the first image and the N second features of the second image, and a set of feature matching relationships includes N matching relationships.

[0036] When calculating whether there is a matching relationship between the first feature point and the second feature point, the first image and the second image are stationary at a certain position. At this time, a set of matching relationships calculated is a set of feature matching relationships between the first image and the second image at a certain position.

[0037] Step S104: obtaining an optimal posture transformation matrix of the object according to a plurality of feature matching relationships at a plurality of positions.

[0038] When the first image and the second image are located at a certain position, one set of feature matching relationships can be obtained. When they are located at multiple positions, multiple sets of feature matching relationships can be obtained.

[0039] It can be understood that by changing the positions of the first image and the second image to change the overlapping relationship (i.e., feature matching relationship) between the objects in the first image and the second image, a better feature matching relationship can be found, and an optimal posture transformation matrix can be obtained based on the better feature matching relationship. Obtaining the optimal posture transformation matrix is ​​to calibrate the external parameters of the electronic device, so that after the electronic device acquires the image, it can correct the erroneous position information in the image based on the optimal posture transformation matrix.

[0040] Step S105: obtaining an object image of the object according to the first image, the second image and the optimal posture transformation matrix.

[0041] The first image carries the location information of the object, and the location information has corresponding color information, for example, the first image has the location of the road, the gray color of the road surface, and the white color of the lane line on the road surface. The second image carries the three-dimensional location information of the object (the three-dimensional location information can be three-dimensional point cloud data).

[0042] After obtaining the position information of the first image, the color information carried by the position information is rendered to the three-dimensional position of the second image of the corresponding object based on the optimal posture transformation matrix, thereby obtaining a three-dimensional image of the object with color (that is, the object image is a three-dimensional image of the object with color).

[0043] It can be understood that the first image provides color for the imaging of the object, the second image provides a three-dimensional position for the imaging of the object, and the optimal posture transformation matrix is ​​used to accurately render the color to the corresponding three-dimensional position.

[0044] In an application scenario, the electronic device 10 obtains a first image of the road (the first image carries the position of the road, and the color corresponding to the position) and a second image (the second image carries the three-dimensional position of the road), and determines the first vector of the first feature point of the road in the first image, and the second vector of the second feature point of the road in the second image. Next, based on the first vector and the second vector, a set of feature matching relationships between the first image and the second image at a certain position is obtained, and the optimal posture transformation matrix of the road is obtained based on multiple sets of feature matching relationships at multiple positions. Finally, the color corresponding to the position of the road carried by the first image is rendered to the three-dimensional position corresponding to the road in the second image based on the optimal posture transformation matrix, and finally a three-dimensional image of the road with color is obtained. The three-dimensional image of the road is as shown in FIG. Figure 3 As shown in b. Figure 3 a is a three-dimensional image of a road with color obtained by the electronic device 10 without using the optimal posture transformation matrix.

[0045] The above technical solution determines the first vector of the first feature point of the object in the first image and the second vector of the second feature point of the object in the second image by acquiring the first image and the second image. Then, based on the first vector and the second vector, a set of feature matching relationships between the first image and the second image at a certain position is obtained, and each set of feature matching relationships can be quickly determined based on the first vector and the second vector, so as to more quickly obtain the optimal posture transformation matrix of the object based on multiple sets of feature matching relationships at multiple positions, and complete the external parameter calibration of the electronic device. Finally, the object image of the object is obtained based on the first image, the second image and the optimal posture transformation matrix, so that the object image of the object is more consistent with the real object.

[0046] In some embodiments, determining a first vector of a first feature point of an object in a first image and a second vector of a second feature point of an object in a second image includes:

[0047] Step S201: extracting a first edge segment of an object in a first image and a second edge segment of an object in a second image.

[0048] See also Figure 4 and Figure 5 After acquiring the first image and the second image, the first edge line segment of the object in the first image is extracted according to the feature extraction algorithm in the pixel coordinate system (such as Figure 4 The edge line segment AB shown as the mark composed of E pixels) and the second edge line segment of the object in the second image (such as Figure 5 The mark shown is an edge line segment MN composed of F pixels).

[0049] Step S202: Obtain a first direction and a first change intensity according to the grayscales of pixels on both sides of the first edge segment.

[0050] The first image is grayed out, and the color depth of the object can be converted into the gray scale. The lighter the color, the larger the gray scale value. The gray scale change from one side of the first edge line segment AB to the other side is converted into the first direction of the first edge line segment AB. In one embodiment, Figure 4 As shown, obtaining the first direction and the first change intensity according to the grayscale of the pixels on both sides of the first edge segment includes the following steps:

[0051] Step S2021: determine a first side edge and a second side edge on both sides of the first edge segment.

[0052] The first side may be one or more pixel segments on one side adjacent to the first edge line segment AB, each pixel segment is composed of a plurality of pixels along the first edge line segment, such as two pixel segments marked as 1 are the first side. The second side may be one or more pixel segments on the other side adjacent to the first edge line segment AB, each pixel segment is composed of a plurality of pixels along the first edge line segment, such as two pixel segments marked as 2 are the second side.

[0053] Step S2022: Determine a first grayscale mean value of pixels on the first side and a second grayscale mean value of pixels on the second side.

[0054] The first grayscale mean is the sum of the grayscale values ​​of all pixels on the first side divided by the number of all pixels, such as adding the grayscale values ​​of 8 pixels marked as 1 and dividing by 8 to obtain the first grayscale mean of the first side. The second grayscale mean is the sum of the grayscale values ​​of all pixels on the second side divided by the number of all pixels, such as adding the grayscale values ​​of 8 pixels marked as 2 and dividing by 8 to obtain the second grayscale mean of the second side.

[0055] Step S2023: Obtain a first orientation according to the first grayscale mean value and the second grayscale mean value.

[0056] The first direction may be a direction from a smaller grayscale mean to a larger grayscale mean. Since the first grayscale mean is smaller than the second grayscale mean, the first direction is from the first side to the second side.

[0057] Step S2024: Obtain a first change intensity according to a difference between the first grayscale mean and the second grayscale mean.

[0058] It can be understood that the first change intensity can be converted into the size of the vector.

[0059] Step S203: Obtain a second direction and a second change intensity according to the grayscales of pixels on both sides of the second edge segment.

[0060] Similarly, the grayscale change from one side to the other side of the second edge segment MN is converted into a second orientation. In one embodiment, Figure 5 As shown, obtaining the second orientation according to the grayscale of pixels on both sides of the second edge segment includes the following steps:

[0061] Step S2031: Determine a third side and a fourth side on both sides of the second edge segment.

[0062] The third side may be one or more pixel segments on one side adjacent to the second edge line segment MN, each pixel segment is composed of multiple pixels along the first edge line segment, such as two pixel segments marked as 1 are the third side. The fourth side may be one or more pixel segments on the other side adjacent to the second edge line segment MN, each pixel segment is composed of multiple pixels along the first edge line segment, such as two pixel segments marked as 2 are the fourth side.

[0063] Step S2032: Determine a third grayscale mean value of pixels on the third side and a fourth grayscale mean value of pixels on the fourth side.

[0064] The third grayscale mean is the sum of the grayscale values ​​of all pixels on the first side divided by the number of all pixels, such as adding the grayscale values ​​of 8 pixels marked as 1 and dividing by 8 to obtain the third grayscale mean of the third side. The fourth grayscale mean is the sum of the grayscale values ​​of all pixels on the second side divided by the number of all pixels, such as adding the grayscale values ​​of 8 pixels marked as 2 and dividing by 8 to obtain the fourth grayscale mean of the fourth side.

[0065] Step S2033: Obtain a second orientation according to the third grayscale mean value and the fourth grayscale mean value.

[0066] The second direction may be a direction from a smaller grayscale mean to a larger grayscale mean. Since the third grayscale mean is smaller than the fourth grayscale mean, the second direction is from the third side to the fourth side.

[0067] Step S2034: Obtain a second change intensity according to the difference between the third grayscale mean and the fourth grayscale mean.

[0068] It can be understood that the second change intensity can be converted into the magnitude of a vector.

[0069] Step S204: Obtain a first vector of a first feature point of the first edge segment according to the first perpendicular line, the first change intensity and the first orientation of the first edge segment.

[0070] like Figure 4 As shown, the center of the first edge segment AB is the vertical point, the vertical line perpendicular to the first edge segment AB is the first vertical line, and the first edge segment AB can be represented by the first straight line equation. It can be understood that the slope of the first vertical line can be converted by the slope of the first straight line equation.

[0071] Thus, the first vector of the first feature point If the first direction is consistent with the first perpendicular line, the direction of the first vector is the first direction, and the first vector The magnitude of can be represented by the first change intensity, the first vector A certain position of can be obtained by the first straight line equation.

[0072] Step S205: Obtain a second vector of a second feature point of the second edge segment according to the second perpendicular line, the second change intensity and the second orientation of the second edge segment.

[0073] like Figure 5 As shown, the center of the second edge segment MN is the vertical point, the vertical line perpendicular to the second edge segment MN is the second vertical line, and the second edge segment MN can be represented by the second straight line equation. It can be understood that the slope of the second vertical line can be converted from the slope of the second straight line equation.

[0074] Thus, the second vector of the second feature point The second direction is consistent with the second perpendicular line, the direction of the second vector is the second direction, the magnitude of the second vector can be represented by the second change intensity, and the second position of the second vector can be obtained from the second straight line equation.

[0075] It can be understood that in extracting image features, edge segment features are extracted (the first feature point of the first edge segment and the second feature point of the second edge segment are extracted), and the direction and size of the vector of the feature point are converted according to the direction of the edge segment (the first direction and the second direction) and the grayscale change of the pixels on both sides of the edge segment (the first change intensity and the second change intensity), so that the direction and size of the feature point have direction information and change intensity values, which is conducive to enhancing the feature matching of objects in the first image and the second image.

[0076] In some embodiments, obtaining a set of feature matching relationships between the first image and the second image at a certain position based on the first vector and the second vector includes the following steps:

[0077] Step S301: placing the first feature point and the second feature point in an image coordinate system.

[0078] After calculating the first vector of the first feature point and the second vector of the second feature point, the first vector and the second vector are placed in the same image coordinate system.

[0079] Step S302: determining a plurality of angles between a first vector of each first feature point and a second vector of a plurality of second feature points.

[0080] The first vector includes a first direction and a first orientation, and the second vector includes a second direction and a second orientation. According to the first direction and the first orientation and the second direction and the second orientation, the angle between the first feature point and the second feature point can be determined.

[0081] Step S303: Determine a matching point from a plurality of second feature points according to a plurality of included angles and angle thresholds. In one embodiment, the method includes the following steps:

[0082] S3031: Based on the first vector and the second vector, determine the distance between each first feature point and multiple second feature points.

[0083] The first vector includes a first position, and the second vector includes a second position. The distance between each first feature point and multiple second features within a preset range can be calculated based on a certain position and the second position. The preset range refers to a specified radius range centered on a certain first feature point.

[0084] S3032: sort the multiple distances from near to far, and compare the multiple angles and angle thresholds in sequence according to the sorting results.

[0085] The angle threshold can be set according to actual conditions. In one embodiment, if the angle is less than the angle threshold, the second feature point corresponding to the angle is determined to be the matching point. If the angle is greater than or equal to the angle threshold, continue to compare the next angle in the sorting result until the distance between the first feature point and the second feature is greater than a preset range, then abandon the first feature point, or if multiple angles are greater than or equal to the angle threshold, abandon the first feature points corresponding to multiple angles.

[0086] In one example, the second feature points within a preset range from the first feature point t1 are t2a, t2b, t2c, and t2d. And the distances from the first feature point t1 from near to far are t2a, t2b, t2c, and t2d. Then, first compare the first angle and the angle threshold between the first feature point t1 and the second feature point t2a. If the first angle is less than the angle threshold, determine that the second feature point t2a is a matching point and stop comparing. If the first angle is greater than or equal to the angle threshold, then compare the second angle and the angle threshold between the first feature point t1 and the second feature point t2b. If the second angle is less than the angle threshold, determine that the second feature point t2b is a matching point and stop comparing. If the second angle is greater than or equal to the angle threshold, then compare the second angle and the angle threshold between the first feature point t1 and the third feature point t2c, and so on, until the fourth feature point t2d is compared.

[0087] Step S304: obtaining a matching relationship according to the first vector of each first feature point and the second vector of the matching point.

[0088] Step S305: obtaining a set of feature matching relationships between the first image and the second image at a certain position according to all the matching relationships.

[0089] After obtaining the first feature point and the corresponding matching point, a matching relationship can be calculated according to the first vector of the first feature point and the second vector of the matching point. Multiple first feature points on the first image basically search for matching points at the same time, so that a set of feature matching relationships between multiple first feature points on the first image and multiple second feature points on the second image can be obtained.

[0090] It can be understood that the embodiment of the present application can provide a direct optimization target for determining the optimal posture transformation matrix by performing feature matching based on the first direction, first orientation and a certain position of the first feature point of the first edge segment of the extracted first image, and the second direction, second orientation and second position of the second feature point of the second edge segment of the second image. Compared with the algorithms of related technologies, the probability of false matching is reduced, the convergence speed of calculating the optimal posture transformation matrix is ​​accelerated, and the accuracy of the optimal posture transformation matrix is ​​improved. In image matching with grid features, the convergence speed of calculating the optimal posture transformation matrix is ​​faster, and the accuracy of the optimal posture transformation matrix is ​​further improved.

[0091] In some embodiments, obtaining an optimal posture transformation matrix of an object according to multiple sets of matching relationships at multiple positions includes the following steps:

[0092] Step S401: moving the first image and the second image.

[0093] Step S402: Determine multiple sets of feature matching relationships between the first image and the second image at multiple locations.

[0094] Step S403: obtaining multiple matching degrees and posture transformation matrices corresponding to the multiple matching degrees according to the multiple sets of feature matching relationships.

[0095] Accumulating all the matching relationships of a set of feature matching relationships can obtain a pose transformation matrix. In an example, the formula is as follows:

[0096]

[0097] in, Indicates the first position of the first feature point having a matching relationship. The second position of the second feature point having the matching relationship is indicated. The matching relationship of all numbers can obtain a pose transformation matrix.

[0098] At the same time, a matching degree can be obtained by accumulating all matching relationships in a set of feature matching relationships. In one example, in a set of feature matching relationships, a matching degree can be obtained based on the first vector of the first feature point, the second vector of the second feature point, and algorithms such as Euclidean distance. The matching degree is used to measure the overlap of objects in the first image and the second image at a certain position. The greater the matching degree, the higher the overlap of the objects.

[0099] Step S404: Determine the posture transformation matrix corresponding to the maximum matching degree among multiple matching degrees as the optimal posture transformation matrix.

[0100] It can be understood that the higher the matching degree, the higher the accuracy of the pose transformation matrix.

[0101] See also Figure 6 , Figure 6 A schematic diagram of the structure of an electronic device 10 provided in an embodiment of the present application. In one embodiment, the electronic device 10 includes a memory 13 and at least one processor 14. Those skilled in the art should understand that Figure 6 The structure of the electronic device 10 shown does not constitute a limitation of the embodiments of the present application. The electronic device 10 may also include more or less other hardware or software than shown in the figure, or a different component arrangement.

[0102] As an optional implementation, the electronic device 10 includes a terminal that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits, programmable gate arrays, digital processors, and embedded devices. As an optional implementation, the memory 13 is used to store computer programs and various data. The memory 13 may include any other computer-readable medium that can be used to carry or store data, such as a read-only memory (ROM), a random access memory (RAM), or a programmable read-only memory (PROM).

[0103] As an optional implementation, the at least one processor 14 may include an integrated circuit, for example, a single packaged integrated circuit, or a plurality of integrated circuits with the same or different functions, including a combination of a microprocessor, a digital processing chip, a graphics processor, and various control chips. The at least one processor 14 is the control core (5ontrol unit) of the electronic device 10, and executes various functions and processes data of the electronic device 10 by running or executing programs or modules stored in the memory 13, and calling data stored in the memory 13. The memory 13 stores a computer program, and the at least one processor 14 can call the computer program stored in the memory 13 to execute and implement the above-mentioned image processing method.

[0104] The present application also provides a storage medium, wherein the storage medium stores computer instructions, and when the instructions are executed on a computing device, the computing device can execute the image processing method and the image processing method provided in the above-mentioned embodiment.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. An image processing method, characterized in that: include: Acquire a first image and a second image of the object; Determine a first vector of a first feature point of the object in the first image, and a second vector of a second feature point of the object in the second image; Based on the first vector and the second vector, obtaining a set of feature matching relationships between the first image and the second image at a certain position; Obtaining an optimal posture transformation matrix of the object according to a plurality of sets of feature matching relationships located at a plurality of positions; An object image of the object is obtained according to the first image, the second image and the optimal posture transformation matrix.

2. The image processing method according to claim 1, characterized in that: The determining of a first vector of a first feature point of the object in the first image and a second vector of a second feature point of the object in the second image comprises: In the pixel coordinate system, Extracting a first edge line segment of the object in the first image and a second edge line segment of the object in the second image; Obtaining a first direction and a first change intensity according to the grayscales of pixels on both sides of the first edge segment, and obtaining a second direction and a second change intensity according to the grayscales of pixels on both sides of the second edge segment; Obtaining the first vector of the first feature point of the first edge segment according to the first perpendicular line of the first edge segment, the first orientation, and the first change intensity; The second vector of the second feature point of the second edge segment is obtained according to the second perpendicular line of the second edge segment, the second orientation, and the second change intensity.

3. The image processing method according to claim 2, characterized in that: The obtaining of a first direction and a first change intensity according to the grayscales of pixels on both sides of the first edge segment, and the obtaining of a second direction and a second change intensity according to the grayscales of pixels on both sides of the second edge segment, include: Determine a first grayscale mean value of pixels on a first side and a second grayscale mean value of pixels on a second side of the first edge segment; Obtaining the first orientation and the first change intensity according to the first grayscale mean value and the second grayscale mean value; Determine a third grayscale mean value of pixels on a third side of the second edge segment and a fourth grayscale mean value of pixels on a fourth side; The second orientation and the second change intensity are obtained according to the third grayscale mean value and the fourth grayscale mean value.

4. The image processing method according to claim 3, characterized in that: The obtaining the first orientation according to the first grayscale mean value and the second grayscale mean value includes: Comparing the first grayscale mean value and the second grayscale mean value, obtaining the first direction from the first side of the smaller first grayscale mean value to the second side of the larger second grayscale mean value; The obtaining the second orientation according to the third grayscale mean value and the fourth grayscale mean value comprises: The third grayscale mean value and the fourth grayscale mean value are compared to obtain the second direction from the third side with the smaller third grayscale mean value to the fourth side with the larger third grayscale mean value.

5. The image processing method according to claim 3, characterized in that: The obtaining the first change intensity according to the first grayscale mean value and the second grayscale mean value includes: Obtaining the first change intensity according to a difference between the first grayscale mean and the second grayscale mean; The obtaining the second change intensity according to the third grayscale mean value and the fourth grayscale mean value comprises: The second change intensity is obtained according to the difference between the third grayscale mean and the fourth grayscale mean.

6. The image processing method according to claim 1, characterized in that: The obtaining, based on the first vector and the second vector, a set of feature matching relationships between the first image and the second image at a certain position includes: In the image coordinate system, Obtain multiple angles between the first vector of each of the first feature points and the second vectors of multiple second feature points; Determine a matching point among the plurality of second feature points according to the plurality of included angles and angle thresholds; Obtain a matching relationship according to the first vector of each of the first feature points and the second vector of the matching point; A set of feature matching relationships between the first image and the second image at a certain position is obtained based on all the matching relationships.

7. The image processing method according to claim 6, characterized in that: The step of determining a matching point from among the plurality of second feature points according to the plurality of included angles and angle thresholds comprises: Based on the first vector and the second vector, determining the distance between each of the first feature points and a plurality of the second feature points; Sort the multiple distances from near to far, and compare the multiple angles with the angle threshold in sequence according to the sorting results; If the multiple included angles are all greater than or equal to the angle threshold, abandoning the first feature points corresponding to the multiple included angles; If the angle is greater than or equal to the angle threshold, the second feature point corresponding to the angle is determined as the matching point.

8. The image processing method according to claim 7, characterized in that: The distance between the first feature point and the second feature point is within a preset range.

9. The image processing method according to claim 1, characterized in that: The step of obtaining the optimal posture transformation matrix of the object according to the plurality of sets of feature matching relationships located at a plurality of positions includes: moving the first image and the second image multiple times; Determining a plurality of sets of feature matching relationships between the first image and the second image located at the plurality of positions; Obtaining multiple matching degrees according to multiple groups of feature matching relationships; Determine the posture transformation matrix corresponding to the maximum matching degree among the multiple matching degrees as the optimal posture transformation matrix.

10. An electronic device, characterized in that: include: a processor suitable for executing a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the image processing method according to any one of claims 1 to 9 is executed.

Images