Image distortion correction method and device, electronic equipment and vehicle

Abstract

The invention relates to an image distortion correction method and device, electronic equipment and a vehicle, and the method comprises the steps: obtaining a first mapping relation and a second mapping relation; wherein the first mapping relation is a transformation relation between a standard image corresponding to the central eye site and visual angle distortion images corresponding to other eye sites, and the second mapping relation is a transformation relation between the standard image corresponding to the central eye site and an actual image corresponding to the central eye site; determining a full distortion correction mapping relation according to the first mapping relation and the second mapping relation; and performing image distortion correction by using the full distortion correction mapping relation.

Description

Technical Field

[0001] The present disclosure relates to image correction technology, and more specifically, to an image distortion correction method, device, electronic equipment and vehicle. Background Art

[0002] With the development of automobile intelligence technology, AR-HUD (Augmented Reality Head-Up Display) technology as an advanced driving assistance system has gradually become a research hotspot in the automotive field. AR-HUD can integrate navigation information, ADAS (Advanced Driver Assistance Systems) and other information with the road ahead, displaying turn instructions, obstacle warnings, lane departure, front vehicle warning, blind spot monitoring and other content, providing drivers with more comprehensive, clear and intuitive driving assistance information.

[0003] In practical applications, AR-HUD technology is affected by image distortion, which directly affects the imaging quality of AR-HUD, causing visual fatigue and even dizziness to users, endangering the driver's driving safety. Therefore, the research on AR-HUD image distortion correction technology has important practical significance and application value.

[0004] In the prior art, there are two main types of AR-HUD image distortion correction solutions, one is hardware correction and the other is software correction. Hardware correction reduces distortion through special optical lens design or optimization of the HUD back-end optical system structure. Software correction uses computer vision and image processing technology to analyze and calculate the distortion of the generated image, and then corrects the image through an algorithm so that the final displayed image can conform to the visual habits of the human eye. The above correction method can only correct a single type of distortion. In addition, the process of calibrating distortion is too cumbersome and complicated, and the calibration scheme has low flexibility.

[0005] Therefore, it is necessary to provide a new technical solution to meet the correction of various types of image distortion. Summary of the invention

[0006] An object of the present invention is to provide a new technical solution for an image distortion correction method.

[0007] According to a first aspect of the present invention, there is provided an image distortion correction method, comprising:

[0008] Obtaining a first mapping relationship and a second mapping relationship; wherein the first mapping relationship is a transformation relationship between a standard image corresponding to a central eye point and perspective distortion images corresponding to other eye points, and the second mapping relationship is a transformation relationship between a standard image corresponding to a central eye point and an actual image corresponding to the central eye point;

[0009] Determining a total distortion correction mapping relationship according to the first mapping relationship and the second mapping relationship;

[0010] The image distortion correction is performed using the full distortion correction mapping relationship.

[0011] Optionally, the method further includes:

[0012] Obtain a standard image corresponding to the central eye position and a perspective distortion image corresponding to each other eye position;

[0013] Extracting first corner point information from the standard image corresponding to the central eye point, and extracting second corner point information from the perspective distortion images corresponding to the other eye points respectively;

[0014] A corresponding first mapping relationship is determined according to the first corner point information and each second corner point information.

[0015] Optionally, the method further includes:

[0016] Acquire projection parameters; wherein the projection parameters include the three-dimensional size of the eye box, the virtual image projection distance, the virtual image imaging size and the lower field of view angle;

[0017] Constructing a simulation model according to the projection parameters;

[0018] The first mapping relationship is determined using the simulation model.

[0019] Optionally, the determining the first mapping relationship by using the simulation model includes:

[0020] Determining the number and positions of eye sites according to a preset eye site density;

[0021] Using virtual cameras placed at the locations of the eye sites, a standard image corresponding to the central eye site and perspective distortion images corresponding to the other eye sites are captured;

[0022] Extracting first corner point information from the standard image corresponding to the central eye point, and extracting second corner point information from the perspective distortion images corresponding to the other eye points respectively;

[0023] A corresponding first mapping relationship is determined according to the first corner point information and each second corner point information.

[0024] Optionally, the first corner point information includes position information of multiple first corner points, and the second corner point information includes position information of multiple second corner points; wherein determining the corresponding first mapping relationship according to the first corner point information and each second corner point information includes:

[0025] Based on each other eye point, determine a one-to-one correspondence between the first corner point and the second corner point;

[0026] Determine a position transformation relationship between each first corner point and the corresponding second corner point according to the position information of each first corner point, the position information of each second corner point, and the one-to-one correspondence between the first corner point and the second corner point;

[0027] The first mapping relationship is determined according to the position transformation relationship between each first corner point and the corresponding second corner point.

[0028] Optionally, the method further includes:

[0029] Obtaining a standard image corresponding to the central eye position and an actual image corresponding to the central eye position;

[0030] Extracting first corner point information from the standard image corresponding to the central eye position, and extracting third corner point information from the actual image corresponding to the central eye position;

[0031] The second mapping relationship is determined according to the first corner point information and the third corner point information.

[0032] Optionally, the first corner point information includes position information of multiple first corner points, and the third corner point information includes position information of multiple third corner points; wherein determining the second mapping relationship according to the first corner point and the third corner point includes:

[0033] Determine a one-to-one correspondence between the first corner point and the third corner point;

[0034] Determine a position transformation relationship between each first corner point and the corresponding third corner point according to the position information of each first corner point, the position information of each third corner point, and the one-to-one correspondence between the first corner point and the third corner point;

[0035] The second mapping relationship is determined according to the position transformation relationship between each first corner point and the corresponding third corner point.

[0036] Optionally, the total distortion correction mapping relationship includes total distortion correction mapping relationships corresponding to different eye loci, wherein the performing image distortion correction using the total distortion correction mapping relationship further includes:

[0037] Obtaining the eye position corresponding to the distorted image to be corrected;

[0038] According to the eye position corresponding to the distorted image to be corrected, searching for a target mapping relationship from the full distortion correction mapping relationship;

[0039] The distorted image to be corrected is corrected using the target mapping relationship.

[0040] According to a second aspect of the present invention, there is provided an image distortion correction device, comprising:

[0041] An acquisition module, used to acquire a first mapping relationship and a second mapping relationship; wherein the first mapping relationship is a transformation relationship between a standard image corresponding to a central eye point and perspective distortion images corresponding to other eye points, and the second mapping relationship is a transformation relationship between a standard image corresponding to a central eye point and an actual image corresponding to the central eye point;

[0042] A total distortion correction mapping relationship determination module, configured to determine a total distortion correction mapping relationship according to the first mapping relationship and the second mapping relationship;

[0043] The correction module is used to perform image distortion correction using the full distortion correction mapping relationship.

[0044] According to a third aspect of the present invention, there is provided an image distortion correction device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is used to control the processor to operate to execute the image distortion correction method according to any one of the first aspect of the present invention.

[0045] According to a fourth aspect of the present invention, an electronic device is provided, comprising the image distortion correction device as described in the second aspect or the third aspect of the present invention.

[0046] According to a fifth aspect of the present invention, there is provided a vehicle, comprising the electronic device according to the fourth aspect of the present invention.

[0047] The image distortion correction method provided by the present invention obtains a more comprehensive full distortion correction mapping relationship by fusing the transformation relationship between the standard image corresponding to the central eye point and the perspective distortion images corresponding to other eye points and the transformation relationship between the standard image corresponding to the central eye point and the actual image corresponding to the central eye point. The full distortion correction mapping relationship is used to correct the distorted image to be corrected, thereby improving the correction accuracy and enhancing the user's visual experience.

[0048] Features and advantages of the embodiments of the present specification will become apparent from the following detailed description of exemplary embodiments of the present specification with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the specification and, together with the description, serve to explain the principles of the embodiments of the specification.

[0050] Figure 1 is a processing flow chart of an image distortion correction method according to an embodiment of the present invention.

[0051] Figure 2 It is a schematic diagram of a standard image corresponding to a central eye position and perspective distorted images corresponding to other eye positions under dynamic distortion according to an embodiment of the present invention.

[0052] Figure 3 Schematic diagram of a standard image and an actual image corresponding to a central eye position under static distortion according to an embodiment of the present invention.

[0053] Figure 4 It is a schematic diagram of a standard image corresponding to a central eye position and actual images corresponding to other eye positions under full distortion according to an embodiment of the present invention.

[0054] Figure 5 It is a schematic diagram of an actual image corresponding to an eye position according to an embodiment of the present invention.

[0055] Figure 6 is a processing flow chart of an image distortion correction method according to an embodiment of the present invention.

[0056] Figure 7 4 is a principle block diagram of an image distortion correction device according to an embodiment of the present invention.

[0057] Figure 8 4 is a schematic diagram of the hardware structure of an image distortion correction device according to an embodiment of the present invention.

[0058] Fig. 9 is a principle block diagram of an electronic device according to an embodiment of the present invention.

[0059] Fig.10 is a functional block diagram of a vehicle according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] Various exemplary embodiments of the present specification will now be described in detail with reference to the accompanying drawings.

[0061] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the embodiments of the present specification and its application or uses.

[0062] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0063] In one embodiment of the present invention, a method for correcting image distortion is provided. Figure 1 As shown, the image distortion correction method of this embodiment may include the following steps S110 to S130.

[0064] Step S110, obtaining a first mapping relationship and a second mapping relationship; wherein the first mapping relationship is a transformation relationship between a standard image corresponding to the central eye point and perspective distortion images corresponding to other eye points, and the second mapping relationship is a transformation relationship between the standard image corresponding to the central eye point and an actual image corresponding to the central eye point.

[0065] The eye locus is the midpoint of the line connecting the centers of the pupils of both eyes of the driver. The eye box is a visual space range corresponding to the complete image that the driver can see. When the eye locus is within the visual space range, the driver can see the complete image. When the eye locus is beyond the visual space range, the driver cannot see the complete image.

[0066] See also Figure 2 , Figure 3 and Figure 4 , the eye box is a cuboid. Take any point in the eye box as the origin O 1 , establish a three-dimensional coordinate system O along the length, width and height directions of the eye box 1 -UVK. The position information of the eye locus can be expressed using this three-dimensional coordinate system.

[0067] When the eye point is the central eye point, the standard image corresponding to the central eye point has no perspective distortion. When the eye point is any eye point except the central eye point, the actual image corresponding to the other eye points will have perspective distortion. This type of distortion is dynamic distortion, which is caused by the movement of the driver's eyes.

[0068] Due to the influence of hardware factors such as equipment installation errors, unreasonable optical system design and inherent characteristics of the display device, the actual image corresponding to the central eye point will also be distorted. This type of distortion is static distortion.

[0069] The first mapping relationship is a mapping function, which represents the transformation relationship between the standard image corresponding to the central eye point and the perspective distortion images corresponding to other eye points.

[0070] The second mapping relationship is a mapping function, which represents the transformation relationship between the standard image corresponding to the central eye point and the actual image corresponding to the central eye point.

[0071] Step S120: determining a total distortion correction mapping relationship according to the first mapping relationship and the second mapping relationship.

[0072] The mapping relationship of the full distortion correction is a mapping function. This mapping function combines the above two types of distortion (dynamic distortion and static distortion), representing the transformation relationship between the standard image corresponding to the central eye point and the distorted images corresponding to other eye points. The reasons for the distortion of the distorted images corresponding to other eye points here include not only the movement of the driver's eyes, but also the influence of the above hardware factors.

[0073] Step S130, performing image distortion correction using the full distortion correction mapping relationship.

[0074] The image distortion correction method provided by the embodiment of the present invention obtains a more comprehensive full distortion correction mapping relationship by fusing the transformation relationship between the standard image corresponding to the central eye point and the perspective distortion images corresponding to other eye points and the transformation relationship between the standard image corresponding to the central eye point and the actual image corresponding to the central eye point. The full distortion correction mapping relationship is used to correct the distorted image to be corrected, thereby improving the correction accuracy and enhancing the user's visual experience.

[0075] In one embodiment, before step S110, the image distortion correction method further includes: obtaining projection parameters; wherein the projection parameters include the three-dimensional size of the eye box, the virtual image projection distance, the virtual image imaging size and the lower field of view angle; constructing a simulation model for determining according to the projection parameters; and determining the first mapping relationship using the simulation model.

[0076] The simulation model is used to combine factors such as the physical properties of the optical system, the display characteristics of the display screen, and the perception characteristics of the human visual system, and can truly reflect the actual operation of the AR-HUD.

[0077] Determining the first mapping relationship using the simulation model specifically includes: determining the number and position of the eye sites according to a preset eye site density; using a virtual camera placed at the location of each eye site to capture a standard image corresponding to the central eye site and perspective distortion images corresponding to other eye sites; extracting first corner point information from the standard image corresponding to the central eye site, and respectively extracting second corner point information from perspective distortion images corresponding to other eye sites; and determining the corresponding first mapping relationship based on the first corner point information and each second corner point information.

[0078] The virtual camera has the same parameters and characteristics as the real camera. The virtual camera placed at each eye locus is used to simulate the actual image observed when the midpoint of the line connecting the pupil centers of both eyes of the observer is at each eye locus.

[0079] It should be noted that the premise of using the simulation model to determine the first mapping relationship is that there is no influence of the above-mentioned hardware factors.

[0080] In one embodiment, the simulation model is also used to increase environmental noise. The environmental noise can simulate various interference factors that may occur in actual use scenarios, such as light changes, temperature changes, vibrations, etc., thereby improving the authenticity and reliability of the simulation model.

[0081] The first mapping relationship is determined by using a simulation model, which realizes the automation of determining the first mapping relationship, improves the accuracy of determining the first mapping relationship, no longer requires manpower input, and reduces costs. In addition, only projection parameters need to be given to construct a corresponding simulation model, and then the simulation model is used to determine the first mapping relationship, which has strong flexibility and portability.

[0082] In one embodiment, before step S110, the image distortion correction method also includes: obtaining a standard image corresponding to the central eye site and a perspective distortion image corresponding to each other eye site; extracting first corner point information from the standard image corresponding to the central eye site, and respectively extracting second corner point information from the perspective distortion images corresponding to each other eye site; and determining a corresponding first mapping relationship based on the first corner point information and each second corner point information.

[0083] In this embodiment, the standard image corresponding to the central eye point is taken by a virtual camera placed at the central eye point in the simulation model, that is, the input original image. The perspective distortion images corresponding to other eye points are taken by virtual physical cameras at different eye point positions.

[0084] In one embodiment, a Harris corner detection algorithm is used to extract first corner point information from a standard image corresponding to a central eye point, and second corner point information is respectively extracted from perspective distortion images corresponding to other eye points.

[0085] Harris corner detection algorithm detects corners by observing the grayscale changes of a local window. Specifically, a fixed-size window is set to slide in the image, and the grayscale value change of the pixel [x, y] is:

[0086] E(Δx,Δy)=∑ x,y w(x,y)[I(x+Δx,y+Δy)-I(x,y)] 2 — (1)

[0087] Among them, E(Δx,Δy) is the gray value change caused by the pixel point [x,y], [Δx,Δy] is the window offset, I(x,y) is the gray value at the pixel point (x,y), w(x,y) represents the window function, and a Gaussian function can be used as the window function.

[0088] Expanding formula (1) using Taylor's formula and ignoring the minimum term, we can obtain the following formula (2):

[0089]

[0090] Among them, Then the calculation formula (2) can be simplified as:

[0091]

[0092] Among them, the calculation formula (2) I x and I y are the grayscale gradients of the pixel in the x and y directions respectively, and the matrix M is the structural tensor at the pixel point (x, y). The eigenvalue of the matrix M reflects the degree of change of the pixel grayscale in the two main axis directions. When the eigenvalue of the matrix M exceeds the preset threshold, it can be determined that the pixel point has obvious grayscale changes in multiple directions and the pixel point is determined to be a potential corner point.

[0093] The corner response function R used to quantify the likelihood of each pixel being a corner is:

[0094] R = Det(M) - k(trace(M)) 2 — (4)

[0095] Where Det(M) represents the determinant of the matrix M, trace(M) is the trace of the matrix M, and the value range of k is [0.05, 0.15]. When the R value exceeds the preset threshold, the pixel is determined to be a corner point.

[0096] After obtaining a pixel-level corner point using the Harris corner detection algorithm, the edge features of the neighborhood of the pixel-level corner point are obtained, and the pixel-level corner point is adjusted and expanded with sub-pixel accuracy using the edge intersection method.

[0097] In one embodiment, taking a checkerboard image as an input image, the standard image corresponding to the central eye position and the perspective distortion images corresponding to other eye positions are checkerboard images under different perspectives. Figure 5 The checkerboard image shown in the figure is the standard image corresponding to the central eye position. Using the Harris corner detection algorithm, all corners in the checkerboard image can be determined. Each corner point is located in the white circle and is at the intersection of four adjacent squares.

[0098] In one embodiment, the first corner point information includes position information of multiple first corner points, and the second corner point information includes position information of multiple second corner points. The above-mentioned determining the corresponding first mapping relationship based on the first corner point information and each second corner point information includes: determining a one-to-one correspondence between the first corner point and the second corner point; determining a position transformation relationship between each first corner point and the corresponding second corner point based on the position information of each first corner point, the position information of each second corner point, and the one-to-one correspondence between the first corner point and the second corner point; determining the first mapping relationship based on the position transformation relationship between each first corner point and the corresponding second corner point.

[0099] The number of first corner points determined based on the standard image corresponding to the central eye position is equal to the number of second corner points determined based on the perspective distortion image corresponding to any other eye position. For each first corner point, there is a corresponding second corner point. Such a one-to-one correspondence means that these two corner points correspond to the same point in the input image.

[0100] Figure 2 The standard image corresponding to the central eye position and the perspective distortion images corresponding to the four other eye positions are shown.

[0101] See also Figure 2 The position information of the first corner point is the lower left corner vertex of the standard image corresponding to the central eye position as the coordinate origin O 2 , a two-dimensional coordinate system O is established along the length and width directions of the standard image 2 -XY position information. The position information of the second corner point is the position information in a two-dimensional coordinate system established along the length and width directions of the perspective distortion image, taking the lower left corner vertex of the perspective distortion image corresponding to another eye point as the coordinate origin.

[0102] The coordinate set of the first corner point in the standard image corresponding to the central eye position is Indicates that the coordinate set of the second corner point in the perspective distortion image corresponding to any other eye position is B p (X,Y). The corresponding relationship between the coordinates of the first corner point and the corresponding coordinates of the second corner point is as follows:

[0103]

[0104] Among them, T is the mapping function corresponding to the viewing angle distortion caused by the change of the eye point position, representing a perspective transformation relationship.

[0105] See also Figure 2 , the coordinate sets of the second corner points in the perspective distortion image corresponding to the other four eye positions are respectively B pk-(m+n)The position transformation relationship between the second corner point coordinate set in the perspective distortion image corresponding to each other eye point and the first corner point coordinate set in the standard image corresponding to the central eye point can be expressed by the above calculation formula (5).

[0106] In one embodiment, the method for correcting image distortion also includes: obtaining a standard image corresponding to the central eye position and an actual image corresponding to the central eye position; extracting first corner point information from the standard image corresponding to the central eye position, and extracting third corner point information from the actual image corresponding to the central eye position; determining a second mapping relationship based on the first corner point information and the third corner point information.

[0107] The actual image corresponding to the central eye position is actually captured by a camera placed at the position where the central eye position is located. The standard image corresponding to the central eye position is a pre-stored image and can be directly obtained.

[0108] In one embodiment, the Harris corner detection algorithm is used to extract the first corner point information from the standard image corresponding to the central eye position, and the third corner point information is extracted from the actual image corresponding to the central eye position. The specific corner point detection algorithm can refer to the above embodiment, and will not be described in detail here.

[0109] After obtaining a pixel-level corner point using the Harris corner detection algorithm, the edge features of the neighborhood of the pixel-level corner point are obtained, and the pixel-level corner point is adjusted and expanded with sub-pixel accuracy using the edge intersection method.

[0110] In one embodiment, the first corner point information includes position information of multiple first corner points, and the third corner point information includes position information of multiple third corner points. The above-mentioned determining the second mapping relationship based on the first corner points and the third corner points includes: determining a one-to-one correspondence between the first corner points and the third corner points; determining a position transformation relationship between each first corner point and the corresponding third corner point based on the position information of each first corner point, the position information of each third corner point, and the one-to-one correspondence between the first corner point and the third corner point; determining the second mapping relationship based on the position transformation relationship between each first corner point and the corresponding third corner point.

[0111] The number of first corner points determined based on the standard image corresponding to the central eye position and the number of third corner points determined based on the actual image corresponding to the central eye position are equal. For each first corner point, there is a corresponding third corner point. This one-to-one correspondence means that these two corner points correspond to the same point in the input image.

[0112] Figure 3 The standard image corresponding to the central eye position and the actual image corresponding to the central eye position are shown.

[0113] See also Figure 3The position information of the first corner point is based on the lower left corner vertex of the standard image corresponding to the central eye position as the coordinate origin O 2 , a two-dimensional coordinate system O is established along the length and width directions of the standard image 2 - XY position information. The position information of the third corner point is the position information in a two-dimensional coordinate system established along the length and width directions of the actual image, with the lower left corner vertex of the actual image corresponding to the central eye position as the coordinate origin.

[0114] The coordinate set of the first corner point in the standard image corresponding to the central eye position is Indicates that the coordinate set of the third corner point in the actual image corresponding to the central eye position is expressed as The corresponding relationship between the coordinates of the first corner point and the corresponding coordinates of the third corner point is as follows:

[0115]

[0116] Among them, F is the mapping function corresponding to the viewing angle distortion caused by hardware factors such as equipment installation error, unreasonable optical system design and inherent characteristics of the display device. The mapping transformation relationship can also be obtained by replacing the LUT corresponding to the pixel coordinate transformation.

[0117] In one embodiment, the full distortion correction mapping relationship represents the transformation relationship between the standard image corresponding to the central eye position and the distorted images corresponding to other eye positions, which can be expressed by the following calculation formula:

[0118]

[0119] in, is the pixel coordinates in the actual image corresponding to any eye position, F is the mapping function corresponding to the viewing angle distortion caused by hardware factors such as device installation error, unreasonable optical system design and inherent characteristics of the display device, T is the mapping function corresponding to the viewing angle distortion caused by the change of the eye position, and Z p (X, Y) is the pixel coordinates in the image after full distortion correction.

[0120] See the schematic diagram of full distortion correction for details. Figure 4 .

[0121] In one embodiment, the full distortion correction mapping relationship includes full distortion correction mapping relationships corresponding to different eye loci, and the position information of each eye loci corresponds to a full distortion correction mapping function.

[0122] Step S130 specifically includes: obtaining the eye locus corresponding to the distorted image to be corrected; searching for a target mapping relationship from the full distortion correction mapping relationship according to the eye locus corresponding to the distorted image to be corrected; and correcting the distorted image to be corrected using the target mapping relationship.

[0123] The eye position corresponding to the distorted image to be corrected is obtained by using the eye tracking algorithm. Specifically, the position information of the eye position corresponding to the distorted image to be corrected is determined by capturing the position information of the driver's eyeball. The position information of the eye position is the position of the eye position in the three-dimensional coordinate system established above. 1 -Position information in UVK.

[0124] The full distortion correction mapping relationship can be a LUT (Look-Up Table). The LUT records the position information of each eye locus and the corresponding full distortion correction mapping function. Using the eye locus position information corresponding to the distorted image to be corrected, the corresponding full distortion correction mapping function is found from the LUT as the target mapping relationship.

[0125] According to the target mapping relationship, an inverse mapping transformation relationship is obtained. The distorted image to be corrected is corrected using the inverse mapping transformation relationship.

[0126] The following is a specific example to illustrate the image distortion correction method provided by the present invention. Figure 6 , the image distortion correction method includes steps S601 to S610.

[0127] Step S601, obtaining projection parameters; wherein the projection parameters include the three-dimensional size of the eye box, the virtual image projection distance, the virtual image imaging size and the lower field angle; and constructing a simulation model for determining the simulation model according to the projection parameters.

[0128] Step S602, determining the number and positions of eye sites according to a preset eye site density; using a virtual camera placed at the position of each eye site, capturing a standard image corresponding to the central eye site and perspective distortion images corresponding to other eye sites.

[0129] Step S603: extracting first corner point information from the standard image corresponding to the central eye point, and extracting second corner point information from the perspective distortion images corresponding to other eye points.

[0130] Step S604, determine the one-to-one correspondence between the first corner point and the second corner point; determine the position transformation relationship between each first corner point and the corresponding second corner point according to the position information of each first corner point, the position information of each second corner point, and the one-to-one correspondence between the first corner point and the second corner point; determine the first mapping relationship according to the position transformation relationship between each first corner point and the corresponding second corner point.

[0131] Step S605, obtaining a standard image corresponding to the central eye position and an actual image corresponding to the central eye position.

[0132] Step S606: extracting first corner point information from the standard image corresponding to the central eye position, and extracting third corner point information from the actual image corresponding to the central eye position.

[0133] Step S607, determine the one-to-one correspondence between the first corner point and the third corner point; determine the position transformation relationship between each first corner point and the corresponding third corner point according to the position information of each first corner point, the position information of each third corner point, and the one-to-one correspondence between the first corner point and the third corner point; determine the second mapping relationship according to the position transformation relationship between each first corner point and the corresponding third corner point.

[0134] Step S608: determining a total distortion correction mapping relationship according to the first mapping relationship and the second mapping relationship.

[0135] Step S609, obtaining the eye position corresponding to the distorted image to be corrected.

[0136] Step S610, according to the eye point corresponding to the distorted image to be corrected, a target mapping relationship is searched from the full distortion correction mapping relationship, and an inverse mapping transformation relationship is obtained according to the target mapping relationship, and the distorted image to be corrected is corrected using the inverse mapping transformation relationship.

[0137] An embodiment of the present invention provides an image distortion correction device, such as Figure 7 The image distortion correction device 700 includes an acquisition module 710 , a full distortion correction mapping relationship determination module 720 and a correction module 730 .

[0138] The acquisition module 710 is used to obtain a first mapping relationship and a second mapping relationship; wherein the first mapping relationship is a transformation relationship between a standard image corresponding to a central eye point and perspective distortion images corresponding to other eye points, and the second mapping relationship is a transformation relationship between a standard image corresponding to a central eye point and an actual image corresponding to the central eye point.

[0139] The total distortion correction mapping relationship determination module 720 is used to determine the total distortion correction mapping relationship according to the first mapping relationship and the second mapping relationship.

[0140] The correction module 730 is used to perform image distortion correction using the full distortion correction mapping relationship.

[0141] In one embodiment, the image distortion correction device further includes a first mapping relationship determination module.

[0142] The first mapping relationship determination module is used to obtain a standard image corresponding to the central eye point and perspective distortion images corresponding to other eye points; extract first corner point information from the standard image corresponding to the central eye point, and respectively extract second corner point information from perspective distortion images corresponding to other eye points; determine the corresponding first mapping relationship based on the first corner point information and each second corner point information.

[0143] In one embodiment, the image distortion correction device further includes a first mapping relationship determination module.

[0144] The first mapping relationship determination module is used to obtain projection parameters; wherein the projection parameters include the three-dimensional size of the eye box, the virtual image projection distance, the virtual image imaging size and the lower field angle; construct a simulation model according to the projection parameters; and determine the first mapping relationship using the simulation model.

[0145] In one embodiment, the first mapping relationship determination module is also used to determine the number and position of eye sites according to a preset eye site density; using a virtual camera placed at the location of each eye site, capture a standard image corresponding to the central eye site and a perspective distortion image corresponding to each other eye site; extract first corner point information from the standard image corresponding to the central eye site, and extract second corner point information from the perspective distortion images corresponding to each other eye site; determine the corresponding first mapping relationship based on the first corner point information and each second corner point information.

[0146] In one embodiment, the first corner point information includes position information of a plurality of first corner points, and the second corner point information includes position information of a plurality of second corner points. The first mapping relationship determination module is further used to determine the one-to-one correspondence between the first corner point and the second corner point based on each other eye point; determine the position transformation relationship between each first corner point and the corresponding second corner point according to the position information of each first corner point, the position information of each second corner point, and the one-to-one correspondence between the first corner point and the second corner point; determine the first mapping relationship according to the position transformation relationship between each first corner point and the corresponding second corner point.

[0147] In one embodiment, the image distortion correction device further includes a second mapping relationship determination module.

[0148] The second mapping relationship determination module is used to obtain a standard image corresponding to the central eye position and an actual image corresponding to the central eye position; extract first corner point information from the standard image corresponding to the central eye position, and extract third corner point information from the actual image corresponding to the central eye position; determine the second mapping relationship based on the first corner point information and the third corner point information.

[0149] In one embodiment, the first corner point information includes position information of a plurality of first corner points, and the third corner point information includes position information of a plurality of third corner points. The second mapping relationship determination module is further used to determine a one-to-one correspondence between the first corner points and the third corner points; determine a position transformation relationship between each first corner point and the corresponding third corner point based on the position information of each first corner point, the position information of each third corner point, and the one-to-one correspondence between the first corner point and the third corner point; and determine a second mapping relationship based on the position transformation relationship between each first corner point and the corresponding third corner point.

[0150] In one embodiment, the full distortion correction mapping relationship includes full distortion correction mapping relationships corresponding to different eye loci. The correction module is also used to obtain the eye loci corresponding to the distorted image to be corrected; according to the eye loci corresponding to the distorted image to be corrected, find the target mapping relationship from the full distortion correction mapping relationship; and use the target mapping relationship to correct the distorted image to be corrected.

[0151] An embodiment of the present invention provides an image distortion correction device, such as Figure 8 The image distortion correction device 800 includes a memory 820 and a processor 810. The memory 820 stores a computer program, and the computer program is used to control the processor 810 to operate so as to execute the image distortion correction method in any of the above embodiments.

[0152] An embodiment of the present invention provides an electronic device, such as Fig. 9 The electronic device includes the image distortion correction device provided by any of the above embodiments.

[0153] In one embodiment, the electronic device is an AR-HUD device. The AR-HUD device projects the corrected image onto the windshield through a projection device, and the corrected image includes navigation information, safety reminders, entertainment information, etc. The AR-HUD device provided in this embodiment can accurately compensate for the image distortion caused by hardware factors such as optical structure design and changes in eye position during driving, ensuring that the driver can observe a clear image with less distortion, thereby improving driving experience and safety.

[0154] An embodiment of the present invention provides a vehicle, including the electronic device as described in any of the above embodiments, see Fig.10 .

[0155] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. For the device embodiment, its related parts can be referred to the partial description of the method embodiment.

[0156] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0157] The embodiments of the present specification may be systems, methods and / or computer program products. The computer program product may include a computer-readable storage medium carrying computer instructions for causing a processor to implement various aspects of the embodiments of the present specification.

[0158] A computer-readable storage medium may be a tangible device that can hold and store computer instructions used by a computer instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which computer instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0159] The computer instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device through a network layer, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network layer may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network layer adapter card or network layer interface in each computing / processing device receives the computer instructions from the network layer and forwards the computer instructions for storage in a computer-readable storage medium in each computing / processing device.

[0160] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to the multiple embodiments of this specification. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a computer instruction, and a part of a module, a program segment or a computer instruction contains one or more executable computer instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0161] The embodiments of the present specification have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art without departing from the scope of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for image distortion correction, characterized in that: include: Obtaining a first mapping relationship and a second mapping relationship; wherein the first mapping relationship is a transformation relationship between a standard image corresponding to a central eye point and perspective distortion images corresponding to other eye points, and the second mapping relationship is a transformation relationship between a standard image corresponding to a central eye point and an actual image corresponding to the central eye point; Determining a total distortion correction mapping relationship according to the first mapping relationship and the second mapping relationship; The image distortion correction is performed using the full distortion correction mapping relationship.

2. The method according to claim 1, characterized in that The method further comprises: Obtain a standard image corresponding to the central eye position and a perspective distortion image corresponding to each other eye position; Extracting first corner point information from the standard image corresponding to the central eye point, and extracting second corner point information from the perspective distortion images corresponding to the other eye points respectively; A corresponding first mapping relationship is determined according to the first corner point information and each second corner point information.

3. The method according to claim 1, characterized in that The method further comprises: Acquire projection parameters; wherein the projection parameters include the three-dimensional size of the eye box, the virtual image projection distance, the virtual image imaging size and the lower field of view angle; Constructing a simulation model according to the projection parameters; The first mapping relationship is determined using the simulation model.

4. The method according to claim 3, characterized in that: The determining the first mapping relationship by using the simulation model includes: Determining the number and positions of eye sites according to a preset eye site density; Using virtual cameras placed at the locations of the eye sites, a standard image corresponding to the central eye site and perspective distortion images corresponding to the other eye sites are captured; Extracting first corner point information from the standard image corresponding to the central eye point, and extracting second corner point information from the perspective distortion images corresponding to the other eye points respectively; A corresponding first mapping relationship is determined according to the first corner point information and each second corner point information.

5. The method according to claim 2 or 4, characterized in that: The first corner point information includes position information of a plurality of first corner points, and the second corner point information includes position information of a plurality of second corner points; wherein determining the corresponding first mapping relationship according to the first corner point information and each second corner point information includes: Based on each other eye point, determine a one-to-one correspondence between the first corner point and the second corner point; Determine a position transformation relationship between each first corner point and the corresponding second corner point according to the position information of each first corner point, the position information of each second corner point, and the one-to-one correspondence between the first corner point and the second corner point; The first mapping relationship is determined according to the position transformation relationship between each first corner point and the corresponding second corner point.

6. The method according to claim 1, characterized in that The method further comprises: Obtaining a standard image corresponding to the central eye position and an actual image corresponding to the central eye position; Extracting first corner point information from the standard image corresponding to the central eye position, and extracting third corner point information from the actual image corresponding to the central eye position; The second mapping relationship is determined according to the first corner point information and the third corner point information.

7. The method according to claim 6, characterized in that The first corner point information includes position information of a plurality of first corner points, and the third corner point information includes position information of a plurality of third corner points; wherein determining the second mapping relationship according to the first corner points and the third corner points includes: Determine a one-to-one correspondence between the first corner point and the third corner point; Determine a position transformation relationship between each first corner point and the corresponding third corner point according to the position information of each first corner point, the position information of each third corner point, and the one-to-one correspondence between the first corner point and the third corner point; The second mapping relationship is determined according to the position transformation relationship between each first corner point and the corresponding third corner point.

8. The method according to claim 1, characterized in that: The total distortion correction mapping relationship includes total distortion correction mapping relationships corresponding to different eye loci, wherein the performing image distortion correction using the total distortion correction mapping relationship further includes: Obtaining the eye position corresponding to the distorted image to be corrected; According to the eye position corresponding to the distorted image to be corrected, searching for a target mapping relationship from the full distortion correction mapping relationship; The distorted image to be corrected is corrected using the target mapping relationship.

9. An image distortion correction device, characterized in that: include: An acquisition module, used to acquire a first mapping relationship and a second mapping relationship; wherein the first mapping relationship is a transformation relationship between a standard image corresponding to a central eye point and perspective distortion images corresponding to other eye points, and the second mapping relationship is a transformation relationship between a standard image corresponding to a central eye point and an actual image corresponding to the central eye point; A total distortion correction mapping relationship determination module, configured to determine a total distortion correction mapping relationship according to the first mapping relationship and the second mapping relationship; The correction module is used to perform image distortion correction using the full distortion correction mapping relationship.

10. An image distortion correction device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and the computer program is used to control the processor to operate so as to execute the image distortion correction method according to any one of claims 1 to 8.

11. An electronic device, characterized in that: It comprises the image distortion correction device as described in claim 9 or 10.

12. A vehicle, characterized in that: Comprising the electronic device as claimed in claim 11.

Images