Coordinate mapping method and image display method

Abstract

The invention discloses a coordinate mapping method and an image display method. The coordinate mapping method comprises the following steps: acquiring grid distortion data of the optical imaging system, and generating a radial distribution function corresponding to a field angle tangent value corresponding to a pixel point based on the grid distortion data; determining a mapping transformation relational expression between the pixel points on the original image and the pixel points on the display screen based on a relational expression between the field angle tangent values corresponding to the pixel points and the radial distribution function; parameter solving is carried out on the mapping transformation relational expression based on at least one preset condition, at least one coordinate mapping transformation relation is determined, the preset condition is set according to the application scene, the problem that only single distortion correction can be carried out on the image is solved, different preset conditions are generated based on different service scenes, different coordinate mapping transformation relations are determined, and the image distortion correction accuracy is improved. Different application scenes are flexibly adapted, and reasonable distortion correction is performed on images in different application scenes, so that a final user can see a distorted virtual image, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a coordinate mapping method and an image display method. Background Art

[0002] After being processed by an optical imaging system, images are distorted. Therefore, displaying these processed images directly on a screen can affect the user's visual experience due to distortion. To eliminate this distortion, images are typically corrected before display. However, existing distortion correction methods only address a single aspect of image distortion and lack the flexibility to adapt to the needs of diverse scenarios. Summary of the Invention

[0003] The present application provides a coordinate mapping method and an image display method to solve the problem of only being able to perform a single distortion correction on an image, and flexibly adapt to different application scenarios.

[0004] According to one aspect of the present application, a coordinate mapping method is provided, comprising:

[0005] Acquiring grid distortion data of the optical imaging system, and generating a radial distribution function corresponding to a tangent value of a field angle corresponding to a pixel point based on the grid distortion data;

[0006] Determining a mapping transformation relationship between a pixel point on the original image and a pixel point on the display screen based on a relationship between a tangent value of the field angle corresponding to the pixel point and the radial distribution function;

[0007] The mapping transformation relationship is parameter-solved based on at least one preset condition to determine at least one coordinate mapping transformation relationship, wherein the preset condition is set according to an application scenario.

[0008] According to another aspect of the present application, there is provided an image display method, comprising:

[0009] Obtain application scenario requirement information;

[0010] Selecting a matching target coordinate mapping transformation relationship from a predetermined coordinate mapping transformation relationship according to the application scenario requirement information, wherein the coordinate mapping transformation relationship is determined according to the coordinate mapping method described in any embodiment of the present application;

[0011] The pixel coordinates in the display screen are subjected to distortion correction based on the target coordinate mapping transformation relationship, and the image is displayed based on the pixel values ​​corresponding to the pixel coordinates after the distortion correction.

[0012] According to another aspect of the present application, a coordinate mapping device is provided, comprising:

[0013] A distortion data acquisition module, configured to acquire grid distortion data of the optical imaging system and generate a radial distribution function corresponding to a tangent value of a field of view angle corresponding to a pixel point based on the grid distortion data;

[0014] A mapping transformation relationship determination module is used to determine a mapping transformation relationship between a pixel point on the original image and a pixel point on the display screen based on a relationship between a tangent value of the field angle corresponding to the pixel point and the radial distribution function;

[0015] The coordinate mapping relationship determination module is used to solve the parameters of the mapping transformation relationship based on preset conditions to determine the coordinate mapping transformation relationship.

[0016] According to another aspect of the present application, there is provided an image display device, comprising:

[0017] Application scenario acquisition module, used to obtain application scenario requirement information;

[0018] a target mapping relationship determination module, configured to select a matching target coordinate mapping transformation relationship from predetermined coordinate mapping transformation relationships according to the application scenario requirement information, wherein the coordinate mapping transformation relationship is determined according to the coordinate mapping method described in any embodiment of the present application;

[0019] The distortion correction module is used to perform distortion correction on the pixel coordinates in the display screen based on the target coordinate mapping transformation relationship, and display the image based on the pixel values ​​corresponding to the pixel coordinates after the distortion correction.

[0020] According to another aspect of the present application, an electronic device is provided, comprising:

[0021] at least one processor, and a memory communicatively coupled to the at least one processor;

[0022] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the coordinate mapping method or image display method described in any embodiment of the present application.

[0023] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the coordinate mapping method or image display method described in any embodiment of the present application when executed.

[0024] According to another aspect of the present application, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, it implements the coordinate mapping method or image display method described in any embodiment of the present application.

[0025] The technical solution of the embodiment of the present application is to obtain grid distortion data of the optical imaging system, generate a radial distribution function corresponding to the tangent value of the field of view angle corresponding to the pixel point based on the grid distortion data; determine the mapping transformation relationship between the pixel point on the original image and the pixel point on the display screen based on the relationship between the tangent value of the field of view angle corresponding to the pixel point and the radial distribution function; solve the parameters of the mapping transformation relationship based on at least one preset condition to determine at least one coordinate mapping transformation relationship, and the preset condition is set according to the application scenario, which solves the problem that only a single distortion correction can be performed on the image, and constructs the relationship between the pixel point on the original image and the display screen based on the relationship between the tangent value of the field of view angle corresponding to the pixel point and the radial distribution function. The mapping transformation relationship between the pixel points on the screen can be solved under different conditions; different preset conditions are set according to different application scenarios, and the mapping transformation relationship is parameter-solved based on at least one preset condition to obtain the parameter value of the mapping transformation relationship, and finally at least one coordinate mapping transformation relationship is determined, and each coordinate mapping transformation relationship can be used to correct the distortion of the image; the embodiment of the present application generates different preset conditions based on different business scenarios, and then determines different coordinate mapping transformation relationships according to different preset conditions. It can flexibly adapt to different application scenarios and perform reasonable distortion correction on images in different application scenarios, so that the end user can see a virtual image without distortion, thereby improving the user experience.

[0026] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0028] Figure 1 This is a flowchart of a coordinate mapping method provided according to Example 1 of the present application;

[0029] Figure 2 This is an example diagram of distortion provided according to the first embodiment of the present application;

[0030] Figure 3 This is an example diagram of data fitting provided according to Example 1 of the present application;

[0031] Figure 4This is a flow chart of a coordinate mapping method provided according to the second embodiment of the present application;

[0032] Figure 5 This is an example diagram of an image that has undergone inverse transformation of a correction model according to the second embodiment of the present application;

[0033] Figure 6 This is an example diagram of a virtual image forming process provided in accordance with the second embodiment of the present application;

[0034] Figure 7 is an example diagram of another virtual image forming process provided in Example 2 of the present application;

[0035] Figure 8 is an example diagram of another virtual image forming process provided in Example 2 of the present application;

[0036] Figure 9 is an example diagram of another virtual image forming process provided in Example 2 of the present application;

[0037] Figure 10 is a flowchart of an image display method provided according to the third embodiment of the present application;

[0038] Figure 11 This is a schematic structural diagram of a coordinate mapping device provided according to the fourth embodiment of the present application;

[0039] Figure 12 is a structural diagram of an image display device provided according to Embodiment 5 of the present application;

[0040] Figure 13 This is a structural diagram of an electronic device provided according to Example 6 of the present application. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0042] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0043] Example 1

[0044] Figure 1 This is a flow chart of a coordinate mapping method provided in the first embodiment of the present application. This embodiment is applicable to the case of mapping coordinates in an image. The method can be executed by a coordinate mapping device. The coordinate mapping device can be implemented in the form of hardware and / or software. The coordinate mapping device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0045] S101 , obtaining grid distortion data of an optical imaging system, and generating a radial distribution function corresponding to a tangent value of a field angle corresponding to a pixel point based on the grid distortion data.

[0046] In this embodiment, the display screen may be a display screen of a near-eye display device, and the near-eye display device may be a display device that produces significant distortion after optical imaging, such as a desktop computer, a mobile phone, a tablet computer, VR glasses, AR glasses, and the like. The grid distortion data may be understood as relevant data describing changes in the shape of the grid. The grid distortion data in the embodiment of the present application at least includes a correspondence between angles and physical coordinates. This physical coordinate may be converted into the coordinates of a pixel point on the display screen, and this angle may be used as the field of view angle of the pixel point relative to the human eye, wherein the coordinates of the pixel point on the display screen use the center of the display screen as the coordinate origin, and may be represented by (x, y). The field of view angle refers to the field of view angle of the point corresponding to the pixel point in the virtual image projected on the x-axis and y-axis relative to the human eye, and may be represented by The radial distribution function is the functional relationship between the distortion data and the distance from the pixel point on the distorted image to the center of the distorted image.

[0047] The virtual image formed on the display screen after passing through the optical imaging system usually produces pincushion distortion. The grid distortion data of the optical imaging system is obtained. The grid distortion data usually includes multiple sets of physical coordinates and their field of view angles. The physical coordinates are divided by the pixel interval to obtain the coordinates of the pixel points on the display screen. Based on the coordinates of the pixel points on the display screen, the functional relationship between the distortion data and the distance from the pixel point on the distorted image to the center of the distorted image is fitted to obtain the radial distribution function corresponding to the tangent value of the field of view angle corresponding to the pixel point.

[0048] For example, Figure 2 An example diagram of distortion is provided, showing the virtual image formed on the screen after passing through the optical imaging system, which generally produces pincushion distortion, such as Figure 2 This distortion can be quantitatively described by formula 1:

[0049] Formula 1;

[0050] Among them, (x, y) is the coordinate of the pixel point on the display screen, with the center of the display screen as the coordinate origin. It is the field of view angle of the point projected on the x-axis and y-axis relative to the human eye.

[0051] For ease of calculation, the tangent value of the field of view angle may be used instead of the angle value to obtain Formula 2, which is equivalent to Formula 1.

[0052] Formula 2;

[0053] For a symmetrical optical imaging system, Equation 2 can be simplified to Equation 3:

[0054] Formula 3;

[0055] For a specific optical imaging system, The relationship between (x, y) and r is also certain. The design software of the optical imaging system can be used to track the imaged points corresponding to the discrete sampling points on the display screen to obtain a set of (x, y) and The data points of the corresponding relationship are the grid distortion data.

[0056] Choose an appropriate fitting method to get The functional relationship between r and , for example, for a symmetrical optical imaging system, polynomial fitting can be used to more accurately determine For example, the embodiment of the present application provides an expression for the radial distribution function F(r) obtained by fitting a 9th-order polynomial:

[0057] Formula 4.

[0058] For example, Figure 3 An example graph of data fitting is provided, where the horizontal axis represents r, the coordinates can be pixels, and the vertical axis represents , Figure 3 for Example graph of the functional relationship with r.

[0059] S102 : Determine a mapping transformation relationship between the pixel points on the original image and the pixel points on the display screen based on a relationship between the tangent value of the field angle corresponding to the pixel point and the radial distribution function.

[0060] In this embodiment, the mapping transformation relationship is a coordinate mapping relationship. It can map pixels in the original image to the display screen according to the coordinates, and can also map pixels on the display screen to the original image according to the coordinates. The mapping transformation relationship in the embodiment of the present application includes unknown quantities. The original image refers to the image that has not been processed by the optical imaging system. During the optical imaging process, the original image is first determined, and then the original image is processed by the optical imaging system to form a virtual image, which is displayed on the display screen.

[0061] The relationship between the tangent value of the field of view angle corresponding to the pixel point and the radial distribution function is determined by analysis and processing. For example, the tangent value of the field of view angle corresponding to the pixel point is equal to the radial distribution function. By analyzing the relationship between the tangent value of the field of view angle and the radial distribution function, on the basis of the radial distribution function having been determined, the expression of the tangent value of the field of view angle can be further determined. This expression is transformed or combined with other expressions of the tangent value of the field of view angle to transform and determine the coordinate transformation expression. This coordinate transformation expression is recorded as a mapping transformation relationship. The mapping transformation relationship is the relationship expression between the pixel points on the original image and the pixel points on the display screen. It can represent the coordinate transformation relationship between the pixel points on the original image and the pixel points on the display screen. The mapping transformation relationship obtained in this step includes unknown quantities and needs to be further solved.

[0062] Exemplarily, the radial distribution function is a functional relationship between the distortion data and the distance from the pixel point on the distorted image to the center of the distorted image. Based on the relationship between the tangent value of the field of view angle corresponding to the pixel point and the radial distribution function, the tangent value of the field of view angle and the distance from the pixel point on the distorted image to the center of the distorted image can be determined. Based on the mathematical expression of the tangent value of the field of view angle, the distance from the pixel point on the original image to the center of the original image is determined. Based on the above expression, the formula is transformed to obtain the relationship expression between the pixel point on the original image and the pixel point on the display screen.

[0063] S103 : Solve the mapping transformation relationship equation for parameters based on at least one preset condition to determine at least one coordinate mapping transformation relationship, where the preset condition is set according to an application scenario.

[0064] In this embodiment, the preset conditions can be set in advance according to different application scenarios. For example, the application scenario can be one or more scenarios such as office scenario, movie viewing scenario, game scenario, social interaction scenario, learning and education scenario, financial management scenario, health care scenario, creative design scenario, multimedia entertainment scenario, travel navigation scenario, smart home scenario, industrial control scenario, shopping application scenario, weather forecast scenario, schedule management scenario, etc. The coordinate mapping transformation relationship can be understood as an expression representing the coordinate transformation relationship between the pixel points on the original image and the pixel points on the display screen, wherein the parameters involved in the formula are known parameters; the coordinate mapping transformation relationship can be used to coordinate map the pixel points on the original image and the display screen, and the pixel points in the original image can be mapped to the display screen according to the coordinates, and the pixel points on the display screen can also be mapped to the original image according to the coordinates. The coordinate mapping transformation relationship and the mapping transformation relationship formula in the embodiment of the present application are the same expression. The mapping transformation relationship formula includes unknown parameters that need to be further solved. The parameters in the coordinate mapping transformation relationship are all known quantities. The coordinates of the pixel points on the display screen can be substituted into the coordinate mapping transformation relationship to obtain the coordinates of the pixel points on the original image, or the coordinates of the pixel points on the original image can be substituted into the coordinate mapping transformation relationship to obtain the coordinates of the pixel points on the display screen.

[0065] Analyze the application scenario in advance, determine the requirements for screen display under different application scenarios, and then generate different preset conditions. Solve the parameters of the mapping transformation relationship based on at least one preset condition to determine at least one coordinate mapping transformation relationship, and the preset conditions are set according to the application scenario. For each preset condition, the mapping transformation relationship is deformed and assigned based on the preset condition, and the parameters are solved to obtain the values ​​of the parameters. The values ​​of the parameters obtained by the solution are substituted into the mapping transformation relationship to obtain the corresponding coordinate mapping transformation relationship. For each preset condition, the embodiment of the present application can solve and obtain at least one corresponding coordinate mapping transformation relationship.

[0066] An embodiment of the present application provides a coordinate mapping method, which solves the problem of only being able to perform a single distortion correction on an image. Based on the relationship between the tangent value of the field of view angle corresponding to the pixel point and the radial distribution function, a mapping transformation relationship between the pixel points on the original image and the pixel points on the display screen is constructed, and the obtained mapping transformation relationship can be solved under different conditions; different preset conditions are set according to different application scenarios, and the mapping transformation relationship is parameter-solved based on at least one preset condition to obtain the parameter value of the mapping transformation relationship, and finally at least one coordinate mapping transformation relationship is determined, and each coordinate mapping transformation relationship can be used to correct the distortion of the image; the embodiment of the present application generates different preset conditions based on different business scenarios, and then determines different coordinate mapping transformation relationships according to different preset conditions. It can flexibly adapt to different application scenarios and perform reasonable distortion correction on images in different application scenarios, so that the end user sees a virtual image without distortion, thereby improving the user experience.

[0067] Example 2

[0068] Figure 4 This is a flow chart of a coordinate mapping method provided in Example 2 of this application. This example is refined based on the above example. Figure 4 As shown, the method includes:

[0069] S201 , obtaining grid distortion data of an optical imaging system, and generating a radial distribution function corresponding to a tangent value of a field angle corresponding to a pixel point based on the grid distortion data.

[0070] S202 : Determine a first expression corresponding to the tangent value of the field angle based on a relationship between the tangent value of the field angle corresponding to the pixel point and the radial distribution function, where the first expression is related to the radial distribution function.

[0071] In this embodiment, the first expression can be understood as a calculation expression for the tangent value of the field angle. The first expression is determined according to the radial distribution function and is related to the radial distribution function.

[0072] Based on the relationship between the tangent value of the field of view angle corresponding to the pixel point and the radial distribution function, the relationship between the tangent value of the field of view angle and the radial distribution function is determined. This relationship is transformed to obtain an expression for the tangent value of the field of view angle, which is recorded as the first expression. The first expression is related to the radial distribution function. When the radial distribution function has been generated based on the grid distortion data, the first expression corresponding to the tangent value of the field of view can also be further determined based on the generated radial distribution function.

[0073] For example, the tangent value of the field angle = radial distribution function, the first expression can be , the first expression can also be .

[0074] S203. Determine a second expression corresponding to the tangent value of the field of view angle based on the distance from the pixel point in the original image to the center of the original image, where the second expression includes a proportional relationship, which is the proportional relationship between the distance from the pixel point in the original image to the center of the original image and the tangent value of the field of view angle.

[0075] In this embodiment, the second expression can be understood as a calculation expression for the tangent value of the field of view angle. The parameters of the second expression include a proportional relationship, which is the proportional relationship between the distance from the pixel point in the original image to the center of the original image and the tangent value of the field of view angle.

[0076] Based on the distance from the pixel point on the original image to the center of the original image, the second expression is determined in combination with the mathematical meaning of the tangent value of the field of view angle. For example, the second expression is: , r' is the distance from the pixel point on the original image to the center of the original image, and L is the proportional relationship, which can also be understood as the distance from the observation point to the center of the virtual image multiplied by r' and the ratio of the distance from the pixel point in the corresponding virtual image to the center of the virtual image. L changes with the change of the preset conditions.

[0077] S204 : Perform transformation based on the first expression and the second expression to determine a mapping transformation relationship between the pixel points on the original image and the pixel points on the display screen.

[0078] Since the first expression and the second expression are both expressions of the tangent value of the field of view angle, the first expression and the second expression can be transformed into a formula, and the resulting relationship is the mapping transformation relationship between the pixel points on the original image and the pixel points on the display screen.

[0079] Exemplarily, the first expression and the second expression are transformed to obtain a mapping transformation relationship: r'=LF(r), Formula 5.

[0080] S205 . For each preset condition, solve the mapping transformation relationship equation based on the preset condition to determine the value of the proportional relationship corresponding to the preset condition.

[0081] In this embodiment, a parameter included in the mapping transformation relationship is a proportional relationship, and this proportional relationship is an unknown parameter. For each preset condition, the mapping transformation relationship is transformed and assigned a value based on the preset condition, and the parameter is solved to obtain the value of the proportional relationship.

[0082] S206: Substitute the value of the proportional relationship into the mapping transformation relationship to obtain a coordinate mapping transformation relationship corresponding to the preset conditions.

[0083] Substituting the proportional relationship into the mapping transformation equation, the resulting parameters are known. This mapping transformation equation is the coordinate mapping transformation corresponding to the preset conditions. Solving for the parameters yields the value of L, and thus the coordinate mapping transformation r' = LF(r).

[0084] Or r'=LF(r) can also be transformed into: , Formula 6.

[0085] Based on the known coordinates (x, y) or r of the pixel point on the display screen, its coordinates (x', y') or r' in the original image can be further determined. The coordinate mapping transformation relationship can also be called the mapping transformation of the correction model. For example, Figure 5 An example diagram of an image that has been inversely transformed by a correction model is provided. The original image without distortion presented in the virtual image is inversely transformed by the correction model to obtain the image displayed on the display screen.

[0086] Optionally, performing parameter solving on the mapping transformation relationship based on preset conditions to determine the value of the proportional relationship corresponding to the preset conditions includes steps A1-A2:

[0087] A1. When the preset condition is the first preset condition, derive the mapping transformation relationship, transform the formula based on the derivation result, and determine the expression of the proportional relationship and the derivative.

[0088] The first preset condition is that the rate of change of the center of the original image during the mapping transformation is 1.

[0089] A first preset condition is set according to the application scenario. The first preset condition is that the rate of change of the center of the original image during the mapping transformation process is 1. When the preset condition is the first preset condition, the mapping transformation relationship is differentiated on both sides of the equation to obtain the derivative result: dr'=L1F'(0)dr. The derivative result is transformed into a formula, and both sides of the equation are divided by dr. The resulting proportional relationship and derivative expression are: dr' / dr=L1F'(0), where L1 is the proportional relationship corresponding to the first preset condition.

[0090] A2. Let the derivative be equal to the rate of change of the center of the original image during the mapping transformation process, solve the expression of the proportional relationship and the derivative, and determine the value of the proportional relationship corresponding to the first preset condition.

[0091] Since the first preset condition is that the rate of change of the center of the original image during the mapping transformation is 1, it can be seen that the rate of change of the center of the original image during the mapping transformation is 1. Let the derivative be equal to the rate of change of the center of the original image during the mapping transformation, that is, let dr' / dr = 1. Solving the above proportional relationship and the derivative expression, the left side of the equation is equal to 1. The right side of the equation F'(0) can be calculated based on the expression of F(r), L1=1 / F'(0). After solving the calculation, the value of L1 can be obtained, that is, the value of the proportional relationship corresponding to the first preset condition can be obtained.

[0092] The coordinate mapping transformation relationship is obtained by using L1, and the coordinate mapping transformation relationship is: r'= F(r) / F'(0). The original image is corrected by the above coordinate mapping transformation relationship and then displayed on the display screen, and finally a virtual image without distortion is obtained. For example, Figure 6 An example diagram of the virtual image formation process is provided. The original image is transformed through a correction model mapping to form an intermediate image, which is then passed through an optical imaging system to form a virtual image. The advantage of the first preset condition is that it preserves the fineness of the image center as much as possible, facilitating the clear presentation of small text or symbols, making it suitable for office scenarios. However, the disadvantage is that after the mapping transformation, the original image is surrounded by black borders, limiting the display's utilization.

[0093] Optionally, performing parameter solving on the mapping transformation relationship based on at least one preset condition to determine the value of the proportional relationship corresponding to each preset condition includes steps B1-B2:

[0094] B1. When the preset condition is the second preset condition, determine the first pixel coordinates of the edge of the original image and the second pixel coordinates of the edge of the display screen based on the width of the display screen.

[0095] The second preset condition is that the edge of the image after the mapping transformation is aligned with the edge of the display screen.

[0096] A second preset condition is set according to the application scenario, and the second preset condition is that the edge of the image after the mapping transformation is aligned with the edge of the display screen. When the preset condition is the second preset condition, the width and coordinate origin of the display screen are determined. In the embodiment of the present application, the center of the display screen is used as the coordinate origin. Based on the coordinate origin and the width W of the display screen, the coordinates of the pixel points at the edge of the original image are determined, which are recorded as the first pixel coordinates, and the coordinates of the pixel points at the edge of the display screen are determined, which are recorded as the second pixel coordinates. For example, the first pixel coordinates and the second pixel coordinates are both (0, (W-1) / 2), or the first pixel coordinates and the second pixel coordinates are both ((W-1) / 2, 0), or one of the first pixel coordinates and the second pixel coordinates is ((W-1) / 2, 0), and the other is (0, (W-1) / 2), and so on; wherein W is the width of the display screen, and the unit of W is the number of pixels, that is, the width of the display screen is W pixels.

[0097] In the embodiment of the present application, due to image distortion, pixels on the edge of the original image that are on the coordinate axis remain on the edge after distortion correction, while pixels on other edges are no longer on the edge after distortion correction. Therefore, the first pixel coordinates of the edge of the original image and the second pixel coordinates of the edge of the display screen selected in the embodiment of the present application are pixels on the edge of the coordinate axis.

[0098] B2. Solve the mapping transformation relationship based on the first pixel coordinate and the second pixel coordinate to determine the value of the proportional relationship corresponding to the second preset condition.

[0099] Determine the distance r between the pixel point at the edge of the original image and the center of the original image based on the coordinates of the first pixel point e , based on the second pixel coordinates, determine the distance r between the pixel point at the edge of the displayed image and the center of the display screen e ', r e and r e 'Substitute into r'=LF(r) to get r e '=L2F(r e ), combined with r e =r e '=(W-1) / 2 to solve the parameters and obtain L2:

[0100] ;

[0101] Wherein, L2 is the proportional relationship corresponding to the second preset condition.

[0102] Alternatively, the embodiment of the present application may also calculate L2 by directly substituting the first pixel coordinate and the second pixel coordinate according to a modified formula of r'=LF(r), ie, Formula 6.

[0103] The coordinate mapping transformation relationship is obtained using L2, and the coordinate mapping transformation relationship is: .

[0104] The original image is corrected by the above coordinate mapping transformation relationship and then displayed on the display screen, and finally a virtual image without distortion is obtained. Figure 7 An example diagram of another virtual image formation process is provided. The original image is transformed through a correction model mapping to form an intermediate image, which is then passed through an optical imaging system to form a virtual image. The advantage of this second preset condition is that the image covers the display area as much as possible, making it suitable for entertainment scenarios such as watching movies or playing games. However, the disadvantage is that the original image is stretched after the mapping transformation, which may cause small text or symbols to appear blurry.

[0105] Optionally, the mapping transformation relationship is parameterized based on at least one preset condition to determine the value of the proportional relationship corresponding to each preset condition, including: when the preset condition is the third preset condition, linearly interpolating the value of the proportional relationship corresponding to the first preset condition and the value of the proportional relationship corresponding to the second preset condition along the radial direction to obtain the value of the proportional relationship corresponding to the third preset condition; wherein the third preset condition is that the rate of change of the center of the original image during the mapping transformation process is 1 and the edge of the image after the transition to the mapping transformation is aligned with the edge of the display screen.

[0106] A third preset condition is set based on the application scenario. The third preset condition requires that the rate of change of the center of the original image during the mapping transformation process is 1, and that the edge of the image after the mapping transformation aligns with the edge of the display screen. This means that the third preset condition combines the first and second preset conditions, taking into account the advantages of both. When the third preset condition is used, the proportional relationship value L1 corresponding to the first preset condition and the proportional relationship value L2 corresponding to the second preset condition are determined. Linear interpolation is performed along the radial direction on L1 and L2 to obtain the proportional relationship value L3 corresponding to the third preset condition.

[0107] Optionally, the value of the proportional relationship corresponding to the third preset condition is:

[0108] ;

[0109] Among them, L3 is the value of the proportional relationship corresponding to the third preset condition, L2 is the value of the proportional relationship corresponding to the second preset condition, L1 is the value of the proportional relationship corresponding to the first preset condition, W is the width of the display screen, and r is the distance from the pixel point in the display screen to the center of the display screen.

[0110] When performing coordinate mapping, the value of L3 can be calculated based on the relationship between the size of the coordinate to be mapped and (W-1) / 2, and the coordinates can be converted. The value of the proportional relationship corresponding to the third preset condition obtained in the embodiment of the present application may be a fixed value or an expression of a value, and the specific value can be calculated from the coordinates.

[0111] The third preset condition can maintain the rate of change of the center of the original image at 1 during the transformation process, and gradually transition to the edge of the image being aligned with the edge of the display screen; L3 is combined with Formula 5 or Formula 6 to obtain a mapping transformation relationship, which can be used to correct the original image and display it on the display screen, ultimately obtaining a virtual image of the corrected image. For example, Figure 8 Another example diagram of the virtual image formation process is provided. The advantage of the third preset condition is that it preserves the fineness of the image center while covering the display area as much as possible, making it suitable for more complex scenes. The disadvantage is that the transition area will produce slight distortion.

[0112] Optionally, the mapping transformation relationship is parameterized based on at least one preset condition to determine the value of the proportional relationship corresponding to each preset condition, including: when the preset condition is the fourth preset condition, linearly interpolating the value of the proportional relationship corresponding to the second preset condition along the radial direction to obtain the value of the proportional relationship corresponding to the fourth preset condition; wherein the fourth preset condition is that the edge of the image after the mapping transformation fits the edge of the display screen and reduces the degree of correction of the four corners.

[0113] A fourth preset condition is set based on the application scenario. The fourth preset condition is that the edges of the transformed image align with the edges of the display screen and that the degree of correction at the four corners is minimized. When the fourth preset condition is set, a value L2 of the proportional relationship corresponding to the second preset condition is determined, and linear interpolation is performed along the radial direction of L2 to obtain a value L4 of the proportional relationship corresponding to the fourth preset condition.

[0114] Optionally, the value of the proportional relationship corresponding to the fourth preset condition is:

[0115] ;

[0116] Among them, L4 is the value of the proportional relationship corresponding to the fourth preset condition, L2 is the value of the proportional relationship corresponding to the second preset condition, W is the width of the display screen, r is the distance from the pixel point in the display screen to the center of the display screen, and D is a pre-set coefficient.

[0117] D is typically a value between 0 and 1. When performing coordinate mapping, the value of L4 can be calculated based on the relationship between the size of the coordinate to be mapped and (W-1) / 2, and the coordinates can be converted. The value of the proportional relationship corresponding to the fourth preset condition obtained in the embodiment of the present application may be a fixed value or an expression of a value, and the specific value can be calculated from the coordinates.

[0118] The fourth preset condition is based on the second preset condition, assuming that the L value is reduced to DL2 at the corner of the display screen, and linear interpolation is performed along the radial direction when r>(W-1) / 2 to obtain a new L value, namely L4. L4 is combined with Formula 5 or Formula 6 to obtain a mapping transformation relationship. This mapping transformation relationship can be used to correct the original image and display it on the display screen, ultimately obtaining a virtual image of the corrected image. For example, Figure 9 Another example diagram of the virtual image formation process is provided, which is generated under the condition of D = 0.8. The advantage of the fourth preset condition is that it covers a larger display area and is suitable for highly immersive scenes.

[0119] The embodiment of the present application provides a coordinate mapping method, which solves the problem of only being able to perform a single distortion correction on an image. Different preset conditions are set according to different application scenarios, and the mapping transformation relationship is parameter-solved based on the different preset conditions to obtain the parameter values ​​of the mapping transformation relationship, which can be flexibly adapted to different application scenarios. The embodiment of the present application further refines the preset conditions into four preset conditions. The coordinate mapping transformation relationship obtained based on the first preset condition can retain the fineness of the image center, the coordinate mapping transformation relationship obtained based on the second preset condition can cover more display areas, the coordinate mapping transformation relationship obtained based on the third preset condition can cover more display areas while retaining the fineness of the image center, and the coordinate mapping transformation relationship obtained based on the fourth preset condition can cover a larger display area. The above-mentioned different preset conditions are used to perform reasonable distortion correction on images in different application scenarios, so that the end user can see a virtual image without distortion, thereby improving the user experience.

[0120] Example 3

[0121] Figure 10 This is a flowchart of an image display method provided in the third embodiment of the present application. This embodiment is applicable to the case of displaying an image on a screen. The method can be executed by an image display device, which can be implemented in the form of hardware and / or software. The image display device can be configured in an electronic device. Figure 10 As shown, the method includes:

[0122] S301. Obtain application scenario requirement information.

[0123] In this embodiment, the application scenario requirement information can be understood as the information required for image display in a certain application scenario; for example, the application scenario requirement information may include clear display of small-size characters, large display area coverage, etc., or the application scenario requirement information may include office scenarios, entertainment scenarios, etc. Different application scenarios may correspond to different application scenario requirement information.

[0124] The acquisition of application scenario requirement information can be automated, for example, by analyzing the currently displayed image or the current application scenario to determine the application scenario requirement information; or the acquisition of application scenario requirement information can be by receiving user control commands, for example, the user inputs control commands through manual operation, voice control, etc., and the control commands can be directly used as application scenario requirement information, or the execution device can perform semantic analysis on the control commands to determine the application scenario requirement information, and so on.

[0125] S302: Select a matching target coordinate mapping transformation relationship from predetermined coordinate mapping transformation relationships according to application scenario requirement information. The coordinate mapping transformation relationship is determined according to the coordinate mapping method described in any embodiment of the present application.

[0126] In this embodiment, the target coordinate mapping transformation relationship can be understood as a coordinate mapping transformation relationship that matches the application scenario requirement information. The coordinate mapping transformation relationship under different preset conditions is pre-determined to obtain at least one coordinate mapping transformation relationship. Exemplarily, the embodiment of the present application can pre-set four preset conditions, and four coordinate mapping transformation relationships are generated and stored accordingly. The four preset conditions are a first preset condition, a second preset condition, a third preset condition, and a fourth preset condition.

[0127] Different coordinate mapping transformation relationships are stored in correspondence with preset conditions in advance, or the application scenarios to which the coordinate mapping transformation relationships are adapted are determined according to the preset conditions, and the coordinate mapping transformation relationships are stored in correspondence with the application scenarios to which they are adapted. After determining the application scenario requirement information, based on the matching relationship between the pre-stored coordinate mapping transformation relationship and the preset conditions or application scenarios, the coordinate mapping transformation relationship that matches the application scenario requirement information is selected as the target coordinate mapping transformation relationship. Exemplarily, the matching relationship between the pre-stored coordinate mapping transformation relationship and the application scenario, after determining the application scenario requirement information, the application scenario requirement information and the application scenario are matched to determine the application scenario that matches the application scenario requirement information, and the coordinate mapping transformation relationship corresponding to the matched application scenario is used as the target coordinate mapping transformation relationship; or, the matching relationship between the pre-stored coordinate mapping transformation relationship and the preset conditions, after determining the application scenario requirement information, the application scenario requirement information is analyzed to determine the preset conditions that it matches, and the coordinate mapping transformation relationship corresponding to the matched preset conditions is used as the target coordinate mapping transformation relationship.

[0128] S303 , performing distortion correction on the pixel coordinates in the display screen based on the target coordinate mapping transformation relationship, and displaying an image based on the pixel values ​​corresponding to the pixel coordinates after the distortion correction.

[0129] Based on the target coordinate mapping transformation relationship, the pixel coordinates in the display screen are corrected for distortion, the coordinates of the pixel coordinates in the display screen in the image to be displayed are determined, and the pixel coordinates after distortion correction are obtained; the pixel values ​​corresponding to the pixel coordinates in the display screen are determined according to the pixel values ​​corresponding to the pixel coordinates after distortion correction, and the image is displayed according to the pixel values ​​corresponding to the pixel coordinates in the display screen.

[0130] The embodiment of the present application provides an image display method, which solves the problem of only being able to perform a single distortion correction on an image. Different preset conditions are generated in advance according to different business scenarios, and then different coordinate mapping transformation relationships are determined according to different preset conditions, which can flexibly adapt to different application scenarios. When displaying an image, the application scenario requirement information is obtained, and a matching target coordinate mapping transformation relationship is selected from the predetermined coordinate mapping transformation relationship according to the application scenario requirement information. The pixel coordinates in the display screen are subjected to distortion correction based on the matching target coordinate mapping transformation relationship, and then the image is displayed; the appropriate target coordinate mapping transformation relationship can be selected according to different application scenarios to perform reasonable distortion correction on the image, so that the end user sees a virtual image without distortion, thereby improving the user experience. The appropriate coordinate mapping transformation relationship is selected according to the specific application scenario requirement information, and the correction of the image is flexible and changeable.

[0131] Optionally, obtaining application scenario requirement information includes: determining the application scenario requirement information according to the type of the currently running application program.

[0132] Obtain the name or identifier of the currently running application, analyze the application based on the name or identifier of the application, and determine the type of application. Analyze the type of application to determine the application scenario requirement information; for example, pre-store the application scenario requirement information corresponding to different types of applications in a data table, and after determining the type of application, search the pre-stored data table based on the type of application to determine its corresponding application scenario requirement information; or, analyze the type of application with the help of a model with analytical capabilities to determine its corresponding application scenario requirement information. Determining the application scenario requirement information based on the type of the currently running application does not require user participation, and can automatically match the appropriate coordinate mapping transformation relationship for distortion correction without the user's knowledge, thereby improving the user experience; and determining the application scenario requirement information based on the type of the currently running application can accurately and quickly determine the application scenario requirement information.

[0133] Optionally, distortion correction is performed on the pixel coordinates in the display screen based on the target coordinate mapping transformation relationship, including: performing coordinate transformation on the first pixel coordinate to be displayed in the display screen based on the target coordinate mapping transformation relationship, and determining the second pixel coordinate to be displayed corresponding to the first pixel coordinate to be displayed in the display screen in the image to be displayed; and determining the pixel value corresponding to the first pixel coordinate to be displayed based on the second pixel coordinate to be displayed and the pixel values ​​of each pixel coordinate in the image to be displayed.

[0134] In this embodiment, the first pixel coordinates to be displayed are pixel coordinates on the display screen, and each pixel coordinate on the display screen is used as the first pixel coordinates to be displayed for coordinate transformation. The second pixel coordinates to be displayed are the first pixel coordinates to be displayed mapped to the corresponding coordinates in the image to be displayed. The image to be displayed can be understood as the original image to be displayed on the display screen.

[0135] When displaying an image, first obtain the image to be displayed. The image to be displayed has the same size as the original image described in any of the above embodiments. Each pixel coordinate in the display screen is sequentially used as the first pixel coordinate to be displayed. For each first pixel coordinate to be displayed, its corresponding pixel value can be determined in the following manner: the first pixel coordinate to be displayed is substituted into the target coordinate mapping transformation relationship to perform coordinate transformation to obtain the second pixel coordinate to be displayed. The second pixel coordinate to be displayed may be exactly one pixel coordinate in the image to be displayed, or it may fall between different pixel coordinates and is not an accurate pixel coordinate. Therefore, based on the second pixel coordinate to be displayed and the pixel values ​​of each pixel coordinate in the image to be displayed, the pixel value Q corresponding to the second pixel coordinate to be displayed is determined, and then the pixel value corresponding to the first pixel coordinate to be displayed is determined based on the pixel value Q corresponding to the second pixel coordinate to be displayed. For example, when the second coordinate to be displayed happens to be a pixel coordinate in the image to be displayed, the pixel value corresponding to this pixel coordinate is used as the pixel value Q corresponding to the second coordinate to be displayed, and the pixel value Q corresponding to the second coordinate to be displayed is used as the pixel value corresponding to the first pixel coordinate to be displayed; or, when the second coordinate to be displayed is not a pixel coordinate in the image to be displayed, the pixel value corresponding to the pixel coordinate in the image to be displayed that is closest to the second coordinate to be displayed is used as the pixel value Q corresponding to the second coordinate to be displayed, or the pixel values ​​corresponding to the four pixel coordinates in the image to be displayed that are closest to the second coordinate to be displayed in the up, down, left, and right directions are used as the pixel value Q corresponding to the second coordinate to be displayed, and the weighted average of the four Qs is calculated as the pixel value corresponding to the first pixel coordinate to be displayed, and so on.

[0136] By performing coordinate transformation on the first pixel coordinate to be displayed in the display screen through the target coordinate mapping transformation relationship, distortion correction can be accurately achieved, and the corresponding second pixel coordinate to be displayed in the image to be displayed can be determined. Then, the pixel value corresponding to the first pixel coordinate to be displayed is determined based on the second pixel coordinate to be displayed and the pixel value of each pixel coordinate in the image to be displayed, so as to display the undistorted image on the display screen and improve the visual experience.

[0137] Example 4

[0138] Figure 11 This is a schematic diagram of the structure of a coordinate mapping device provided in Example 4 of this application. Figure 11 As shown, the device includes: a distortion data acquisition module 410, a mapping transformation relationship determination module 420 and a coordinate mapping relationship determination module 430.

[0139] The distortion data acquisition module 410 is used to acquire grid distortion data of the optical imaging system and generate a radial distribution function corresponding to the tangent value of the field angle corresponding to the pixel point based on the grid distortion data;

[0140] A mapping transformation relationship determination module 420 is configured to determine a mapping transformation relationship between a pixel point on the original image and a pixel point on the display screen based on a relationship between a tangent value of a field angle corresponding to the pixel point and the radial distribution function;

[0141] The coordinate mapping relationship determination module 430 is used to perform parameter solution on the mapping transformation relationship based on at least one preset condition to determine at least one coordinate mapping transformation relationship, wherein the preset condition is set according to an application scenario.

[0142] An embodiment of the present application provides a coordinate mapping device, which solves the problem of only being able to perform a single distortion correction on an image. Based on the relationship between the tangent value of the field of view angle corresponding to the pixel point and the radial distribution function, a mapping transformation relationship between the pixel points on the original image and the pixel points on the display screen is constructed, and the obtained mapping transformation relationship can be solved under different conditions; different preset conditions are set according to different application scenarios, and the mapping transformation relationship is parameter-solved based on at least one preset condition to obtain the parameter value of the mapping transformation relationship, and finally at least one coordinate mapping transformation relationship is determined, and each coordinate mapping transformation relationship can be used to correct the distortion of the image; the embodiment of the present application generates different preset conditions based on different business scenarios, and then determines different coordinate mapping transformation relationships according to different preset conditions. It can flexibly adapt to different application scenarios and perform reasonable distortion correction on images in different application scenarios, so that the end user sees a virtual image without distortion, thereby improving the user experience.

[0143] Optionally, the mapping transformation relationship determination module 420 includes:

[0144] A first expression determining unit, configured to determine a first expression corresponding to the tangent value of the field angle based on a relationship between the tangent value of the field angle corresponding to the distorted pixel point and the radial distribution function, wherein the first expression is related to the radial distribution function;

[0145] a second expression determining unit, configured to determine a second expression corresponding to the tangent value of the field of view angle based on a distance from a pixel point in the original image to a center of the original image, wherein the second expression includes a proportional relationship, the proportional relationship being a proportional relationship between the distance from the pixel point in the original image to the center of the original image and the tangent value of the field of view angle;

[0146] A transformation unit is used to perform transformation based on the first expression and the second expression to determine a mapping transformation relationship between the pixel points on the original image and the pixel points on the display screen.

[0147] Optionally, the coordinate mapping relationship determination module 430 includes:

[0148] a parameter solving unit, configured to solve the mapping transformation relationship equation for each preset condition based on the preset condition, and determine a value of the proportional relationship corresponding to the preset condition;

[0149] The coordinate mapping relationship determining unit is used to substitute the value of the proportional relationship into the mapping transformation relationship formula to obtain the coordinate mapping transformation relationship corresponding to the preset condition.

[0150] Optionally, a parameter solving unit is specifically used to: when the preset condition is the first preset condition, derive the mapping transformation relationship, transform the formula based on the derivation result, and determine the expression of the proportional relationship and the derivative; let the derivative be equal to the rate of change of the center of the original image during the mapping transformation process, solve the expression of the proportional relationship and the derivative, and determine the value of the proportional relationship corresponding to the first preset condition; wherein, the first preset condition is that the rate of change of the center of the original image during the mapping transformation process is 1.

[0151] Optionally, a parameter solving unit is specifically used to: when the preset condition is the second preset condition, determine the first pixel coordinates of the edge of the original image and the second pixel coordinates of the edge of the display screen based on the width of the display screen; solve the mapping transformation relationship based on the first pixel coordinates and the second pixel coordinates to determine the value of the proportional relationship corresponding to the second preset condition; wherein, the second preset condition is that the edge of the image after the mapping transformation is aligned with the edge of the display screen.

[0152] Optionally, a parameter solving unit is specifically used to: when the preset condition is the third preset condition, perform radial linear interpolation on the value of the proportional relationship corresponding to the first preset condition and the value of the proportional relationship corresponding to the second preset condition to obtain the value of the proportional relationship corresponding to the third preset condition; wherein, the third preset condition is that the rate of change of the center of the original image during the mapping transformation process is 1 and the edge of the image after the transition to the mapping transformation is aligned with the edge of the display screen.

[0153] Optionally, the value of the proportional relationship corresponding to the third preset condition is:

[0154] ;

[0155] Among them, L3 is the value of the proportional relationship corresponding to the third preset condition, L2 is the value of the proportional relationship corresponding to the second preset condition, L1 is the value of the proportional relationship corresponding to the first preset condition, W is the width of the display screen, and r is the distance from the pixel point in the display screen to the center of the display screen.

[0156] Optionally, a parameter solving unit is specifically used to: when the preset condition is the fourth preset condition, perform linear interpolation along the radial direction on the value of the proportional relationship corresponding to the second preset condition to obtain the value of the proportional relationship corresponding to the fourth preset condition; wherein, the fourth preset condition is that the edge of the image after mapping transformation is aligned with the edge of the display screen and the degree of correction of the four corners is reduced.

[0157] Optionally, the value of the proportional relationship corresponding to the fourth preset condition is:

[0158] ;

[0159] Among them, L4 is the value of the proportional relationship corresponding to the fourth preset condition, L2 is the value of the proportional relationship corresponding to the second preset condition, W is the width of the display screen, r is the distance from the pixel point in the display screen to the center of the display screen, and D is a pre-set coefficient.

[0160] The coordinate mapping device provided in the embodiments of the present application can execute the coordinate mapping method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0161] Example 5

[0162] Figure 12 This is a structural diagram of an image display device provided in Example 5 of the present application. Figure 12 As shown, the device includes: an application scenario acquisition module 510, a target mapping relationship determination module 520 and a distortion correction module 530.

[0163] Application scenario acquisition module 510, used to obtain application scenario requirement information;

[0164] a target mapping relationship determination module 520, configured to select a matching target coordinate mapping transformation relationship from predetermined coordinate mapping transformation relationships according to the application scenario requirement information, wherein the coordinate mapping transformation relationship is determined according to the coordinate mapping method described in any embodiment of the present application;

[0165] The distortion correction module 530 is configured to perform distortion correction on pixel coordinates in the display screen based on the target coordinate mapping transformation relationship, and display an image based on pixel values ​​corresponding to the pixel coordinates after the distortion correction.

[0166] The embodiment of the present application provides an image display device, which solves the problem of only being able to perform a single distortion correction on an image. Different preset conditions are generated in advance according to different business scenarios, and then different coordinate mapping transformation relationships are determined according to different preset conditions, which can flexibly adapt to different application scenarios. When displaying an image, application scenario requirement information is obtained, and a matching target coordinate mapping transformation relationship is selected from the predetermined coordinate mapping transformation relationship according to the application scenario requirement information. The pixel coordinates in the display screen are subjected to distortion correction based on the matching target coordinate mapping transformation relationship, and then the image is displayed; a suitable target coordinate mapping transformation relationship can be selected according to different application scenarios to perform reasonable distortion correction on the image, so that the end user sees a virtual image without distortion, thereby improving the user experience. The appropriate coordinate mapping transformation relationship is selected according to the specific application scenario requirement information, and the correction of the image is flexible and changeable.

[0167] Optionally, the application scenario acquisition module 510 is specifically configured to determine application scenario requirement information according to the type of the currently running application program.

[0168] Optionally, the distortion correction module 530 includes:

[0169] a coordinate mapping unit, configured to perform coordinate transformation on the first pixel coordinates to be displayed in the display screen based on the target coordinate mapping transformation relationship, and determine the second pixel coordinates to be displayed corresponding to the first pixel coordinates to be displayed in the display screen in the image to be displayed;

[0170] The pixel value determining unit is configured to determine the pixel value corresponding to the first pixel coordinate to be displayed based on the second pixel coordinate to be displayed and the pixel value of each pixel coordinate in the image to be displayed.

[0171] The image display device provided in the embodiments of the present application can execute the image display method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0172] Example 6

[0173] Figure 13 A schematic diagram of an electronic device 60 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0174] like Figure 13 As shown, electronic device 60 includes at least one processor 61 and memory, such as read-only memory (ROM) 62 and random access memory (RAM) 63, communicatively connected to at least one processor 61. The memory stores computer programs executable by the at least one processor. Processor 61 can perform various appropriate actions and processes based on the computer programs stored in ROM 62 or loaded from storage unit 68 into RAM 63. RAM 63 can also store various programs and data required for the operation of electronic device 60. Processor 61, ROM 62, and RAM 63 are interconnected via bus 64. An input / output (I / O) interface 65 is also connected to bus 64.

[0175] Multiple components in the electronic device 60 are connected to the I / O interface 65, including an input unit 66, such as a keyboard, a mouse, etc.; an output unit 67, such as various types of displays, speakers, etc.; a storage unit 68, such as a magnetic disk, an optical disk, etc.; and a communication unit 69, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 69 allows the electronic device 60 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0176] Processor 61 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. Processor 61 executes the various methods and processes described above, such as the coordinate mapping method or the image display method.

[0177] In some embodiments, the coordinate mapping method or image display method can be implemented as a computer program that is tangibly contained in a computer-readable storage medium, such as storage unit 68. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 60 via ROM 62 and / or communication unit 69. When the computer program is loaded into RAM 63 and executed by processor 61, one or more steps of the coordinate mapping method or image display method described above can be performed. Alternatively, in other embodiments, processor 61 can be configured to perform the coordinate mapping method or image display method in any other appropriate manner (e.g., by means of firmware).

[0178] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0179] Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0180] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the coordinate mapping method or image display method described in any embodiment of the present application.

[0181] In the context of the present application, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0182] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0183] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0184] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0185] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved. This is not limited herein.

[0186] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A coordinate mapping method, characterized in that: include: Acquiring grid distortion data of the optical imaging system, and generating a radial distribution function corresponding to a tangent value of a field angle corresponding to a pixel point based on the grid distortion data; Determining a mapping transformation relationship between a pixel point on the original image and a pixel point on the display screen based on a relationship between a tangent value of the field angle corresponding to the pixel point and the radial distribution function; The mapping transformation relationship is parameter-solved based on at least one preset condition to determine at least one coordinate mapping transformation relationship, wherein the preset condition is set according to an application scenario.

2. The method according to claim 1, characterized in that The determining of a mapping transformation relationship between a pixel point on the original image and a pixel point on the display screen based on a relationship between the tangent value of the field angle corresponding to the pixel point and the radial distribution function includes: Determining a first expression corresponding to the tangent value of the field angle based on a relationship between the tangent value of the field angle corresponding to the pixel point and the radial distribution function, where the first expression is related to the radial distribution function; Determining a second expression corresponding to the tangent value of the field of view angle based on a distance from a pixel point in the original image to a center of the original image, wherein the second expression includes a proportional relationship, the proportional relationship being a proportional relationship between the distance from the pixel point in the original image to the center of the original image and the tangent value of the field of view angle; Transformation is performed based on the first expression and the second expression to determine a mapping transformation relationship between the pixel points on the original image and the pixel points on the display screen.

3. The method according to claim 1, characterized in that Solving the mapping transformation relationship equation for parameters based on at least one preset condition to determine at least one coordinate mapping transformation relationship includes: For each preset condition, performing parameter solving on the mapping transformation relationship based on the preset condition to determine a value of the proportional relationship corresponding to the preset condition; Substitute the value of the proportional relationship into the mapping transformation relationship to obtain the coordinate mapping transformation relationship corresponding to the preset condition.

4. The method according to claim 3, characterized in that Solving the parameters of the mapping transformation relationship based on the preset conditions to determine the value of the proportional relationship corresponding to the preset conditions includes: When the preset condition is the first preset condition, deriving the mapping transformation relationship, transforming the formula based on the derivation result, and determining an expression of the proportional relationship and the derivative; Setting the derivative equal to the rate of change of the center of the original image during the mapping transformation process, solving the expression of the proportional relationship and the derivative, and determining the value of the proportional relationship corresponding to the first preset condition; The first preset condition is that the rate of change of the center of the original image during the mapping transformation is 1.

5. The method according to claim 3, characterized in that Solving the mapping transformation relationship equation for parameters based on at least one preset condition to determine a value of a proportional relationship corresponding to each preset condition includes: When the preset condition is the second preset condition, determining the first pixel coordinates of the edge of the original image and the second pixel coordinates of the edge of the display screen based on the width of the display screen; Solving the mapping transformation relationship based on the first pixel point coordinates and the second pixel point coordinates to determine a value of the proportional relationship corresponding to the second preset condition; The second preset condition is that the edge of the image after the mapping transformation is aligned with the edge of the display screen.

6. The method according to claim 3, characterized in that Solving the mapping transformation relationship equation for parameters based on at least one preset condition to determine a value of a proportional relationship corresponding to each preset condition includes: When the preset condition is the third preset condition, linear interpolation is performed along the radial direction on the value of the proportional relationship corresponding to the first preset condition and the value of the proportional relationship corresponding to the second preset condition to obtain the value of the proportional relationship corresponding to the third preset condition; The third preset condition is that the rate of change of the center of the original image during the mapping transformation is 1 and the edge of the image after the mapping transformation is aligned with the edge of the display screen.

7. The method according to claim 6, characterized in that The proportional relationship corresponding to the third preset condition is: ； Among them, L3 is the value of the proportional relationship corresponding to the third preset condition, L2 is the value of the proportional relationship corresponding to the second preset condition, L1 is the value of the proportional relationship corresponding to the first preset condition, W is the width of the display screen, and r is the distance from the pixel point in the display screen to the center of the display screen.

8. The method according to claim 3, characterized in that Solving the mapping transformation relationship equation for parameters based on at least one preset condition to determine a value of a proportional relationship corresponding to each preset condition includes: When the preset condition is the fourth preset condition, linear interpolation is performed along the radial direction on the value of the proportional relationship corresponding to the second preset condition to obtain the value of the proportional relationship corresponding to the fourth preset condition; The fourth preset condition is that the edge of the image after the mapping transformation is aligned with the edge of the display screen and the degree of correction of the four corners is reduced.

9. The method according to claim 8, characterized in that The proportional relationship corresponding to the fourth preset condition is: ； Among them, L4 is the value of the proportional relationship corresponding to the fourth preset condition, L2 is the value of the proportional relationship corresponding to the second preset condition, W is the width of the display screen, r is the distance from the pixel point in the display screen to the center of the display screen, and D is a pre-set coefficient.

10. An image display method, characterized in that: include: Obtain application scenario requirement information; selecting a matching target coordinate mapping transformation relationship from a predetermined coordinate mapping transformation relationship according to the application scenario requirement information, wherein the coordinate mapping transformation relationship is determined by the coordinate mapping method according to any one of claims 1 to 9; The pixel coordinates in the display screen are subjected to distortion correction based on the target coordinate mapping transformation relationship, and the image is displayed based on the pixel values ​​corresponding to the pixel coordinates after the distortion correction.

11. The method according to claim 10, characterized in that The obtaining of application scenario requirement information includes: Determine the application scenario requirements based on the type of application currently running.

12. The method according to claim 10, characterized in that The performing distortion correction on pixel coordinates in the display screen based on the target coordinate mapping transformation relationship includes: Performing coordinate transformation on the first pixel coordinate to be displayed in the display screen based on the target coordinate mapping transformation relationship, and determining the second pixel coordinate to be displayed corresponding to the first pixel coordinate to be displayed in the display screen in the image to be displayed; The pixel value corresponding to the first pixel coordinate to be displayed is determined based on the second pixel coordinate to be displayed and the pixel value of each pixel coordinate in the image to be displayed.

13. A coordinate mapping device, characterized in that: include: A distortion data acquisition module, configured to acquire grid distortion data of the optical imaging system and generate a radial distribution function corresponding to a tangent value of a field of view angle corresponding to a pixel point based on the grid distortion data; A mapping transformation relationship determination module is used to determine a mapping transformation relationship between a pixel point on the original image and a pixel point on the display screen based on a relationship between a tangent value of the field angle corresponding to the pixel point and the radial distribution function; The coordinate mapping relationship determination module is used to solve the parameters of the mapping transformation relationship based on preset conditions to determine the coordinate mapping transformation relationship.

14. An image display device, characterized in that: include: Application scenario acquisition module, used to obtain application scenario requirement information; a target mapping relationship determination module, configured to select a matching target coordinate mapping transformation relationship from predetermined coordinate mapping transformation relationships according to the application scenario requirement information, wherein the coordinate mapping transformation relationship is determined according to the coordinate mapping method according to any one of claims 1 to 9; The distortion correction module is used to perform distortion correction on the pixel coordinates in the display screen based on the target coordinate mapping transformation relationship, and display the image based on the pixel values ​​corresponding to the pixel coordinates after the distortion correction.

15. An electronic device, characterized in that: The electronic device comprises: at least one processor, and a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the coordinate mapping method described in any one of claims 1 to 9 or the image display method described in any one of claims 10 to 12.

16. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the coordinate mapping method according to any one of claims 1 to 9 or the image display method according to any one of claims 10 to 12 when executed.

17. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the coordinate mapping method according to any one of claims 1 to 9 or the image display method according to any one of claims 10 to 12.

Images