Image correction method and related device

Abstract

The invention discloses an image correction method and a related device, and relates to the field of image processing, and the method comprises the steps: obtaining original texture data, a deviation angle and rendering region parameters, the deviation angle being a deviation angle of a rendering region relative to a fragment region, and the fragment region being a region range corresponding to a shader; based on the deviation angle and the rendering area parameters, a shader is adopted to conduct rotation correction on the original texture data, target texture data are obtained, and the target texture data are matched with a projection area of a projection light machine. According to the method, the parameters of the projection light machines are obtained in advance to determine the parameters of the rendering area on the basis of installation for different projection light machines, the deviation angle data of each AR HUB collector only need to serve as the parameters to be input into the shader of the OpenGL frame, texture rotation for the rendered image is achieved through the shader, the efficiency is high, the processing delay is low, and the method is suitable for large-scale popularization and application. And the AR HUD projection light machine requiring high-frame-rate refresh display is facilitated.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image correction method and related devices. Background Art

[0002] During the installation of an in-vehicle AR (Augmented Reality) HUD (Head Up Display) projection engine, process limitations can cause precision errors in the seats, which visually appear as an angle offset in the projection area and require correction at the software level.

[0003] Correction for angular offset needs to be performed after distortion correction. Traditionally, the distorted data points are rotated and offset by a fixed angle, then interpolated. This outputs a pixel bubble map with a fixed deflection angle, storing the pixel displacement of the original rendered pixel in the x and y directions in the R (red) and G (green) channels of the bubble map. In the rendering engine, off-screen rendering technology is used to fuse the current frame texture with the bubble map to obtain a texture that has been corrected for both distortion and angular offset. Finally, this texture is added to the rendering area, achieving software-level correction for angular offset in the projection area.

[0004] However, since it is necessary to pre-process the distortion data and generate pixel bubble maps based on the angle offset, different offset angles need to be processed separately. This method is not suitable for correcting dynamic offset angles, and has low correction efficiency for large quantities of in-vehicle AR HUD devices. Summary of the Invention

[0005] A first aspect of the present application provides an image correction method, comprising: Obtain original texture data, an offset angle, and rendering area parameters, where the offset angle is the angle at which the rendering area is offset relative to the fragment area, and the fragment area is the area range corresponding to the shader; Based on the offset angle and the rendering area parameters, a shader is used to perform rotation correction on the original texture data to obtain target texture data, and the target texture data matches the projection area of the projection light machine.

[0006] In a possible implementation, performing rotation correction on the original texture data based on the offset angle and the rendering area parameter to obtain target texture data includes: Determining a coordinate transformation relationship between a fragment region and the rendering region based on the rendering region parameters; Based on the coordinate transformation relationship and the offset angle, coordinate rotation is performed on each pixel coordinate in the original texture data to obtain a target coordinate set; The target coordinate set is sampled to obtain target texture data matching the rendering area.

[0007] In a possible implementation, performing coordinate rotation on each pixel coordinate in the original texture data based at least on the coordinate transformation relationship and the offset angle to obtain a target coordinate set includes: Based on the coordinate transformation relationship, transform each pixel coordinate in the original texture data to obtain a first coordinate set, where the first coordinate set corresponds to the fragment area; Based at least on the offset angle, rotation processing is performed on each coordinate in the first coordinate set to obtain a target coordinate set.

[0008] In one possible implementation, performing rotation processing on each coordinate in the first coordinate set based at least on the offset angle to obtain a target coordinate set includes: Based on the offset angle, rotation processing is performed on each coordinate in the first coordinate set to obtain a second coordinate set, where the horizontal coordinate and the vertical coordinate of any coordinate in the second coordinate set meet the rendering area parameters; Based on the coordinate conversion relationship, each coordinate in the second coordinate set is restored to the first coordinate system of the shader to obtain a target coordinate set.

[0009] In a possible implementation, sampling the target coordinate set to obtain target texture data matching the rendering area includes: Determining a sampling range based on the rendering area parameters; Within the sampling range, texture sampling is performed on the texture data corresponding to the target coordinate set to obtain target texture data matching the rendering area.

[0010] In a possible implementation, before performing rotation correction on the original texture data based on the offset angle and the rendering area parameter to obtain target texture data, the method includes: Initialize the shader according to the offset angle and rendering area parameters; The original texture data is input into the shader, triggering the step of performing rotation correction on the original texture data based on the offset angle and the rendering area parameter to obtain target texture data.

[0011] In a possible implementation, after performing rotation correction on the original texture data based on the offset angle and the rendering area parameter to obtain target texture data, the method further includes: storing the target texture data in a texture buffer; Binding material parameters to the texture buffer to obtain material information; The mesh parameters are bound to the material information to obtain the texture data to be output.

[0012] A second aspect of the present application provides an image correction device, comprising: An acquisition module is used to obtain original texture data, an offset angle, and rendering area parameters, wherein the offset angle is the angle at which the rendering area is offset relative to the fragment area, and the fragment area is the area range corresponding to the shader; A correction module is used to perform rotation correction on the original texture data using a shader based on the offset angle and the rendering area parameters to obtain target texture data, where the target texture data matches the projection area of the projection light machine.

[0013] A third aspect of the present application provides a computer program product, comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the image correction method according to the first aspect or any implementation of the first aspect.

[0014] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the image correction method of the first aspect or any implementation of the first aspect.

[0015] In a fifth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can use the image correction method of the first aspect or any implementation of the first aspect.

[0016] In summary, the present application provides an image correction method and related devices, the method comprising: obtaining original texture data, an offset angle, and rendering area parameters, the offset angle being the angle at which the rendering area is offset relative to the fragment area corresponding to the shader; based on the offset angle and rendering area parameters, using a shader to perform rotation correction on the original texture data to obtain target texture data that matches the projection area of the projection light machine. For the installation of different projection light machines, the parameters of the projection light machine are obtained in advance to determine the rendering area parameters. It is only necessary to use the offset angle data of each AR HUB collector as a parameter and input it into the shader of the OpenGL framework. The shader then implements texture rotation for the rendered image. Compared with traditional offset correction schemes, this method is more efficient and has extremely low processing delay, which is more beneficial for AR HUD projection light machines that require high frame rate refresh displays. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0018] Figure 1 This is a flow chart of an image correction method provided in an embodiment of the present application; Figure 2 This is a flow chart of performing rotation correction on the texture data based on the offset angle and the rendering area parameters to obtain target texture data, as provided in an embodiment of the present application; Figure 3 Schematic diagram of the coordinate system of the shader and rendering area provided in the embodiment of the present application; Figure 4 is a schematic diagram of a process of moving the rotation anchor point of a rendering area to the center of an image, provided by an embodiment of the present application; Figure 5 is a flow chart of performing coordinate rotation on each pixel coordinate in the original texture data based on at least the coordinate transformation relationship and the offset angle to obtain a target coordinate set, as provided by an embodiment of the present application; Figure 6 is a flowchart of performing rotation processing on each coordinate in the first coordinate set based at least on the offset angle to obtain a target coordinate set, provided by an embodiment of the present application; Figure 7 This is a schematic diagram of a pixel rotation restoration process provided by an embodiment of the present application; Figure 8 This is a flow chart of sampling the target coordinate set to obtain target texture data matching the rendering area, as provided in an embodiment of the present application; Figure 9 This is a sampling diagram provided in the embodiments of the present application; Figure 10 This is a schematic diagram of a process for initializing a shader provided by an embodiment of the present application; Figure 11 This is a schematic diagram of a process for processing target texture data provided by an embodiment of the present application; Figure 12 This is a flowchart of an application scenario of an image correction method provided by this application; Figure 13 This is a flowchart of shader rendering in the application scenario of an image correction method provided by this application Figure 14 Schematic diagram of the structure of an image correction device provided in an embodiment of the present application; Figure 15 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The terms used in the implementation methods of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0020] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0021] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0022] Figure 1 This is a flow chart of an image correction method provided in an embodiment of the present application. Figure 1 As shown, an image correction method provided by an embodiment of the present application may include steps 101 to 102, and these steps are described in detail below.

[0023] 101. Obtain original texture data, an offset angle, and rendering area parameters. The offset angle is the angle at which the rendering area is offset relative to the fragment area. The fragment area is the area range corresponding to the shader. Among them, the original texture data is the image frame data obtained after rendering, and the original texture data is the data that should be output to the window for display. In this solution, the original texture data is output to the window for display after the image correction method provided by this solution. From a visual perspective, the angle of the projection area is no longer offset, thereby achieving the purpose of correcting the offset angle.

[0024] In a possible implementation, the original texture data may be distortion-corrected data.

[0025] Among them, the rendering area refers to the area corresponding to the output of the original texture data, the offset angle is the angular offset of the rendering area relative to the fragment area, and the angular offset is the angular offset of the projection area visually presented by the output image of the projection light machine.

[0026] The fragment area is the area range where the fragment shader processes the image and can be set according to the shader.

[0027] Among them, for the projection light machine installed in different vehicles, the parameters of the projection light machine can be obtained in advance, and the rendering area parameters can be determined based on the parameters. Even if the accuracy error of the seat occurs during the installation of the projection light machine, it is only necessary to detect and obtain the offset angle, and apply it to the image correction method provided in this application. The image correction method is suitable for correcting various offset angles.

[0028] The region range of the shader may correspond to one coordinate system, and the region range of the rendering region may correspond to another coordinate system. There is a certain difference between the two coordinate systems, and the difference corresponds to the offset angle.

[0029] The rendering area parameters may include parameters such as the resolution of the rendering area in the horizontal and vertical directions, and the aspect ratio of the rendering area.

[0030] 102. Based on the offset angle and the rendering area parameter, a shader is used to perform rotation correction on the original texture data to obtain target texture data, where the target texture data matches the projection area of the projection light machine.

[0031] The shader is a shader in an OpenGL (Open Graphics Library) framework, and the shader may include a vertex shader and a fragment shader, so as to implement rotation correction for original texture data through the shader.

[0032] Among them, OpenGL controls the GPU (Graphics Processing Unit) computing unit to perform calculations. Since OpenGL is more inclined to three-dimensional graphics calculations, the shader is the way graphics rendering controls the GPU computing unit to perform calculations. OpenGL has a higher efficiency in three-dimensional rendering capabilities.

[0033] The shader uses GLSL (OpenGL Shading Language) scripts for GPU acceleration.

[0034] In practical applications, the process of performing rotation correction on the original texture data by the shader is the process of rendering the original texture data.

[0035] The offset angle and rendering area parameters are configured into a shader, and the shader performs rotation correction on the original texture data according to the configured offset angle and rendering area parameters to obtain target texture data.

[0036] In one possible implementation, based on the rendering area parameters, the coordinate transformation relationship between the fragment area and the rendering area is first determined, and then based on the coordinate transformation relationship and the offset angle, the coordinate rotation in the original texture data is realized, and finally the rotated coordinates are adopted to obtain the target texture data. Figure 2 This process is described in detail in .

[0037] In this embodiment, an off-screen rendering technology is used. Specifically, a shader in an OpenGL framework is used to perform off-screen rendering processing on the original texture data to rotate the texture and correct the offset angle.

[0038] In one possible implementation, the OpenGL framework renders the original image to obtain the original texture data, which is then corrected by the image correction method in this solution and output to the window for display. The off-screen rendering method adopted in this solution serves as a buffer between OpenGL and the window, and the original texture data is corrected in this buffer.

[0039] In actual applications, since the video displayed in the window includes multiple frames of images, step 102 can be executed cyclically for each frame of image until the display stops, thereby terminating the cyclic process.

[0040] In this loop process, the buffered original texture data is first cleared, the frame buffer is bound, and the shader GLSL is specified and enabled. The GLSL performs the above-mentioned rotation correction process on the original texture data received in real time to achieve rendering for the original texture data.

[0041] In a possible implementation, after obtaining the offset angle and rendering area parameters, the shader can be initialized according to the offset angle and rendering area parameters. Figure 10 The initialization process is described in .

[0042] In a possible implementation, the target texture data also needs to be bound to the material parameters and mesh parameters to obtain the output texture data that can be output to the window. Figure 11 The processing process of the target texture data is described in .

[0043] In this embodiment, the original texture data, offset angle, and rendering area parameters are obtained. The offset angle is the angle at which the rendering area is offset relative to the fragment area corresponding to the shader. Based on the offset angle and rendering area parameters, the shader is used to perform rotation correction on the original texture data to obtain target texture data that matches the projection area of the projector. For different projector installations, the projector parameters are obtained in advance to determine the rendering area parameters. Only the offset angle data of each AR HUB collector needs to be input as a parameter into the shader of the OpenGL framework. The shader then implements texture rotation for the rendered image. Compared with traditional offset correction solutions, this is more efficient and has extremely low processing latency, making it more beneficial for AR HUD projectors that require high frame rate refresh displays.

[0044] Figure 2 This is a flowchart of performing rotation correction on the texture data based on the offset angle and the rendering area parameters to obtain target texture data provided by an embodiment of the present application. It can include steps 201 to 203, and these steps are described in detail below.

[0045] 201. Determine a coordinate transformation relationship between a fragment region and the rendering region based on the rendering region parameters; The rendering area parameters may include the resolutions of the rendering area in the horizontal and vertical directions. The conversion relationship between the fragment area and the rendering area is determined by the rendering area parameters.

[0046] Among them, the fragment area and the rendering area use different coordinate systems, and there is a certain offset between the two coordinate systems. In this embodiment, the rendering area is first scaled to unify its coordinate system with the fragment area, and the center of the corresponding image of the rendering area is offset to the origin of the original coordinate system, and the coordinate transformation relationship between the two is further determined, and then the texture data is rotated through this coordinate transformation relationship.

[0047] Figure 3 : This is a schematic diagram of the coordinate system of the shader and rendering area provided in an embodiment of the present application, wherein (a) shows the coordinate system of the shader. In the coordinate system of the shader, the Y axis is in the increasing direction upward and the X axis is in the increasing direction to the right. The coordinates of four points are shown in the figure, namely the origin (0, 0), (0, 1), (1, 0) and (1, 1). The rectangular area determined by the four points is the fragment area. (b) shows the coordinate system of the rendering area. In the coordinate system of the rendering area, the Y axis is in the increasing direction downward and the X axis is in the increasing direction to the right. The coordinates of four points are shown in the figure, namely the origin (0, 0), (0, 1), (1, 0) and (1, 1). The rectangular area determined by the four points is the rendering area. In the figure, dotted lines are used to connect points with the same coordinates to indicate the correspondence between the coordinates in the two coordinate systems.

[0048] Among them, the coordinate system of the fragment area adopts Indicates that the original coordinates of the pixels in the original texture data are Indicates that the coordinate system is offset so that the coordinate system corresponding to the original texture data is offset to the origin of the coordinate system corresponding to the center of the rendering area (the origin is also the origin of the original coordinate system). The following formula (1) is used to express the offset: (1) in, Represents the original coordinate normalization matrix, Trans represents the coordinate transformation matrix, Represents the original coordinate system.

[0049] Should The normalized matrix can be expressed as follows: (2) in, ,

[0050] Among them, the Indicates the compression of the original texture data to the horizontal coordinate corresponding to the rendering area. Indicates the vertical coordinate corresponding to the rendering area for compressing the original texture data; Indicates the resolution of the horizontal axis of the rendering area. Indicates the resolution of the rendering area in the vertical axis direction; u represents the horizontal coordinate of the original coordinate of the pixel in the original texture data, and v represents the vertical coordinate of the original coordinate of the pixel in the original texture data.

[0051] Among them, the and The value range is [0,1]. Trans is the coordinate transformation matrix. The coordinate system is scaled by two units and then shifted right by one unit so that the rotation anchor point is at the center of the image corresponding to the original texture data. At this time, the transformation matrix is expressed as follows: (3) The aspect ratio of the rendering area can be expressed using the following formula (4): (4) According to the above content, the corresponding coordinate transformation relationship is expressed as follows: (5) in, Represents the horizontal coordinate of the pixel in the original coordinate system, Indicates the vertical coordinate of the pixel in the original coordinate system, where the center of the image is the origin; u indicates the horizontal coordinate of the original coordinate of the pixel in the original texture data, and v indicates the vertical coordinate of the original coordinate of the pixel in the original texture data. Indicates the compression of the original texture data to the horizontal coordinate corresponding to the rendering area. Indicates the compression of the original texture data to the vertical coordinate corresponding to the rendering area. Indicates inverted sampling.

[0052] Among them, the horizontal axis The corresponding rendering interval is [-1,1], the vertical axis The corresponding rendering interval is [- , ], since the origin of the original texture data coordinate system is in the upper left corner, and the origin of the fragment area coordinate system is in the lower left corner, the correct sampling of the subsequent texture is guaranteed by inverting the vertical axis.

[0053] For example, the coordinates in the fragment region coordinate system are (0,0), and the corresponding sampling texture point coordinates are (0,1). The built-in symbols in the GLSL script can be used to perform inverse sampling.

[0054] Subsequently, the coordinate transformation relationship may be used to process each pixel coordinate in the original texture data.

[0055] Figure 4 This is a schematic diagram of the process of moving the rotation anchor point of the rendering area to the center of the image provided by an embodiment of the present application.

[0056] Among them, the Figure 4 (a) shows the fragment region coordinate system and the original coordinates of the pixels The origin O of the fragment region coordinate system is marked in (a).

[0057] Among them, the Figure 4 In (b), the coordinate system corresponding to the original texture data is shifted to the position of the origin of the coordinate system corresponding to the center of the rendering area. The offset coordinate system is Coordinates after pixel shift ,Should It is the offset coordinate system, which is the original coordinate system with the center of the image as the origin.

[0058] According to the coordinate transformation relationship (transformation matrix Trans), the center of the rendering area corresponding to the original texture data can be offset to the origin of the original coordinate system. The offset can be ( ).

[0059] 202. Based on the coordinate transformation relationship and the offset angle, perform coordinate rotation on each pixel coordinate in the original texture data to obtain a target coordinate set; Wherein, based on the coordinate conversion relationship determined in the above steps and the offset angle, coordinate rotation is performed on each pixel coordinate in the original texture data, and the rotated pixel coordinates are coordinates after the offset angle is corrected.

[0060] In a possible implementation, the pixel coordinates can be first transformed based on the coordinate transformation relationship, and then rotated based on the offset angle. Figure 5 The rotation process is described in detail in .

[0061] 203. Sample the target coordinate set to obtain target texture data matching the rendering area.

[0062] The target coordinate set is sampled to restore the rotated coordinate points to the coordinate system of the shader, so that the shader can process the pixels therein to obtain target texture data.

[0063] The function of the linear sampling operation may be preset, for example, a built-in function in GLSL may be used.

[0064] For example, this built-in function can take a texture2D.

[0065] In a possible implementation, the sampling range can be determined based on the rendering area parameters. Sampling is effective within the sampling range. The sampling range is within the fragment area, and only the target texture data within the fragment area is sampled. Figure 8 The sampling process is described in detail.

[0066] In this embodiment, based on the rendering area parameters, the coordinate transformation relationship between the fragment area and the rendering area is determined; based on the coordinate transformation relationship and the offset angle, the coordinate rotation is performed on each pixel coordinate in the original texture data to obtain a target coordinate set; the target coordinate set is sampled to obtain target texture data matching the rendering area, and by determining the coordinate transformation relationship between the fragment area and the rendering area, combined with the offset angle, and then sampling to obtain the target texture data matching the rendering area, rotation correction is achieved for the original texture data and texture data corresponding to the projection area of the projection light machine is obtained. This process is a universal processing flow, which can be performed for different shaders, is suitable for offset correction of various projection light machines, and has wide applicability.

[0067] Figure 5The embodiment of the present application provides a flow chart for performing coordinate rotation on each pixel coordinate in the original texture data based at least on the coordinate transformation relationship and the offset angle to obtain a target coordinate set, which may include steps 501 to 502. These steps are described in detail below.

[0068] 501. Based on the coordinate transformation relationship, transform each pixel coordinate in the original texture data to obtain a first coordinate set, where the first coordinate set corresponds to the fragment area; The coordinate transformation relationship is the coordinate transformation relationship between the fragment area and the rendering area.

[0069] The original texture data includes a number of pixels, and the coordinates of each pixel are converted to obtain coordinates corresponding to the fragment area, and the obtained coordinates are used as the first coordinate set.

[0070] In a specific implementation, the coordinate transformation relationship formula determined by the aforementioned formulas (3), (4) and (5) can be used to substitute the coordinate point of each pixel in the original texture data into the coordinate transformation relationship formula to obtain the corresponding coordinate set.

[0071] Among them, each pixel coordinate in the original texture data is represented by (u, v), and correspondingly, each coordinate in the first coordinate set is represented by ( , )express.

[0072] 502. Perform rotation processing on each coordinate in the first coordinate set based at least on the offset angle to obtain a target coordinate set.

[0073] The center of the image composed of each coordinate in the first coordinate set coincides with the origin of the original coordinate system.

[0074] Among them, the offset angle is the angle between the coordinate systems of the fragment area and the rendering area. Through the above-mentioned coordinate transformation, the original texture data is moved to the origin of the original coordinate system as the center. However, the angle between the coordinate systems of the fragment area and the rendering area has not been removed. In this step, each coordinate in the first coordinate set is rotated at least based on the offset angle to obtain the target coordinate set.

[0075] Among them, each pixel in the target coordinate set corresponds to a fragment area, and the pixels corresponding to the target coordinate set can be output to a window later, and the texture data displayed in the window corresponds to the projection area of the projection light machine.

[0076] In a possible implementation, the rotation mapping and coordinate restoration process can be performed for each coordinate point in the rendering area, and the aspect ratio of the rendering area is preserved during the restoration process. Figure 6The rotation reduction process is described in detail in .

[0077] In one possible implementation, the GLSL script performs parallel calculations on the coordinates of each pixel in the original texture and utilizes a GPU for acceleration.

[0078] In this embodiment, each pixel coordinate in the original texture data is first converted according to the coordinate conversion relationship determined in the aforementioned process to obtain a first coordinate set, which corresponds to the fragment area; then, based on at least the offset angle, each coordinate in the first coordinate set is rotated to obtain a target coordinate set, which is a coordinate set that compensates for the offset angle. The image composed of the pixels in the coordinate set matches the projection area of the projection light machine, and the image output by the projection area visually presents a consistent angle, thereby achieving correction of the original texture data.

[0079] Figure 6 The embodiment of the present application provides a flow chart of performing rotation processing on each coordinate in the first coordinate set based at least on the offset angle to obtain a target coordinate set, which may include steps 601 to 602. These steps are described in detail below.

[0080] 601. Based on the offset angle, rotate each coordinate in the first coordinate set to obtain a second coordinate set, where the horizontal coordinate and the vertical coordinate of any coordinate in the second coordinate set satisfy the rendering area parameters. The coordinates of each pixel in the original texture data are transformed through the coordinate transformation relationship, so as to move the original texture data to the origin of the original coordinate system as the center.

[0081] Then, a rotation process is performed on each coordinate in the first coordinate set, and the coordinates after the rotation process meet the rendering area parameters. The area formed by the coordinates after the rotation process corresponds to the rendering area.

[0082] The rotation process can be expressed by the following formula (6): (6) in, Represents the horizontal coordinate of the rotated pixel, Represents the vertical coordinate of the rotated pixel, represents the offset angle, Represents the horizontal coordinate of the pixel in the original coordinate system (the horizontal coordinate after pixel conversion), Represents the vertical coordinate of the pixel in the original coordinate system (the vertical coordinate after pixel conversion).

[0083] 602. Based on the coordinate conversion relationship, restore each coordinate in the second coordinate set to the first coordinate system of the shader to obtain a target coordinate set.

[0084] Among them, after the pixel coordinates are rotated, the rotated coordinates are restored to the coordinate system of the shader, so that the shader processes each pixel based on the coordinates of each pixel to obtain the processed texture data. When the texture data is sent to the window output, it matches the projection area of the projector, and the image output by the projection area visually presents a consistent angle.

[0085] The coordinate transformation relationship determined in the above steps is used to restore each coordinate in the second coordinate set to the first coordinate system of the shader by adopting an inverse process, thereby obtaining data that the shader can directly use to process pixels using the coordinates.

[0086] The reduction process can be expressed by the following formula (7): (7) in, Represents the horizontal coordinate of the pixel in the first coordinate system, Represents the vertical coordinate of the pixel in the first coordinate system, Represents the horizontal coordinate of the rotated pixel, Represents the vertical coordinate of the rotated pixel, Represents the inverse process of coordinate transformation relationship.

[0087] The above formula (7) is used to restore the rotated coordinates to the first coordinate system of the shader, where the coordinates of the horizontal axis are When restored to the first coordinate system, the vertical axis coordinate retains the aspect ratio of the rendering area and the inversion operation.

[0088] Figure 7 This is a schematic diagram of the pixel rotation restoration process provided by the embodiment of the present application. Figure 7 (a) shows the range of pixels after rotation. represents the radius of rotation, Represents the original coordinate system, wherein the corresponding range 701 of each pixel after rotation is framed by solid lines, and the corresponding range 702 before rotation is framed by dotted lines. Figure 7 (b) shows the pixels after rotation restored to the shader coordinate system, wherein the pixel corresponding range 703 restored to the shader coordinate system is framed by solid lines, and the corresponding fragment area is framed by dotted lines as a rendering range 704.

[0089] In this embodiment, based on the offset angle, each coordinate in the first coordinate set is rotated to obtain a second coordinate set, and the horizontal coordinate and the vertical coordinate of any coordinate in the second coordinate set meet the rendering area parameters; based on the coordinate conversion relationship, each coordinate in the second coordinate set is restored to the first coordinate system of the shader to obtain a target coordinate set. In this process, each coordinate in the first coordinate set is first rotated based on the offset angle, and then the rotated coordinates are restored to the coordinate system of the shader based on the rendering area parameters, so that the aspect ratio of the pixels restored to the shader coordinate system is maintained to ensure the integrity of the texture.

[0090] Figure 8 This is a flow chart of sampling the target coordinate set to obtain target texture data matching the rendering area provided by an embodiment of the present application, which may include steps 801 to 802. These steps are described in detail below.

[0091] 801. Determine a sampling range based on the rendering area parameters; GLSL performs texture sampling on the rotated and restored texture data to obtain target texture data, which is sent to the window for display.

[0092] In order to ensure the display effect, the aspect ratio of the original texture needs to be maintained.

[0093] Accordingly, in this embodiment, the sampling range is determined based on the rendering area parameters to ensure that the image composed of pixels corresponding to the pixel coordinates sampled in the sampling range maintains the display effect of the original texture.

[0094] The sampling range can be determined using the following formulas (8) and (9): (8) (9) in, Represents the horizontal coordinate of the pixel in the first coordinate system, α represents the horizontal axis in the first coordinate system The sampling coordinates of represents the vertical coordinate of the pixel in the first coordinate system, and β represents the vertical axis in the first coordinate system The sampling coordinates of Indicates the aspect ratio of the rendering area. Its calculation method can refer to the above formula (4).

[0095] The above formula (8) expresses the restriction The sampling area, formula (9) expresses the restriction sampling area.

[0096] 802. Within the sampling range, perform texture sampling on the texture data corresponding to the target coordinate set to obtain target texture data matching the rendering area.

[0097] After the sampling range is determined, texture sampling is performed only on the texture data corresponding to each pixel in the pixel coordinate set within the sampling range to obtain target texture data.

[0098] In a possible implementation, the sampling step may be performed using a built-in function of the shader GLSL.

[0099] The texture sampling process can be performed using the following formula (10): (10) Among them, O represents the target texture obtained by sampling and rendering, I represents the off-screen texture to be sampled, is the linear sampling operation function, Indicates the coordinate point is The sampling point vector.

[0100] Among them, due to The aspect ratio is maintained. The coordinate axes are normalized and scaled in the above process. Therefore, the vertical coordinate of the texture sampling point needs to be restored and scaled to ensure the integrity of the texture.

[0101] After the aforementioned rotation process, the target texture data is sampled and input into GLSL (shader script) for subsequent processing such as input color texture by GLSL. Figure 11 This process is described in detail in .

[0102] Figure 9 is a sampling diagram provided in an embodiment of the present application, including an initial image 901 corresponding to the original texture data, an intermediate image 902 that has been rotated and restored, a rendering area 903, and a final image 904 corresponding to the final target texture data. The original texture data corresponding to the initial image undergoes processes such as rotation center offset, rotation, and restoration to obtain intermediate image 902. The overlapping area between intermediate image 902 and rendering area 903 serves as a restricted sampling area. Texture sampling is performed on the texture data corresponding to the pixels in this restricted sampling area to obtain a final image 904 corresponding to the target texture data. In the figure, long dashed lines represent the rendering area, and short dashed lines represent corresponding points between the initial image, intermediate image, and final image.

[0103] In this embodiment, a sampling range is determined based on the rendering area parameters; within the sampling range, texture sampling is performed on the texture data corresponding to the target coordinate set to obtain target texture data that matches the rendering area. In this process, a rotated area restriction is added to retain the pixel area ratio of the original image, thereby ensuring the integrity of the texture sampling.

[0104] Figure 10 This is a flowchart for shader initialization provided by an embodiment of the present application, which may include steps 1001 to 1002. These steps are described in detail below.

[0105] 1001. Initialize the shader according to the offset angle and rendering area parameters; Among them, this solution is applied to the shader of the OpenGL framework.

[0106] After the framework is started, the shader program performs preparation work to achieve initialization.

[0107] Among them, since the offset angle and rendering area parameters of the installed AR HUB projector are fixed, the offset angle and rendering area can be used to initialize the shader during initialization.

[0108] The initialization process may include: frame buffer creation, shader parameter setting, etc.

[0109] The frame buffer is used to store the output color of the shader; the shader parameters may include an offset angle, a rendering area parameter, etc. The shader parameters may also include input and output off-screen texture parameters.

[0110] In a specific implementation, the shader parameter may be passed into the shader program from host memory.

[0111] Wherein, based on the offset angle and the rendering area parameter, the original texture data is subjected to rotation correction, and the shader is initialized before the target texture data is obtained.

[0112] 1002. Input the original texture data into the shader, trigger the step of performing rotation correction on the original texture data based on the offset angle and the rendering area parameter to obtain target texture data.

[0113] After the shader is initialized, a rendering cycle can be started. The rendering cycle is a process of performing rotation correction and rendering on the initial texture data of each frame image received in real time.

[0114] In the rendering cycle, the initial texture data of each frame image is input into the shader, processed by the shader, and the color is output to the frame buffer. In addition, the frame buffer is cleared before the next cycle starts.

[0115] In this embodiment, it also includes: initializing the shader based on the offset angle and the rendering area parameters; inputting the original texture data into the shader, triggering the rotation correction of the original texture data based on the offset angle and the rendering area parameters, and obtaining the target texture data step, and pre-initializing the shader to provide a basis for subsequent rotation correction of the original texture data.

[0116] Figure 11 This is a flow chart of processing target texture data provided by an embodiment of the present application, which may include steps 1101 to 1103. These steps are described in detail below.

[0117] 1101. Store the target texture data into a texture buffer; After rotation correction is performed on the original texture data to obtain target texture data, the target texture data is first stored in the texture buffer.

[0118] The texture buffer caches information such as the color of the target texture data of the current frame, and the color information corresponds to the target texture data.

[0119] 1102. Bind the material parameters to the texture buffer to obtain material information; Among them, the material parameters are bound to the texture buffer to obtain the material information (Material).

[0120] The shader rendering engine consists of entities, materials, and GLSL scripts (including vertex shaders and fragment shaders). The shader includes vertex shaders and fragment shaders, which specify the rendering position and color information, respectively, to achieve the final rendering effect. Some calculation operations can also be implemented in GLSL scripts.

[0121] Among them, since the entity itself has no color, it needs to be bound to the material to see the color, area and other information.

[0122] Among them, the vertex shader and fragment shader bound in the material parameters, the vertex shader stores the vertex information of the entity, the corresponding texture coordinate information, etc., and the fragment shader can generate color data.

[0123] 1103. Bind the mesh parameters to the material information to obtain texture data to be output.

[0124] The grid parameter is a method of dividing the area corresponding to the window display. The grid parameter is bound to the material information to determine the display content of each area in the window and obtain the texture data to be output.

[0125] After the texture data to be output is obtained, the texture data to be output is sent to the window for display output. The texture buffer is cleared to provide cache space for the original texture data processing of the next frame image.

[0126] In this embodiment, the target texture data is first stored in a texture buffer, the texture buffer is bound to the material parameters in turn to obtain material information, and then the grid parameters are bound to the material information to obtain the texture data to be output. The texture data to be output is data that can be directly displayed on the window, realizing the complete off-screen rendering process.

[0127] Figure 12 This is a flow chart of an application scenario of an image correction method provided by this application. In this application scenario, the image correction method is executed using a shader of the OpenGL framework.

[0128] The flowchart includes two parts: OpenGL initialization process and shader rendering cycle process.

[0129] The OpenGL initialization process includes the following steps: 1201. Import shader script; 1202. Create a frame buffer; The frame buffer is a buffer area created for the image correction method, and the buffer area can buffer various data generated during the rendering process of the original texture data of a frame of image.

[0130] 1203. Initialize shader parameters; Among them, OpenGL and its shader are initialized in this process.

[0131] The shader rendering cycle process includes the following steps: Trigger the rendering process based on the received offset angle information and the aspect ratio of the rendering area.

[0132] 1204. Frame buffer data is cleared; After the rendering cycle is started, the frame buffer is first processed for data to provide storage space for subsequent rendering processes.

[0133] 1205. Bind frame buffer; The purpose of binding the frame buffer is to ensure that the frame buffer provides storage space for the current rendering process.

[0134] 1206. Specify and start the shader program; There may be multiple shaders in the device applying the method, and the shader for executing the image correction method this time is selected.

[0135] 1207. Rendering is performed based on the received original texture data.

[0136] The shader is used to perform rendering on the received original texture data, and the rendering process is a process of rotating the original texture data.

[0137] Figure 13 This is a flowchart of shader rendering in an application scenario of an image correction method provided by this application, including the following steps: 1301. Scale the coordinate interval based on the offset angle information and the aspect ratio of the rendering area. 1302. Aspect ratio application and rotation center offset; The rotation center is offset to the image center.

[0138] 1303. Rotate and map each pixel; 1304. Restore the coordinate interval corresponding to each pixel point; 1305. Texture sampling based on restricted sampling range; 1306. Render texture output.

[0139] Among them, for the explanation of the shader drawing rendering process, please refer to the explanation of the corresponding content in the aforementioned method embodiment, which will not be repeated here.

[0140] The above describes an image correction method provided by an embodiment of the present application. The following describes a device for executing the above image correction method.

[0141] Figure 14 This is a schematic diagram of the structure of an image correction device provided in an embodiment of the present application. Figure 14 As shown, the image correction device 1400 includes: An acquisition module 1401 is configured to obtain original texture data, an offset angle, and rendering region parameters, wherein the offset angle is the angle at which the rendering region is offset relative to the fragment region, and the fragment region is the region range corresponding to the shader; The correction module 1402 is configured to perform rotation correction on the original texture data using a shader based on the offset angle and the rendering area parameter to obtain target texture data, where the target texture data matches the projection area of the projection light machine.

[0142] In a possible implementation, the correction module includes: a determining unit, configured to determine a coordinate transformation relationship between a fragment region and the rendering region based on the rendering region parameters; a rotation unit, configured to perform coordinate rotation on each pixel coordinate in the original texture data based on the coordinate transformation relationship and the offset angle to obtain a target coordinate set; The sampling unit is used to sample the target coordinate set to obtain target texture data matching the rendering area.

[0143] In a possible implementation, the rotating unit includes: a conversion subunit, configured to convert each pixel coordinate in the original texture data based on the coordinate conversion relationship to obtain a first coordinate set, where the first coordinate set corresponds to the fragment area; The rotation subunit is configured to perform rotation processing on each coordinate in the first coordinate set based at least on the offset angle to obtain a target coordinate set.

[0144] In a possible implementation, the rotating subunit is specifically configured to: Based on the offset angle, a rotation process is performed on each coordinate in the first coordinate set to obtain a second coordinate set, where the horizontal coordinate and the vertical coordinate of any coordinate in the second coordinate set satisfy the rendering area parameter; Based on the coordinate conversion relationship, each coordinate in the second coordinate set is restored to the first coordinate system of the shader to obtain a target coordinate set.

[0145] In a possible implementation, the sampling unit is specifically configured to: Based on the rendering area parameters, a sampling range is determined; Within the sampling range, texture sampling is performed on the texture data corresponding to the target coordinate set to obtain target texture data matching the rendering area.

[0146] In a possible implementation, the method further includes: an initialization module, configured to initialize the shader according to the offset angle and the rendering area parameter before performing rotation correction on the original texture data to obtain target texture data based on the offset angle and the rendering area parameter; The input module is used to input the original texture data into the shader to trigger the correction module.

[0147] In a possible implementation, the method further includes: A storage module, used for storing the target texture data into a texture buffer; A first binding module is used to bind material parameters to the texture buffer to obtain material information; The second binding module is used to bind the mesh parameters with the material information to obtain texture data to be output.

[0148] It should be noted that for the functional explanation of each component structure in the image correction device provided in this embodiment, please refer to the explanation in the aforementioned method embodiment, and no further details will be given here.

[0149] In this embodiment, the acquisition module is used to obtain the original texture data, the offset angle, and the rendering area parameters. The offset angle is the angle at which the rendering area is offset relative to the fragment area corresponding to the shader. The correction module is used to perform rotation correction on the original texture data using a shader based on the offset angle and the rendering area parameters to obtain target texture data that matches the projection area of the projector. For different projector installations, the parameters of the projector are obtained in advance to determine the rendering area parameters. It is only necessary to input the offset angle data of each AR HUB collector as a parameter into the shader of the OpenGL framework. The shader then implements texture rotation for the rendered image. Compared with traditional offset correction solutions, this solution is more efficient and has extremely low processing latency, making it more beneficial for AR HUD projectors that require high frame rate refresh displays.

[0150] An electronic device is also provided in an embodiment of the present application. Figure 15 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic device in the embodiments of the present application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 15 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0151] like Figure 15 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1501, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1502 or programs loaded from a storage device 1508 into a random access memory (RAM) 1503. When the electronic device is powered on, the RAM 1503 also stores various programs and data required for the operation of the electronic device. The processing device 1501, ROM 1502, and RAM 1503 are interconnected via a bus 1504. An input / output (I / O) interface 1505 is also connected to the bus 1504.

[0152] Typically, the following devices may be connected to the I / O interface 1505: an input device 1506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1508 including, for example, a memory card, a hard disk, etc.; and a communication device 1509. The communication device 1509 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Figure 15 The electronic device is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0153] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the image correction methods provided in the embodiments of the present application.

[0154] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any image correction method provided in the embodiment of the present application.

[0155] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0156] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0157] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0158] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. An image correction method, characterized in that: include: Obtain original texture data, an offset angle, and rendering area parameters, where the offset angle is the angle at which the rendering area is offset relative to the fragment area, and the fragment area is the area range corresponding to the shader; Based on the offset angle and the rendering area parameters, a shader is used to perform rotation correction on the original texture data to obtain target texture data, and the target texture data matches the projection area of the projection light machine.

2. The image correction method according to claim 1, wherein: The step of performing rotation correction on the original texture data based on the offset angle and the rendering area parameter to obtain target texture data includes: Determining a coordinate transformation relationship between a fragment region and the rendering region based on the rendering region parameters; Based on the coordinate transformation relationship and the offset angle, coordinate rotation is performed on each pixel coordinate in the original texture data to obtain a target coordinate set; The target coordinate set is sampled to obtain target texture data that matches the rendering area.

3. The image correction method according to claim 2, wherein: The step of performing coordinate rotation on each pixel coordinate in the original texture data based at least on the coordinate transformation relationship and the offset angle to obtain a target coordinate set includes: Based on the coordinate transformation relationship, transform each pixel coordinate in the original texture data to obtain a first coordinate set, where the first coordinate set corresponds to the fragment area; Based at least on the offset angle, rotation processing is performed on each coordinate in the first coordinate set to obtain a target coordinate set.

4. The image correction method according to claim 3, wherein: Based at least on the offset angle, performing rotation processing on each coordinate in the first coordinate set to obtain a target coordinate set, including: Based on the offset angle, rotation processing is performed on each coordinate in the first coordinate set to obtain a second coordinate set, where the horizontal coordinate and the vertical coordinate of any coordinate in the second coordinate set meet the rendering area parameters; Based on the coordinate conversion relationship, each coordinate in the second coordinate set is restored to the first coordinate system of the shader to obtain a target coordinate set.

5. The image correction method according to claim 2, wherein: The sampling of the target coordinate set to obtain target texture data matching the rendering area includes: Determining a sampling range based on the rendering area parameters; Within the sampling range, texture sampling is performed on the texture data corresponding to the target coordinate set to obtain target texture data matching the rendering area.

6. The image correction method according to claim 1, wherein: Before performing rotation correction on the original texture data based on the offset angle and the rendering area parameter to obtain target texture data, the method includes: Initialize the shader according to the offset angle and rendering area parameters; The original texture data is input into the shader, triggering the step of performing rotation correction on the original texture data based on the offset angle and the rendering area parameter to obtain target texture data.

7. The image correction method according to claim 1, wherein: After performing rotation correction on the original texture data based on the offset angle and the rendering area parameter to obtain target texture data, the method further includes: storing the target texture data in a texture buffer; Binding material parameters to the texture buffer to obtain material information; The mesh parameters are bound to the material information to obtain the texture data to be output.

8. An image correction device, characterized in that: include: An acquisition module is used to obtain original texture data, an offset angle, and rendering area parameters, wherein the offset angle is the angle at which the rendering area is offset relative to the fragment area, and the fragment area is the area range corresponding to the shader; A correction module is used to perform rotation correction on the original texture data using a shader based on the offset angle and the rendering area parameters to obtain target texture data, where the target texture data matches the projection area of the projection light machine.

9. A computer program product, characterized in that The method comprises computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the image correction method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: comprising at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the image correction method according to any one of claims 1 to 7.

Images