Image distortion simulation method and electronic device

Abstract

This application provides an image distortion simulation method and electronic device. The method includes: acquiring first image parameters of a source image, second image parameters of a target image, and lens parameters of a camera, wherein the target image is a distorted image of the source image after being imaged by the camera's lens; determining the spatial coordinates of each target pixel in the target image based on the second image parameters; performing reverse ray path tracing on each target pixel based on the spatial coordinates, the first image parameters, and the lens parameters to obtain the mapping relationship between each target pixel and each non-distorted pixel in the source image; assigning color values ​​to each target pixel in the target image based on the mapping relationship, and outputting the assigned target image to improve the accuracy, efficiency, and flexibility of image distortion simulation.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an image distortion simulation method and electronic device. Background Technology

[0002] Image distortion simulation is a key technology in computer vision, autonomous driving simulation, and robot perception training, and is widely used in scenarios such as fisheye lens modeling and wide-angle camera calibration.

[0003] Currently, image distortion simulation mainly relies on polynomial mathematical models. On the one hand, the parameters of polynomial mathematical models have no direct physical correspondence with the geometry of real lenses, making it impossible to simulate the impact of specific optical materials and complex lens geometries on imaging. Furthermore, the model has poor transferability; when lens parameters change due to lens design modifications, a large number of calibrated images must be reacquired and new mathematical coefficients fitted, resulting in poor flexibility. On the other hand, when dealing with large-scale generative deep learning training datasets, the computational cost of polynomial mathematical models increases linearly, making it difficult to meet real-time and throughput requirements, thus creating a computational efficiency bottleneck.

[0004] Therefore, there are certain limitations in the existing technology for simulating image distortion. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of the prior art by providing an image distortion simulation method and electronic device, thereby solving the practical problem of the limitations of image distortion simulation in the prior art.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide an image distortion simulation method, the method comprising: Acquire first image parameters of the source image, second image parameters of the target image, and lens parameters of the camera, wherein the target image is a distorted image of the source image after being imaged by the lens of the camera; Based on the second image parameters, determine the spatial coordinates of each target pixel in the target image; Based on the spatial coordinates of each target pixel, the first image parameters, and the lens parameters, reverse ray path tracing is performed on each target pixel to obtain the mapping relationship between each target pixel and each non-distorted pixel in the source image. Based on the mapping relationship, color values ​​are assigned to each target pixel in the target image, and the target image with the assigned values ​​is output.

[0007] As an optional implementation, the second image parameters include: the first horizontal resolution, the first vertical resolution, the first horizontal pixel density, the first vertical pixel density of the target image, and the first distance from the plane of the target image to the center of the lens sphere; The step of determining the spatial coordinates of each target pixel in the target image based on the second image parameters includes: Obtain the horizontal and vertical pixel coordinates of the target pixel in the target image; Based on the first horizontal resolution, the first horizontal pixel density, and the horizontal pixel coordinates of the target pixel, the X coordinate of the target pixel in the spatial coordinate system is determined, wherein the origin of the spatial coordinate system is the center of the lens. The Y coordinate of the target pixel in the spatial coordinate system is determined based on the first vertical resolution, the first vertical pixel density, and the vertical pixel coordinate of the target pixel. Based on the first distance, the Z coordinate of the target pixel in the spatial coordinate system is determined.

[0008] As an optional implementation, the step of performing reverse ray path tracing on each of the target pixels based on their spatial coordinates, the first image parameters, and the lens parameters to obtain the mapping relationship between each target pixel and each non-distorted pixel in the source image includes: Based on the spatial coordinates of the target pixel and the lens parameters, determine the first intersection point between the incident ray and the lens spherical surface, as well as the normal to the first intersection point. Based on the normal of the first intersection point and the incident ray, determine the angle of incidence at the first intersection point, and based on the angle of incidence and the lens parameters, determine the angle of exit. Based on the first image parameters, the emission angle, the normal of the first intersection point, and the spatial coordinates of the target pixel, the direction vector of the emitted ray and the second intersection point of the emitted ray and the plane where the source image is located are determined, and the source image pixel coordinates of the second intersection point are determined. The second intersection point is taken as the non-distorted pixel point corresponding to the target pixel point. A mapping relationship between the target pixel point and the non-distorted pixel point is established based on the pixel coordinates of the target pixel point and the source image pixel coordinates of the second intersection point.

[0009] As an optional implementation, determining the first intersection point between the incident light ray and the lens spherical surface based on the spatial coordinates of the target pixel and the lens parameters includes: Based on the spatial coordinates of the target pixel and the lens parameters, determine the direction vector of the incident light ray from the optical center of the camera to the target pixel; Based on the direction vector of the incident light and the lens parameters, establish the equation of the straight line at the first intersection point of the incident light from the optical center of the camera to the target pixel and the spherical surface of the lens. The spatial coordinates of the first intersection point are determined based on the linear equation of the first intersection point, the spherical equation of the lens, and the lens parameters.

[0010] As an optional implementation, determining the direction vector of the emitted ray and the second intersection point between the emitted ray and the plane containing the source image based on the first image parameters, the emission angle, the normal of the first intersection point, and the spatial coordinates of the target pixel includes: Based on the emission angle and the angle between the normal of the first intersection point and the Z-axis, the angle between the emitted ray and the Z-axis is determined, and based on the spatial coordinates of the target pixel, the azimuth angle of the emitted ray around the Z-axis is determined. Based on the angle between the emitted ray and the Z-axis and the azimuth angle of the emitted ray around the Z-axis, the direction vector of the emitted ray is determined. Based on the first image parameters, the spatial coordinates of the first intersection point, and the direction vector of the emitted ray, the spatial coordinates of the second intersection point between the emitted ray and the plane where the source image is located are determined.

[0011] As an optional implementation, the first image parameter includes at least: a second distance from the plane containing the source image to the center of the lens sphere; Determining the spatial coordinates of the second intersection point between the outgoing ray and the plane containing the source image, based on the first image parameters, the spatial coordinates of the first intersection point, and the direction vector of the outgoing ray, includes: Based on the second distance, determine the spatial coordinates of the target point on the plane where the source image is located, and determine the normal vector of the plane where the source image is located; Based on the spatial coordinates of the first intersection point and the direction vector of the outgoing ray, establish the linear equation of the second intersection point between the outgoing ray and the plane where the source image is located; The spatial coordinates of the second intersection point are determined based on the equation of the line at the second intersection point, the spatial coordinates of the first intersection point, the spatial coordinates of the target point, the normal vector of the plane containing the source image, and the direction vector of the outgoing ray.

[0012] As an optional implementation, before determining the source image pixel coordinates of the second intersection point, the following steps are included: The rectangular boundary of the source image is determined based on the first image parameters; Based on the rectangular boundary of the source image and the spatial coordinates of the second intersection point, determine whether the second intersection point is a valid pixel that does not exceed the rectangular boundary of the source image; If so, then the source image pixel coordinates of the second intersection point are determined based on the first image parameters and the spatial coordinates of the second intersection point.

[0013] As an optional implementation, the first image parameters further include: the second horizontal resolution, the second vertical resolution, the second horizontal pixel density, the second vertical pixel density of the source image, and the spatial coordinates of the center of the source image; Determining the rectangular boundary of the source image based on the first image parameters includes: Based on the second horizontal resolution, the second vertical resolution, the spatial coordinates of the center of the source image, the second horizontal pixel density, and the second vertical pixel density, determine the spatial coordinates of the upper left corner and the lower right corner of the rectangle of the source image; The rectangular boundary of the source image is determined based on the spatial coordinates of the upper left corner and the lower right corner of the rectangle.

[0014] As an optional implementation, determining the source image pixel coordinates of the second intersection point includes: Based on the spatial coordinates of the second intersection point, determine the horizontal and vertical distances of the second intersection point relative to the top left corner of the rectangle of the source image; The source image horizontal pixel coordinates of the second intersection point are determined based on the second horizontal resolution of the source image, the horizontal distance, and the second horizontal pixel density. The source image vertical pixel coordinates of the second intersection point are determined based on the second vertical resolution of the source image, the vertical distance, and the second vertical pixel density.

[0015] Secondly, embodiments of this application provide an image distortion simulation device, the device comprising: The acquisition module is used to acquire first image parameters of a source image, second image parameters of a target image, and lens parameters of a camera, wherein the target image is a distorted image of the source image after being imaged by the lens of the camera; The determining module is used to determine the spatial coordinates of each target pixel in the target image based on the second image parameters; The determining module is further configured to perform reverse ray path tracing on each of the target pixels based on the spatial coordinates of each target pixel, the first image parameters, and the lens parameters, to obtain the mapping relationship between each target pixel and each non-distorted pixel in the source image; The assignment module is used to assign color values ​​to each target pixel in the target image according to the mapping relationship, and output the assigned target image.

[0016] As an optional implementation, the determining module is specifically used for: Obtain the horizontal and vertical pixel coordinates of the target pixel in the target image; Based on the first horizontal resolution, the first horizontal pixel density, and the horizontal pixel coordinates of the target pixel, the X coordinate of the target pixel in the spatial coordinate system is determined, wherein the origin of the spatial coordinate system is the center of the lens. The Y coordinate of the target pixel in the spatial coordinate system is determined based on the first vertical resolution, the first vertical pixel density, and the vertical pixel coordinate of the target pixel. Based on the first distance, the Z coordinate of the target pixel in the spatial coordinate system is determined.

[0017] As an optional implementation, the determining module is specifically used for: Based on the spatial coordinates of the target pixel and the lens parameters, determine the first intersection point between the incident ray and the lens spherical surface, as well as the normal to the first intersection point. Based on the normal of the first intersection point and the incident ray, determine the angle of incidence at the first intersection point, and based on the angle of incidence and the lens parameters, determine the angle of exit. Based on the first image parameters, the emission angle, the normal of the first intersection point, and the spatial coordinates of the target pixel, the direction vector of the emitted ray and the second intersection point of the emitted ray and the plane where the source image is located are determined, and the source image pixel coordinates of the second intersection point are determined. The second intersection point is taken as the non-distorted pixel point corresponding to the target pixel point. A mapping relationship between the target pixel point and the non-distorted pixel point is established based on the pixel coordinates of the target pixel point and the source image pixel coordinates of the second intersection point.

[0018] As an optional implementation, the determining module is specifically used for: Based on the spatial coordinates of the target pixel and the lens parameters, determine the direction vector of the incident light ray from the optical center of the camera to the target pixel; Based on the direction vector of the incident light and the lens parameters, establish the equation of the straight line at the first intersection point of the incident light from the optical center of the camera to the target pixel and the spherical surface of the lens. The spatial coordinates of the first intersection point are determined based on the linear equation of the first intersection point, the spherical equation of the lens, and the lens parameters.

[0019] As an optional implementation, the determining module is specifically used for: Based on the emission angle and the angle between the normal of the first intersection point and the Z-axis, the angle between the emitted ray and the Z-axis is determined, and based on the spatial coordinates of the target pixel, the azimuth angle of the emitted ray around the Z-axis is determined. Based on the angle between the emitted ray and the Z-axis and the azimuth angle of the emitted ray around the Z-axis, the direction vector of the emitted ray is determined. Based on the first image parameters, the spatial coordinates of the first intersection point, and the direction vector of the emitted ray, the spatial coordinates of the second intersection point between the emitted ray and the plane where the source image is located are determined.

[0020] As an optional implementation, the determining module is specifically used for: Based on the second distance, determine the spatial coordinates of the target point on the plane where the source image is located, and determine the normal vector of the plane where the source image is located; Based on the spatial coordinates of the first intersection point and the direction vector of the outgoing ray, establish the linear equation of the second intersection point between the outgoing ray and the plane where the source image is located; The spatial coordinates of the second intersection point are determined based on the equation of the line at the second intersection point, the spatial coordinates of the first intersection point, the spatial coordinates of the target point, the normal vector of the plane containing the source image, and the direction vector of the outgoing ray.

[0021] As an optional implementation, the determining module is further configured to: The rectangular boundary of the source image is determined based on the first image parameters; Based on the rectangular boundary of the source image and the spatial coordinates of the second intersection point, determine whether the second intersection point is a valid pixel that does not exceed the rectangular boundary of the source image; If so, then the source image pixel coordinates of the second intersection point are determined based on the first image parameters and the spatial coordinates of the second intersection point.

[0022] As an optional implementation, the determining module is specifically used for: Based on the second horizontal resolution, the second vertical resolution, the spatial coordinates of the center of the source image, the second horizontal pixel density, and the second vertical pixel density, determine the spatial coordinates of the upper left corner and the lower right corner of the rectangle of the source image; The rectangular boundary of the source image is determined based on the spatial coordinates of the upper left corner and the lower right corner of the rectangle.

[0023] As an optional implementation, the determining module is specifically used for: Based on the spatial coordinates of the second intersection point, determine the horizontal and vertical distances of the second intersection point relative to the top left corner of the rectangle of the source image; The source image horizontal pixel coordinates of the second intersection point are determined based on the second horizontal resolution of the source image, the horizontal distance, and the second horizontal pixel density. The source image vertical pixel coordinates of the second intersection point are determined based on the second vertical resolution of the source image, the vertical distance, and the second vertical pixel density.

[0024] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the image distortion simulation method described in the first aspect above.

[0025] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the image distortion simulation method described in the first aspect above.

[0026] The beneficial effects of this application are: This application provides an image distortion simulation method and electronic device. It acquires first image parameters of a source image, second image parameters of a target image, and lens parameters of a camera. Based on the second image parameters, it determines the spatial coordinates of each target pixel in the target image. Using the spatial coordinates of each target pixel, the first image parameters, and the lens parameters, it performs reverse ray tracing on each target pixel to obtain the mapping relationship between each target pixel and each non-distorted pixel in the source image. Based on this mapping relationship, it assigns color values ​​to each target pixel in the target image and outputs the assigned target image. By accurately obtaining the mapping relationship between each target pixel in the target image and each non-distorted pixel in the source image through reverse ray tracing, it precisely characterizes the optical distortion transmission law of the camera lens, enabling the target image to accurately simulate the real light and shadow effect of the source image after imaging through the camera lens, thus improving the accuracy and efficiency of image distortion simulation. When the lens parameters change, the mapping relationship between each target pixel in the target image and each non-distorted pixel in the source image changes accordingly, improving the flexibility of image distortion simulation. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 1 ; Figure 2 A schematic diagram of reverse ray path tracing provided in an embodiment of this application; Figure 3 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 2 ; Figure 4 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 3 ; Figure 5 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 4 ; Figure 6 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 5 ; Figure 7 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 6 ; Figure 8 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 7 ; Figure 9 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 8 ; Figure 10 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 9 ; Figure 11 This is a module structure diagram of the image distortion simulation device provided in the embodiments of this application; Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0030] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0031] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0032] Currently, image distortion simulation mainly relies on polynomial mathematical models. However, the parameters of these models have no direct physical correspondence with the geometry of real lenses, making it impossible to simulate the impact of specific optical materials and complex lens geometries on imaging. Furthermore, the models have poor transferability; when lens parameters change due to a change in lens design, a large number of calibrated images must be reacquired and new mathematical coefficients fitted, resulting in poor flexibility. Additionally, when dealing with large-scale generative deep learning training datasets, the computational cost of polynomial mathematical models increases linearly, making it difficult to meet real-time and throughput requirements, thus creating a computational efficiency bottleneck.

[0033] Based on the above-mentioned problems, this application provides an image distortion simulation method. By tracing the reverse ray path, it accurately simulates the nonlinear offset of light after entering the lens, and obtains the mapping relationship between each target pixel in the target image and each non-distorted pixel in the source image. This makes the simulated target image more consistent with the real light and shadow imaging effect under the fisheye lens, and improves the accuracy, efficiency and flexibility of image distortion simulation in the fisheye lens simulation process.

[0034] Figure 1A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 1 The subject executing this method can be any electronic device with computing power. For example... Figure 1 As shown, the method includes: S101. Obtain the first image parameters of the source image, the second image parameters of the target image, and the lens parameters of the camera, wherein the target image is the distorted image of the source image after being imaged by the lens of the camera.

[0035] Optionally, Figure 2 This is a schematic diagram of reverse ray path tracing provided in an embodiment of this application, as shown below. Figure 2 As shown, the source image is a distortion-free reference image. The first image parameters of the source image may include the resolution, pixel density, and the plane in which the source image is located. Figure 2 The distance between the green plane in the middle and the center of the lens sphere.

[0036] The target image is a distorted image of the source image after refraction through the camera's lens. The second image parameters of the target image may include its resolution, pixel density, and the plane in which the target image is located. Figure 2 The distance between the white plane in the image and the center of the lens sphere. The camera can be a fisheye lens, and the lens parameters can include optical parameters such as the lens's spherical radius and refractive index.

[0037] in, Figure 2 In the diagram, A represents the lens focal point, which is the optical center of the camera; OB represents the distance from the plane containing the target image to the center O of the lens sphere; C represents any pixel in the target image; P is the intersection point of the red incident ray inside the lens and the lens spherical surface; and Its is the intersection point of the red outgoing ray outside the lens and the plane containing the source image. Angle of incidence ' is the emission angle, and D represents the distance from the plane containing the source image to the center O of the lens sphere. and These represent the X-axis offset and Y-axis offset from the center of the source image to the center of the lens sphere O, respectively.

[0038] S102. Determine the spatial coordinates of each target pixel in the target image based on the second image parameters.

[0039] Optionally, based on the second image parameters of the target image, a correspondence between each target pixel in the target image and optical space is established, and the two-dimensional pixel coordinates of each target pixel in the target image are converted into three-dimensional physical space coordinates in a spatial coordinate system. The origin of the spatial coordinate system is the center of the lens sphere, and the spatial coordinate system can be the camera coordinate system.

[0040] By establishing the correspondence between each target pixel in the target image and the optical space, a spatial reference is provided for reverse ray path tracing, so that the ray path can be traced in reverse starting from each target pixel in the target image.

[0041] S103. Based on the spatial coordinates of each target pixel, the first image parameters, and the lens parameters, perform reverse ray path tracing on each target pixel to obtain the mapping relationship between each target pixel and each non-distorted pixel in the source image.

[0042] Optionally, based on the spatial coordinates of each target pixel, starting from each target pixel, and combining the first image parameters of the source image and the lens parameters, reverse ray path tracing is performed on each target pixel to reverse simulate the ray propagation path, trace the non-distorted pixels in the source image corresponding to each target pixel, and establish the mapping relationship between each target pixel in the target image and each non-distorted pixel in the source image.

[0043] The mapping relationship between each target pixel in the target image and each non-distorted pixel in the source image accurately characterizes the optical distortion transmission law of the camera lens.

[0044] S104. Based on the mapping relationship, assign color values ​​to each target pixel in the target image and output the target image after assignment.

[0045] Optionally, based on the mapping relationship between each target pixel in the constructed target image and each non-distorted pixel in the source image, the color value of the corresponding non-distorted pixel in the source image is assigned to the corresponding target pixel in the target image. After completing the color assignment / filling of the target image, the assigned target image is output. The assigned target image conforms to the real light and shadow imaging effect under the fisheye lens.

[0046] It is worth noting that the non-distorted pixels in the source image corresponding to each target pixel in the target image can be traced in parallel using a Graphics Processing Unit (GPU) deployed on an electronic device, and color values ​​can be assigned to each target pixel, greatly improving the efficiency of distortion simulation of the target image. Furthermore, multiple target images (such as thousands) can be packaged into a batch, and the distortion simulation of thousands of target images can be processed in parallel on the GPU, achieving high-precision real-time distortion simulation of massive images.

[0047] In this embodiment, first image parameters of the source image, second image parameters of the target image, and lens parameters of the camera are obtained. Based on the second image parameters, the spatial coordinates of each target pixel in the target image are determined. Based on the spatial coordinates of each target pixel, the first image parameters, and the lens parameters, reverse ray tracing is performed on each target pixel to obtain the mapping relationship between each target pixel and each undistorted pixel in the source image. Based on the mapping relationship, color values ​​are assigned to each target pixel in the target image, and the assigned target image is output. By accurately obtaining the mapping relationship between each target pixel in the target image and each undistorted pixel in the source image through reverse ray tracing, the optical distortion transmission law of the camera lens is accurately characterized, enabling the target image to accurately simulate the real light and shadow effect after the source image is imaged through the camera lens, improving the accuracy and efficiency of image distortion simulation. When the lens parameters change, the mapping relationship between each target pixel in the target image and each undistorted pixel in the source image changes accordingly, improving the flexibility of image distortion simulation.

[0048] As an optional implementation, the second image parameters include: the first horizontal resolution, the first vertical resolution, the first horizontal pixel density, the first vertical pixel density of the target image, and the first distance from the plane of the target image to the center of the lens sphere.

[0049] Optionally, the second image parameter of the target image includes the first horizontal resolution of the target image. First vertical resolution First horizontal pixel density First vertical pixel density and the first distance from the plane containing the target image to the center of the lens sphere .

[0050] Among them, the first horizontal resolution The first vertical resolution represents the number of pixels contained in the target image in the horizontal direction. This indicates the number of pixels contained in the target image in the vertical direction. First horizontal pixel density. The first vertical pixel density represents the number of pixels per unit length in the horizontal direction of the target image. This indicates the number of pixels contained in each unit length of the target image in the vertical direction.

[0051] Figure 3 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 2 ,like Figure 3 As shown, in step S102 above, determining the spatial coordinates of each target pixel in the target image based on the second image parameters includes: S201. Obtain the horizontal and vertical pixel coordinates of the target pixel in the target image.

[0052] Optionally, continue to refer to Figure 2 Taking a target pixel C in the target image as an example, obtain the horizontal pixel coordinate u and the vertical pixel coordinate v of the target pixel C.

[0053] In other words, obtain the horizontal and vertical pixel coordinates of the target pixel in the target image. .

[0054] S202. Based on the first horizontal resolution, the first horizontal pixel density, and the horizontal pixel coordinates of the target pixel, determine the X coordinate of the target pixel in the spatial coordinate system, where the origin of the spatial coordinate system is the center of the lens.

[0055] Optionally, based on the first horizontal resolution of the target image First horizontal pixel density Given the horizontal pixel coordinate u of the target pixel C, determine the X-coordinate of the target pixel C in a spatial coordinate system with the lens sphere as the origin based on the following formula. :

[0056] in, The first horizontal resolution of the target image. The first horizontal pixel density of the target image. Let C be the horizontal pixel coordinate of the target pixel. Let X be the X coordinate of the target pixel C in the spatial coordinate system.

[0057] S203. Determine the Y coordinate of the target pixel in the spatial coordinate system based on the first vertical resolution, the first vertical pixel density, and the vertical pixel coordinate of the target pixel.

[0058] Optionally, based on the first vertical resolution of the target image First vertical pixel density Given the vertical pixel coordinate v of the target pixel C, determine the Y coordinate of the target pixel C in a spatial coordinate system with the lens sphere as the origin based on the following formula. :

[0059] in, The first vertical resolution of the target image. The first vertical pixel density of the target image. Let C be the vertical pixel coordinate of the target pixel. Let C be the Y-coordinate of the target pixel C in the spatial coordinate system.

[0060] S204. Based on the first distance, determine the Z coordinate of the target pixel in the spatial coordinate system.

[0061] Optionally, refer to Figure 2 Since the distance from the plane containing the target image to the center of the lens sphere is The distance from each target pixel in the target image to the center of the lens sphere is... Then the Z coordinate of the target pixel C in the spatial coordinate system for .

[0062] In this embodiment, the horizontal and vertical pixel coordinates of the target pixel in the target image are obtained. Based on the first horizontal resolution, the first horizontal pixel density, and the horizontal pixel coordinates of the target pixel, the X-coordinate of the target pixel in a spatial coordinate system with the center of the lens as the origin is determined. Based on the first vertical resolution, the first vertical pixel density, and the vertical pixel coordinates of the target pixel, the Y-coordinate of the target pixel in the spatial coordinate system is determined. Based on the first distance, the Z-coordinate of the target pixel in the spatial coordinate system is determined. The target pixel in the target image is converted into a three-dimensional spatial point in a spatial coordinate system with the center of the lens as the origin, in order to perform reverse ray path tracing.

[0063] Figure 4 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 3 ,like Figure 4 As shown, in step S103 above, based on the spatial coordinates of each target pixel, the first image parameters, and the lens parameters, reverse ray path tracing is performed on each target pixel to obtain the mapping relationship between each target pixel and each non-distorted pixel in the source image, including: S301. Based on the spatial coordinates of the target pixel and the lens parameters, determine the first intersection point of the incident ray with the lens sphere and the normal of the first intersection point.

[0064] Optionally, the lens parameters may include the spatial coordinates of the lens focal point A. spherical radius of the lens and lens refractive index Wherein, lens focal point A is the optical center of the camera, and lens focal point A is located on the Z-axis, as referenced. Figure 2 , This is the distance from the lens focal point A to the lens center O. The lens refractive index. This indicates the lens's ability to refract light.

[0065] Based on the spatial coordinates of the target pixel C and the spatial coordinates of the lens focal point A, the optical path from the camera's optical center (i.e., lens focal point A) to the target pixel C is established, forming the incident light ray. Among them, incident light The direction vector is .

[0066] Based on the spatial coordinates of the lens focal point A and the incident ray... Direction vector and the spherical radius of the lens Determine the incident light The first intersection point is the point of intersection with the spherical surface of the lens. The spatial coordinates, and based on the first intersection point Determine the first intersection point using the spatial coordinates. The normals, where the first intersection point The normal vector of the normal line is , First intersection point For incident light The position where the light hits the lens.

[0067] S302. Determine the angle of incidence at the first intersection point based on the normal and the incident ray, and determine the angle of exit based on the angle of incidence and the lens parameters.

[0068] Optionally, based on the first intersection point The normal vector of the normal line and incident light Direction vector The first intersection point is determined based on the following formula. Angle of incidence at point :

[0069]

[0070] in, For incident light directional vector, The first intersection point The normal vector of the normal line, The first intersection point The angle of incidence at point , i.e., the incident ray Intersection with the first point The angle between the normals.

[0071] According to the angle of incidence and lens refractive index Using Snell's law of refraction, the exit angle is determined based on the following formula. :

[0072]

[0073] in, Angle of incidence The angle of departure. is the refractive index of the lens.

[0074] S303. Based on the first image parameters, the emission angle, the normal of the first intersection point, and the spatial coordinates of the target pixel, determine the direction vector of the emitted ray and the second intersection point between the emitted ray and the plane where the source image is located, and determine the source image pixel coordinates of the second intersection point.

[0075] Optionally, based on the first image parameters of the source image and the emission angle... The normal to the first intersection point and the spatial coordinates of the target pixel C are used to determine the outgoing ray. Direction vector .

[0076] According to the emitted light Direction vector First image parameters and first intersection point Spatial coordinates to determine the outgoing ray The second intersection point with the plane containing the source image Spatial coordinates.

[0077] Based on the first image parameters, the second intersection point The spatial coordinates are converted to the source image pixel coordinates of the second intersection point, the second intersection point. This refers to the non-distorted pixel point corresponding to target pixel point C after reverse ray path tracing.

[0078] S304. Take the second intersection point as the non-distorted pixel point corresponding to the target pixel point, and establish the mapping relationship between the target pixel point and the non-distorted pixel point based on the pixel coordinates of the target pixel point and the source image pixel coordinates of the second intersection point.

[0079] Optionally, due to the second intersection point For the undistorted pixel point corresponding to target pixel point C after reverse ray path tracing, based on the horizontal and vertical pixel coordinates of target pixel point C and the second intersection point... The source image pixel coordinates are used to establish the intersection point between the target pixel C and the non-distorted pixel (i.e., the second intersection point). The mapping relationship between ).

[0080] In this embodiment, based on the spatial coordinates of the target pixel and the lens parameters, the first intersection point of the incident ray and the lens spherical surface, as well as the normal of the first intersection point, are determined. Based on the normal of the first intersection point and the incident ray, the incident angle at the first intersection point is determined, and based on the incident angle and the lens parameters, the exit angle is determined. Based on the first image parameters, the exit angle, the normal of the first intersection point, and the spatial coordinates of the target pixel, the direction vector of the exit ray and the second intersection point of the exit ray with the plane containing the source image are determined, and the source image pixel coordinates of the second intersection point are determined. The second intersection point is taken as the non-distorted pixel corresponding to the target pixel. Based on the pixel coordinates of the target pixel and the source image pixel coordinates of the second intersection point, a mapping relationship between the target pixel and the non-distorted pixel is established. This achieves reverse ray path tracing starting from the target pixel, accurately establishing the mapping relationship between the target pixel in the target image and the non-distorted pixel in the source image.

[0081] Figure 5 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 4 ,like Figure 5 As shown, in step S301 above, determining the first intersection point between the incident ray and the lens spherical surface based on the spatial coordinates of the target pixel and the lens parameters includes: S401. Based on the spatial coordinates of the target pixel and the lens parameters, determine the direction vector of the incident light ray from the optical center of the camera to the target pixel.

[0082] Optionally, the incident light ray from the camera's optical center to the target pixel is determined based on the spatial coordinates of the target pixel C and the spatial coordinates of the lens focal point A. Direction vector ,in, .

[0083] Among them, incident light Direction vector Used to characterize the incident light rays pointing from the optical center of the camera to the target pixel C. The direction of its spread.

[0084] S402. Based on the direction vector of the incident ray and the lens parameters, establish the equation of the straight line at the first intersection point of the incident ray from the optical center of the camera to the target pixel and the lens spherical surface.

[0085] Alternatively, depending on the incident light Direction vector and the spatial coordinates of the lens focal point A, starting from the lens focal point A and using the incident ray. Direction vector Establish the first intersection point to determine the direction of dissemination. The equation of the line that satisfies .in, .

[0086] In other words, the first intersection point satisfy This is the equation of the straight line.

[0087] S403. Determine the spatial coordinates of the first intersection point based on the equation of the straight line at the first intersection point, the equation of the spherical surface of the lens, and the lens parameters.

[0088] Optionally, the first intersection point It also satisfies the spherical equation of the lens. ,in, The center of the lens sphere is the origin of the spatial coordinate system. Let the radius of the lens spherical surface be [radius value]. Let the first intersection point [be the point of intersection]. The equation of the line that satisfies Substituting into the spherical equation of the lens, we obtain the following about The quadratic equation of .

[0089]

[0090] in, For incident light directional vector, The spatial coordinates of the lens focal point. Let be the radius of the lens sphere.

[0091] use Solve Take the first intersection point The Z-coordinate is greater than or equal to 0. Substitute into the equation of the straight line The first intersection point was obtained. Spatial coordinates.

[0092] In this embodiment, based on the spatial coordinates of the target pixel and the lens parameters, the direction vector of the incident light ray from the optical center of the camera to the target pixel is determined. Based on the direction vector of the incident light ray and the lens parameters, the equation of the straight line at the first intersection point between the incident light ray from the optical center of the camera to the target pixel and the first intersection point of the lens spherical surface is established. Based on the equation of the straight line at the first intersection point, the equation of the lens spherical surface, and the lens parameters, the spatial coordinates of the first intersection point are determined. Improving the accuracy of the spatial coordinates of the first intersection point, thereby improving the accuracy of reverse ray path tracking.

[0093] Figure 6 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 5 ,like Figure 6As shown, in step S303 above, determining the direction vector of the outgoing ray and the second intersection point between the outgoing ray and the plane containing the source image based on the first image parameters, the emission angle, the normal of the first intersection point, and the spatial coordinates of the target pixel includes: S501. Based on the emission angle and the angle between the normal of the first intersection point and the Z-axis, determine the angle between the emitted ray and the Z-axis, and based on the spatial coordinates of the target pixel, determine the azimuth angle of the emitted ray around the Z-axis.

[0094] Optionally, determine the first intersection point. The angle between the normal and the Z-axis The launch angle and included angle The sum of the emitted rays Angle with the Z-axis ,Right now .

[0095] Based on the X coordinate of the target pixel C and Y coordinate The outgoing ray is determined by the two-parameter arctangent function. Azimuth angle around the Z-axis ,in, .

[0096] S502. Based on the angle between the outgoing ray and the Z-axis and the azimuth angle of the outgoing ray around the Z-axis, determine the direction vector of the outgoing ray. Based on the first image parameters, the spatial coordinates of the first intersection point, and the direction vector of the outgoing ray, determine the spatial coordinates of the second intersection point between the outgoing ray and the plane where the source image is located.

[0097] Optionally, based on the emitted light rays Angle with the Z-axis and emitted light Azimuth angle around the Z-axis The emitted ray is determined based on the following formula. Direction vector :

[0098] in, For the emitted light Azimuth angle around the Z-axis, For the emitted light The angle with the Z-axis, For the emitted light The direction vector is used to characterize the incident ray. The direction of propagation after refraction through the lens.

[0099] According to the emitted light Direction vector First image parameters and first intersection point Spatial coordinates to determine the outgoing ray The second intersection point with the plane containing the source image The spatial coordinates of the second intersection point are given. This refers to the non-distorted pixel point corresponding to target pixel point C after reverse ray path tracing.

[0100] In this embodiment, the angle between the outgoing ray and the Z-axis is determined based on the emission angle and the angle between the normal of the first intersection point and the Z-axis. The azimuth angle of the outgoing ray around the Z-axis is determined based on the spatial coordinates of the target pixel. The direction vector of the outgoing ray is determined based on the angle between the outgoing ray and the Z-axis and the azimuth angle of the outgoing ray around the Z-axis. Finally, the spatial coordinates of the second intersection point between the outgoing ray and the plane containing the source image are determined based on the first image parameters, the spatial coordinates of the first intersection point, and the direction vector of the outgoing ray. This improves the accuracy of the spatial coordinates of the second intersection point.

[0101] As an optional implementation, the first image parameter includes at least the second distance from the plane containing the source image to the center of the lens sphere.

[0102] Optionally, continue to refer to Figure 2 The first image parameter of the source image includes at least the second distance from the plane containing the source image to the center of the lens sphere. .

[0103] Figure 7 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 6 ,like Figure 7 As shown, in step S502 above, determining the spatial coordinates of the second intersection point between the outgoing ray and the plane containing the source image based on the first image parameters, the spatial coordinates of the first intersection point, and the direction vector of the outgoing ray includes: S601. Based on the second distance, determine the spatial coordinates of the target point on the plane where the source image is located, and determine the normal vector of the plane where the source image is located.

[0104] Optionally, the plane of the plane containing the source image passes through the point As the target point, since the second distance from the plane containing the source image to the center of the lens sphere is... Then the target point Spatial coordinates can be The normal vector of the plane containing the source image for .

[0105] S602. Based on the spatial coordinates of the first intersection point and the direction vector of the outgoing ray, establish the equation of the straight line at the second intersection point between the outgoing ray and the plane where the source image is located.

[0106] Optionally, using the first intersection point Starting from the outgoing ray, and with the outgoing ray Direction vector Establish a second intersection point to determine the direction of propagation. Satisfactory straight line direction .in, .

[0107] In other words, the second intersection point satisfy This is the equation of the straight line.

[0108] S603. Determine the spatial coordinates of the second intersection point based on the equation of the line at the second intersection point, the spatial coordinates of the first intersection point, the spatial coordinates of the target point, the normal vector of the plane containing the source image, and the direction vector of the outgoing ray.

[0109] Optionally, when the emitted light When falling on the plane of the source image, from the target point Pointing out beams The vector and the normal vector of the plane containing the source image Perpendicular, satisfying the plane point normal form equation .

[0110] Equation of the straight line Substituting the equation into the plane point normal form and rearranging, we get:

[0111] in, The spatial coordinates of the target point, The spatial coordinates of the first intersection point, For the emitted light directional vector, is the normal vector of the plane containing the source image.

[0112] Will Substitution This equation of the line yields the second intersection point. Spatial coordinates.

[0113] In this embodiment, the spatial coordinates of the target point on the plane containing the source image are determined based on the second distance, and the normal vector of the plane containing the source image is also determined. Based on the spatial coordinates of the first intersection point and the direction vector of the outgoing ray, the equation of the straight line at the second intersection point between the outgoing ray and the plane containing the source image is established. Then, based on the equation of the straight line at the second intersection point, the spatial coordinates of the first intersection point, the spatial coordinates of the target point, the normal vector of the plane containing the source image, and the direction vector of the outgoing ray, the spatial coordinates of the second intersection point are determined. This improves the accuracy of the spatial coordinates of the second intersection point, thereby improving the accuracy of the reverse ray path tracking.

[0114] Figure 8 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 7 ,like Figure 8 As shown, before determining the source image pixel coordinates of the second intersection point in step S304 above, the following steps are included: S701. Determine the rectangular boundary of the source image based on the first image parameters.

[0115] Optionally, the rectangular boundary of the source image is determined based on the first image parameters of the source image, wherein the rectangular boundary of the source image includes the boundary of the maximum x-value. Minimum x-value boundary Maximum y-value boundary and the boundary of minimum y value .

[0116] S702. Based on the rectangular boundary of the source image and the spatial coordinates of the second intersection point, determine whether the second intersection point is a valid pixel that does not exceed the rectangular boundary of the source image.

[0117] Optionally, based on the second intersection point X coordinate in spatial coordinates and Y coordinate And the boundary of the maximum x-value within the rectangular boundary of the source image. Minimum x-value boundary Maximum y-value boundary and the boundary of minimum y value Determine the second intersection point Whether it is a valid pixel that does not exceed the rectangular boundary of the source image.

[0118] Specifically, if and Then the second intersection point is determined. Valid pixels that do not exceed the rectangular boundary of the source image; otherwise, determine the second intersection point. Invalid pixels that exceed the rectangular boundary of the source image.

[0119] S703. If so, then determine the source image pixel coordinates of the second intersection point based on the first image parameters and the spatial coordinates of the second intersection point.

[0120] Optionally, if the second intersection point The second intersection point is defined as the valid pixels that do not exceed the rectangular boundary of the source image. If the pixel is in the source image, then based on the first image parameters of the source image, the second intersection point is... The spatial coordinates are converted to the pixel coordinates of the source image.

[0121] Accordingly, if the second intersection point Invalid pixels that exceed the rectangular boundary of the source image represent the second intersection point. If the pixel is not in the source image, then the second intersection point will be... The color value is set to black to avoid assigning the wrong color to the target pixel C in the target image.

[0122] In this embodiment, the rectangular boundary of the source image is determined based on the first image parameters. Then, based on the rectangular boundary of the source image and the spatial coordinates of the second intersection point, it is determined whether the second intersection point is a valid pixel not exceeding the rectangular boundary of the source image. If so, the source image pixel coordinates of the second intersection point are determined based on the first image parameters and the spatial coordinates of the second intersection point. This accurately verifies the validity of the second intersection point that has a mapping relationship with the target pixel.

[0123] As an optional implementation, the first image parameters also include: the second horizontal resolution, the second vertical resolution, the second horizontal pixel density, the second vertical pixel density of the source image, and the spatial coordinates of the center of the source image.

[0124] Optionally, the first image parameter of the source image includes the second horizontal resolution of the source image. Second vertical resolution Second horizontal pixel density Second vertical pixel density and the spatial coordinates of the center of the source image .

[0125] Among them, the second horizontal resolution The second vertical resolution represents the number of pixels contained in the source image in the horizontal direction. This indicates the number of pixels contained in the source image in the vertical direction. Second horizontal pixel density. The second horizontal pixel density represents the number of pixels contained per unit length in the horizontal direction of the source image. Equal to the first horizontal pixel density Second vertical pixel density The second vertical pixel density represents the number of pixels contained per unit length in the vertical direction of the source image. Equal to the first vertical pixel density .

[0126] Figure 9 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 8 ,like Figure 9 As shown, in step S701 above, determining the rectangular boundary of the source image based on the first image parameters includes: S801. Based on the second horizontal resolution, the second vertical resolution, the spatial coordinates of the center of the source image, the second horizontal pixel density, and the second vertical pixel density, determine the spatial coordinates of the upper left corner and the lower right corner of the rectangle of the source image.

[0127] Alternatively, based on the second horizontal resolution of the source image Second vertical resolution Spatial coordinates of the center of the source image Second horizontal pixel density and the second vertical pixel density The top left corner of the rectangle of the source image is determined based on the following formula. Spatial coordinates and lower right corner Spatial coordinates:

[0128]

[0129] in, and These are the X and Y coordinates of the center of the source image in the spatial coordinate system, respectively. The first horizontal resolution, This is the second vertical resolution. The second horizontal pixel density, The second vertical pixel density, and These are the top left and bottom right corners of the rectangle in the source image, respectively. This is the second distance.

[0130] S802. Determine the rectangular boundary of the source image based on the spatial coordinates of the upper left corner and the lower right corner of the rectangle.

[0131] Optionally, based on the top left corner of the rectangle of the source image. Spatial coordinates and lower right corner The spatial coordinates are used to determine the boundary of the maximum x-value within the rectangular boundary of the source image based on the following formula. Minimum x-value boundary Maximum y-value boundary and the boundary of minimum y value :

[0132]

[0133]

[0134]

[0135] in, , These are the X and Y coordinates of the top-left corner of the rectangle in the source image, respectively. and These are the X and Y coordinates of the lower right corner of the rectangle in the source image, respectively. , , and These represent the maximum x-value boundary, minimum x-value boundary, maximum y-value boundary, and minimum y-value boundary within the rectangular boundary of the source image.

[0136] In this embodiment, based on the second horizontal resolution, the second vertical resolution, the spatial coordinates of the center of the source image, the second horizontal pixel density, and the second vertical pixel density, the spatial coordinates of the upper left corner and the lower right corner of the rectangle of the source image are determined. Then, based on the spatial coordinates of the upper left corner and the lower right corner of the rectangle of the source image, the rectangular boundary of the source image is determined. This provides a precise rectangular boundary for verifying the pixel validity of the second intersection point.

[0137] Figure 10 A flowchart illustrating the image distortion simulation method provided in this application embodiment. Figure 9 ,like Figure 10 As shown, determining the source image pixel coordinates of the second intersection point in step S304 above includes: S901. Based on the spatial coordinates of the second intersection point, determine the horizontal and vertical distances of the second intersection point relative to the upper left corner of the rectangle of the source image.

[0138] Optionally, based on the second intersection point The spatial coordinates are used to determine the second intersection point based on the following formulas. Relative to the top left corner of the rectangle of the source image lateral distance and longitudinal distance :

[0139]

[0140] in, and The second intersection point Relative to the top left corner of the rectangle of the source image The horizontal and vertical distances and The second intersection point The X and Y coordinates in the spatial coordinate system.

[0141] S902. Determine the horizontal pixel coordinates of the source image for the second intersection point based on the second horizontal resolution, horizontal distance, and second horizontal pixel density of the source image.

[0142] Alternatively, based on the second horizontal resolution of the source image Second intersection point Relative to the top left corner of the rectangle of the source image lateral distance and second horizontal pixel density The second intersection point is determined based on the following formula. Horizontal pixel coordinates of the source image :

[0143] in, The second intersection point The horizontal pixel coordinates of the source image, This is the second horizontal resolution. The second horizontal pixel density, The second intersection point Relative to the top left corner of the rectangle of the source image lateral distance, This indicates rounding down to the nearest integer to prevent a second intersection. Horizontal pixel coordinates of the source image Crossing the boundary.

[0144] S903. Determine the source image vertical pixel coordinates of the second intersection point based on the second vertical resolution, vertical distance, and second vertical pixel density of the source image.

[0145] Optionally, based on the second vertical resolution of the source image Second intersection point Relative to the top left corner of the rectangle of the source image longitudinal distance and the second vertical pixel density The second intersection point is determined based on the following formula. Vertical pixel coordinates of the source image :

[0146] in, The second intersection point The vertical pixel coordinates of the source image, This is the second vertical resolution. The second vertical pixel density, The second intersection point Relative to the top left corner of the rectangle of the source image The longitudinal distance, This indicates rounding down to the nearest integer to prevent a second intersection. Vertical pixel coordinates of the source image Crossing the boundary.

[0147] In this embodiment, based on the spatial coordinates of the second intersection point, the horizontal and vertical distances of the second intersection point relative to the top-left corner of the rectangle of the source image are determined. Based on the second horizontal resolution, horizontal distance, and second horizontal pixel density of the source image, the horizontal pixel coordinates of the second intersection point in the source image are determined. Finally, based on the second vertical resolution, vertical distance, and second vertical pixel density of the source image, the vertical pixel coordinates of the second intersection point in the source image are determined. This improves the accuracy of the source image pixel coordinates of the second intersection point.

[0148] It is worth noting that, in the target image When assigning a color value to target pixel C, the color of target pixel C is set to the color of the source image. Second intersection The color, that is .

[0149] Based on the same inventive concept, this application also provides an image distortion simulation device corresponding to the image distortion simulation method. Since the principle of the device in this application is similar to the image distortion simulation method described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0150] Figure 11 This is a module structure diagram of the image distortion simulation device provided in the embodiments of this application, such as... Figure 11 As shown, the device includes: The acquisition module 1001 is used to acquire the first image parameters of the source image, the second image parameters of the target image, and the lens parameters of the camera, wherein the target image is the distorted image of the source image after being imaged by the lens of the camera; The determining module 1002 is used to determine the spatial coordinates of each target pixel in the target image based on the second image parameters; The determination module 1002 is also used to perform reverse ray path tracing on each target pixel according to the spatial coordinates of each target pixel, the first image parameters and the lens parameters, to obtain the mapping relationship between each target pixel and each non-distorted pixel in the source image. The assignment module 1003 is used to assign color values ​​to each target pixel in the target image according to the mapping relationship, and output the assigned target image.

[0151] As an optional implementation, the determining module 1002 is specifically used for: Obtain the horizontal and vertical pixel coordinates of the target pixel in the target image; Based on the first horizontal resolution, the first horizontal pixel density, and the horizontal pixel coordinates of the target pixel, the X coordinate of the target pixel in the spatial coordinate system is determined, where the origin of the spatial coordinate system is the center of the lens sphere. Based on the first vertical resolution, the first vertical pixel density, and the vertical pixel coordinates of the target pixel, determine the Y coordinate of the target pixel in the spatial coordinate system; Based on the first distance, determine the Z coordinate of the target pixel in the spatial coordinate system.

[0152] As an optional implementation, the determining module 1002 is specifically used for: Based on the spatial coordinates of the target pixel and the lens parameters, determine the first intersection point between the incident ray and the lens spherical surface, as well as the normal to the first intersection point; Based on the normal at the first intersection point and the incident ray, determine the angle of incidence at the first intersection point, and based on the angle of incidence and the lens parameters, determine the angle of exit. Based on the first image parameters, the emission angle, the normal of the first intersection point, and the spatial coordinates of the target pixel, determine the direction vector of the emitted ray and the second intersection point between the emitted ray and the plane where the source image is located, and determine the source image pixel coordinates of the second intersection point. The second intersection point is taken as the non-distorted pixel point corresponding to the target pixel point. Based on the pixel coordinates of the target pixel point and the source image pixel coordinates of the second intersection point, a mapping relationship between the target pixel point and the non-distorted pixel point is established.

[0153] As an optional implementation, the determining module 1002 is specifically used for: Based on the spatial coordinates of the target pixel and the lens parameters, determine the direction vector of the incident light ray from the optical center of the camera to the target pixel; Based on the direction vector of the incident ray and the lens parameters, establish the equation of the straight line at the first intersection point of the incident ray from the optical center of the camera to the target pixel and the spherical surface of the lens. Determine the spatial coordinates of the first intersection point based on the equation of the line at the first intersection point, the equation of the spherical surface of the lens, and the lens parameters.

[0154] As an optional implementation, the determining module 1002 is specifically used for: Based on the emission angle and the angle between the normal of the first intersection point and the Z-axis, determine the angle between the emitted ray and the Z-axis, and based on the spatial coordinates of the target pixel, determine the azimuth angle of the emitted ray around the Z-axis. Based on the angle between the outgoing ray and the Z-axis and the azimuth angle of the outgoing ray around the Z-axis, the direction vector of the outgoing ray is determined. Based on the first image parameters, the spatial coordinates of the first intersection point, and the direction vector of the outgoing ray, the spatial coordinates of the second intersection point between the outgoing ray and the plane where the source image is located are determined.

[0155] As an optional implementation, the determining module 1002 is specifically used for: Based on the second distance, determine the spatial coordinates of the target point on the plane where the source image is located, and determine the normal vector of the plane where the source image is located; Based on the spatial coordinates of the first intersection point and the direction vector of the outgoing ray, establish the equation of the straight line at the second intersection point between the outgoing ray and the plane containing the source image. The spatial coordinates of the second intersection point are determined based on the equation of the line at the second intersection point, the spatial coordinates of the first intersection point, the spatial coordinates of the target point, the normal vector of the plane containing the source image, and the direction vector of the outgoing ray.

[0156] As an optional implementation, the determining module 1002 is further configured to: Determine the rectangular boundary of the source image based on the first image parameters; Based on the rectangular boundary of the source image and the spatial coordinates of the second intersection point, determine whether the second intersection point is a valid pixel that does not exceed the rectangular boundary of the source image; If so, then the source image pixel coordinates of the second intersection point are determined based on the first image parameters and the spatial coordinates of the second intersection point.

[0157] As an optional implementation, the determining module 1002 is specifically used for: Based on the second horizontal resolution, the second vertical resolution, the spatial coordinates of the center of the source image, the second horizontal pixel density, and the second vertical pixel density, determine the spatial coordinates of the upper left corner and the lower right corner of the rectangle of the source image. The rectangular boundary of the source image is determined based on the spatial coordinates of the upper left corner and the lower right corner of the rectangle.

[0158] As an optional implementation, the determining module 1002 is specifically used for: Based on the spatial coordinates of the second intersection point, determine the horizontal and vertical distances of the second intersection point relative to the top left corner of the rectangle of the source image; The horizontal pixel coordinates of the source image at the second intersection point are determined based on the second horizontal resolution, horizontal distance, and second horizontal pixel density of the source image. The vertical pixel coordinates of the source image at the second intersection point are determined based on the second vertical resolution, vertical distance, and second vertical pixel density of the source image.

[0159] This application also provides an electronic device, such as... Figure 12The diagram shown is a schematic representation of the structure of an electronic device provided in an embodiment of this application, including a processor 111, a memory 112, and a bus 113. The memory 112 stores machine-readable instructions executable by the processor 111. When the electronic device is running, the processor 111 communicates with the memory 112 via the bus 113, and the processor 111 executes the machine-readable instructions to perform the steps of the image distortion simulation method in the aforementioned embodiment.

[0160] This application also provides a computer-readable storage medium storing a computer program, which, when run by a processor, performs the steps of the image distortion simulation method described in the foregoing embodiments.

[0161] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0162] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0163] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An image distortion simulation method, characterized in that, include: Acquire first image parameters of the source image, second image parameters of the target image, and lens parameters of the camera, wherein the target image is a distorted image of the source image after being imaged by the lens of the camera; Based on the second image parameters, determine the spatial coordinates of each target pixel in the target image; Based on the spatial coordinates of each target pixel, the first image parameters, and the lens parameters, reverse ray path tracing is performed on each target pixel to obtain the mapping relationship between each target pixel and each non-distorted pixel in the source image. Based on the mapping relationship, color values ​​are assigned to each target pixel in the target image, and the target image with the assigned values ​​is output.

2. The method according to claim 1, characterized in that, The second image parameters include: the first horizontal resolution, the first vertical resolution, the first horizontal pixel density, the first vertical pixel density of the target image, and the first distance from the plane of the target image to the center of the lens sphere; The step of determining the spatial coordinates of each target pixel in the target image based on the second image parameters includes: Obtain the horizontal and vertical pixel coordinates of the target pixel in the target image; Based on the first horizontal resolution, the first horizontal pixel density, and the horizontal pixel coordinates of the target pixel, the X coordinate of the target pixel in the spatial coordinate system is determined, wherein the origin of the spatial coordinate system is the center of the lens. The Y coordinate of the target pixel in the spatial coordinate system is determined based on the first vertical resolution, the first vertical pixel density, and the vertical pixel coordinate of the target pixel. Based on the first distance, the Z coordinate of the target pixel in the spatial coordinate system is determined.

3. The method according to claim 1, characterized in that, The step of performing reverse ray path tracing on each of the target pixels based on their spatial coordinates, the first image parameters, and the lens parameters to obtain the mapping relationship between each target pixel and each non-distorted pixel in the source image includes: Based on the spatial coordinates of the target pixel and the lens parameters, determine the first intersection point between the incident ray and the lens spherical surface, as well as the normal to the first intersection point. Based on the normal of the first intersection point and the incident ray, determine the angle of incidence at the first intersection point, and based on the angle of incidence and the lens parameters, determine the angle of exit. Based on the first image parameters, the emission angle, the normal of the first intersection point, and the spatial coordinates of the target pixel, the direction vector of the emitted ray and the second intersection point of the emitted ray and the plane where the source image is located are determined, and the source image pixel coordinates of the second intersection point are determined. The second intersection point is taken as the non-distorted pixel point corresponding to the target pixel point. A mapping relationship between the target pixel point and the non-distorted pixel point is established based on the pixel coordinates of the target pixel point and the source image pixel coordinates of the second intersection point.

4. The method according to claim 3, characterized in that, Based on the spatial coordinates of the target pixel and the lens parameters, the first intersection point between the incident ray and the lens spherical surface is determined, including: Based on the spatial coordinates of the target pixel and the lens parameters, determine the direction vector of the incident light ray from the optical center of the camera to the target pixel; Based on the direction vector of the incident light and the lens parameters, establish the equation of the straight line at the first intersection point of the incident light from the optical center of the camera to the target pixel and the spherical surface of the lens. The spatial coordinates of the first intersection point are determined based on the linear equation of the first intersection point, the spherical equation of the lens, and the lens parameters.

5. The method according to claim 3, characterized in that, The step of determining the direction vector of the emitted ray and the second intersection point between the emitted ray and the plane containing the source image based on the first image parameters, the emission angle, the normal of the first intersection point, and the spatial coordinates of the target pixel includes: Based on the emission angle and the angle between the normal of the first intersection point and the Z-axis, the angle between the emitted ray and the Z-axis is determined, and based on the spatial coordinates of the target pixel, the azimuth angle of the emitted ray around the Z-axis is determined. Based on the angle between the emitted ray and the Z-axis and the azimuth angle of the emitted ray around the Z-axis, the direction vector of the emitted ray is determined. Based on the first image parameters, the spatial coordinates of the first intersection point, and the direction vector of the emitted ray, the spatial coordinates of the second intersection point between the emitted ray and the plane where the source image is located are determined.

6. The method according to claim 5, characterized in that, The first image parameter includes at least: a second distance from the plane containing the source image to the center of the lens sphere; Determining the spatial coordinates of the second intersection point between the outgoing ray and the plane containing the source image, based on the first image parameters, the spatial coordinates of the first intersection point, and the direction vector of the outgoing ray, includes: Based on the second distance, determine the spatial coordinates of the target point on the plane where the source image is located, and determine the normal vector of the plane where the source image is located; Based on the spatial coordinates of the first intersection point and the direction vector of the outgoing ray, establish the linear equation of the second intersection point between the outgoing ray and the plane where the source image is located; The spatial coordinates of the second intersection point are determined based on the equation of the line at the second intersection point, the spatial coordinates of the first intersection point, the spatial coordinates of the target point, the normal vector of the plane containing the source image, and the direction vector of the outgoing ray.

7. The method according to claim 3, characterized in that, Before determining the source image pixel coordinates of the second intersection point, the following steps are included: The rectangular boundary of the source image is determined based on the first image parameters; Based on the rectangular boundary of the source image and the spatial coordinates of the second intersection point, determine whether the second intersection point is a valid pixel that does not exceed the rectangular boundary of the source image; If so, then the source image pixel coordinates of the second intersection point are determined based on the first image parameters and the spatial coordinates of the second intersection point.

8. The method according to claim 7, characterized in that, The first image parameters also include: the second horizontal resolution, the second vertical resolution, the second horizontal pixel density, the second vertical pixel density of the source image, and the spatial coordinates of the center of the source image; Determining the rectangular boundary of the source image based on the first image parameters includes: Based on the second horizontal resolution, the second vertical resolution, the spatial coordinates of the center of the source image, the second horizontal pixel density, and the second vertical pixel density, determine the spatial coordinates of the upper left corner and the lower right corner of the rectangle of the source image; The rectangular boundary of the source image is determined based on the spatial coordinates of the upper left corner and the lower right corner of the rectangle.

9. The method according to claim 3, characterized in that, Determining the source image pixel coordinates of the second intersection point includes: Based on the spatial coordinates of the second intersection point, determine the horizontal and vertical distances of the second intersection point relative to the top left corner of the rectangle of the source image; The source image horizontal pixel coordinates of the second intersection point are determined based on the second horizontal resolution of the source image, the horizontal distance, and the second horizontal pixel density. The source image vertical pixel coordinates of the second intersection point are determined based on the second vertical resolution of the source image, the vertical distance, and the second vertical pixel density.

10. An electronic device, characterized in that, include: The electronic device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the image distortion simulation method as described in any one of claims 1 to 9.

Images