A method and device for generating a virtual distorted image

Abstract

An embodiment of the present invention provides a method and apparatus for generating a virtual distorted image, which relates to the field of virtual reality technology. The method includes: determining the color parameters to be rendered for each pixel point in the image plane of a virtual camera; wherein, the color parameters to be rendered are obtained by mapping each virtual object in a three-dimensional simulation scene to the image plane in the case of no camera distortion; determining, based on the distortion parameters corresponding to the camera distortion type, the distorted pixel points corresponding to each pixel point in the image plane; and rendering the color parameters of each pixel point as the color parameters to be rendered for the distorted pixel points corresponding to each pixel point, so as to obtain a virtual distorted image. Compared with the prior art, by applying the solution provided by the embodiment of the present invention, it is possible to avoid pixel loss in the virtual distorted image and improve the image quality.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular, to a method and device for generating virtual distorted images. Background Art

[0002] With the rapid development of multimedia technology and computer technology, virtual reality technology has gradually entered the public's view and become a part of people's daily lives.

[0003] Generally, using virtual reality technology, people can model a preset virtual camera and a preset three-dimensional simulation scene. Thus, image acquisition of the preset three-dimensional simulation scene is performed through the preset virtual camera to achieve the simulation of image acquisition of a real scene by a real camera.

[0004] However, when image acquisition of a real scene is performed by a real camera, due to reasons such as the lens shape of the camera lens and errors during the assembly process, various types of camera distortions will occur. Thus, the acquired images usually have distortions.

[0005] In order to increase the realism of the images acquired by the virtual camera, generally, a certain degree of distortion can be added to the images acquired by the virtual camera to obtain virtual distorted images, making them have a more realistic effect and enhancing the user experience.

[0006] However, in the related art, after performing post-processing operations on the images acquired by the virtual camera, the edges of the obtained virtual distorted images are blurred, pixels in the virtual distorted images are prone to loss, and the image quality is poor. Summary of the Invention

[0007] The purpose of the embodiments of the present invention is to provide a method and device for generating virtual distorted images to avoid pixel loss in virtual distorted images and improve image quality. The specific technical solutions are as follows:

[0008] In a first aspect, the embodiments of the present invention provide a method for generating a virtual distorted image, the method including:

[0009] Determine the color parameters to be rendered for each pixel point in the image plane of the virtual camera; wherein, the color parameters to be rendered are obtained by mapping each virtual object in the three-dimensional simulation scene to the image plane in the absence of camera distortion;

[0010] Based on the distortion parameters corresponding to the camera distortion type, determine the distorted pixel points corresponding to each pixel point in the image plane;

[0011] Render the color parameter of each pixel point as the color parameter to be rendered of the distorted pixel point corresponding to this pixel point to obtain a virtual distorted image.

[0012] Optionally, in a specific implementation, determining the color parameters to be rendered for each pixel in the image plane of the virtual camera includes:

[0013] For each pixel, determine a path that starts from the coordinate point corresponding to this pixel in the camera coordinate system of the virtual camera and has the direction pointing from the coordinate point to the origin of the camera coordinate system as the propagation direction, as the first light propagation path corresponding to this pixel;

[0014] Using the conversion relationship between the camera coordinate system and the world coordinate system corresponding to the three-dimensional simulation scene, map the first light propagation path corresponding to each pixel into the world coordinate system to obtain the second light propagation path corresponding to each pixel;

[0015] Determine the coloring result after collision with each virtual object when the second light propagation path corresponding to each pixel propagates in the three-dimensional simulation scene, as the color parameter to be rendered for each pixel.

[0016] Optionally, in a specific implementation, determining the coloring result after collision with each virtual object when the second light propagation path corresponding to each pixel propagates in the three-dimensional simulation scene includes:

[0017] For each pixel, determine the color parameters of the intersection points generated by the collision of the second light propagation path corresponding to this pixel with each virtual object when propagating in the three-dimensional simulation scene; and use the determined color parameters to determine the coloring result after collision.

[0018] Optionally, in a specific implementation, the step of using the determined color parameters to determine the coloring result after collision includes:

[0019] According to the second light propagation path corresponding to this pixel, determine the arrangement order of each intersection point;

[0020] According to the color parameters of the last intersection point and the transformation method of the second light propagation path corresponding to this pixel at the last intersection point, determine the color parameters carried by the light connecting the last intersection point and the previous intersection point of the last intersection point;

[0021] For each intersection point except the last intersection point and the first intersection point, according to the color parameters of this intersection point, the color parameters carried by the light connecting this intersection point and the previous intersection point of this intersection point, and the transformation method of the second light propagation path corresponding to this pixel at this intersection point, determine the color parameters carried by the light connecting this intersection point and the previous intersection point of this intersection point;

[0022] For the first intersection point, based on the color parameter of the first intersection point, the color parameter carried by the light connecting the first intersection point and the previous intersection point of the first intersection point, and the transformation manner of the second light propagation path corresponding to the pixel point at the first intersection point, determine the color parameter carried by the light connecting the first intersection point and the starting point of the second light propagation path corresponding to the pixel point, as the coloring result after the collision.

[0023] Optionally, in a specific implementation, the determining the distorted pixel point corresponding to each pixel point in the image plane based on the distortion parameter corresponding to the camera distortion type includes:

[0024] Based on the distortion parameter corresponding to the camera distortion type, determine the distorted coordinate point corresponding to each pixel point in the camera coordinate system, and obtain the distorted coordinate point of each pixel point;

[0025] Determine the pixel point corresponding to the distorted coordinate point of each pixel point in the image plane, as the distorted pixel point corresponding to each pixel point in the image plane.

[0026] In a second aspect, an embodiment of the present invention provides a virtual distorted image generation device, the device includes:

[0027] A color parameter determination module, configured to determine the color parameter to be rendered of each pixel point in the image plane of the virtual camera; wherein, the color parameter to be rendered is obtained by mapping each virtual object in the three-dimensional simulation scene to the image plane in the case of no camera distortion;

[0028] A distorted pixel point determination module, configured to determine the distorted pixel point corresponding to each pixel point in the image plane based on the distortion parameter corresponding to the camera distortion type;

[0029] A pixel point rendering module, configured to render the color parameter of each pixel point as the color parameter to be rendered of the distorted pixel point corresponding to the pixel point, to obtain a virtual distorted image.

[0030] Optionally, in a specific implementation, the device further includes:

[0031] A task execution module, configured to use the virtual distorted image to execute a scene analysis task on the real scene corresponding to the virtual camera, to obtain a scene analysis result on the real scene.

[0032] Optionally, in a specific implementation, the color parameter determination module includes:

[0033] The first path determination sub-module is configured to, for each pixel point, determine a path with the coordinate point corresponding to the pixel point in the camera coordinate system of the virtual camera as the starting point and the direction from the coordinate point to the origin of the camera coordinate system as the propagation direction, as the first light propagation path corresponding to the pixel point;

[0034] The second path determination sub-module is configured to use the conversion relationship between the camera coordinate system and the world coordinate system corresponding to the three-dimensional simulation scene to map the first light propagation path corresponding to each pixel point into the world coordinate system, obtaining the second light propagation path corresponding to each pixel point;

[0035] The color parameter determination sub-module is configured to determine the coloring result after collision with each virtual object during the propagation of the second light propagation path corresponding to each pixel point in the three-dimensional simulation scene, as the color parameter to be rendered for each pixel point.

[0036] Optionally, in a specific implementation manner, the color parameter determination sub-module includes:

[0037] The color parameter determination unit is configured to, for each pixel point, determine the color parameter of the intersection point generated by the collision of the second light propagation path corresponding to the pixel point with each virtual object during the propagation in the three-dimensional simulation scene; and use the determined color parameters to determine the coloring result after the collision.

[0038] Optionally, in a specific implementation manner, the color parameter determination unit is specifically configured to:

[0039] Determine the arrangement order of each intersection point according to the second light propagation path corresponding to the pixel point;

[0040] According to the color parameter of the last intersection point and the transformation method of the second light propagation path corresponding to the pixel point at the last intersection point, determine the color parameter carried by the light connecting the last intersection point and the previous intersection point of the last intersection point;

[0041] For each intersection point except the last intersection point and the first intersection point, according to the color parameter of the intersection point, the color parameter carried by the light connecting the intersection point and the previous intersection point of the intersection point, and the transformation method of the second light propagation path corresponding to the pixel point at the intersection point, determine the color parameter carried by the light connecting the intersection point and the previous intersection point of the intersection point;

[0042] For the first intersection point, based on the color parameter of the first intersection point, the color parameter carried by the light connecting the first intersection point and the previous intersection point of the first intersection point, and the transformation manner of the second light propagation path corresponding to the pixel point at the first intersection point, determine the color parameter carried by the light connecting the first intersection point and the starting point of the second light propagation path corresponding to the pixel point, as the coloring result after the collision.

[0043] Optionally, in a specific implementation manner, the distorted pixel point determination module includes:

[0044] A coordinate point determination sub-module, configured to determine the distorted coordinate point corresponding to each pixel point in the camera coordinate system based on the distortion parameter corresponding to the camera distortion type, and obtain the distorted coordinate point of each pixel point;

[0045] A distortion point determination sub-module, configured to determine the pixel point corresponding to the distorted coordinate point of each pixel point in the image plane as the distorted pixel point corresponding to each pixel point in the image plane.

[0046] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0047] The memory is used to store a computer program;

[0048] The processor is configured to implement the steps of any virtual distortion image generation method in the first aspect when executing the program stored on the memory.

[0049] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of any virtual distortion image generation method in the first aspect are implemented.

[0050] In a fifth aspect, an embodiment of the present invention further provides a computer program product containing instructions, which when running on a computer, causes the computer to execute the steps of any virtual distortion image generation method provided in the first aspect.

[0051] Beneficial effects of the embodiments of the present invention:

[0052] As can be seen above, when applying the solution provided by the embodiments of the present invention to collect images of a three-dimensional simulation scene using a virtual camera to obtain a virtual distorted image, the color parameters to be rendered for each pixel in the image plane of the virtual camera can be determined first; among them, the color parameters to be rendered for each pixel are the color parameters of each pixel in the non-distorted image obtained when each virtual object in the three-dimensional simulation scene is mapped to the image plane of the virtual camera in the absence of camera distortion. In this way, without generating a non-distorted image, the color parameters of each pixel in the non-distorted image can be directly obtained as the color parameters to be rendered for each pixel. Furthermore, based on the distortion parameters corresponding to the type of camera distortion, the distorted pixel corresponding to each pixel in the above image plane can be determined. Then, the color parameter of each pixel in the image plane can be rendered as the color parameter to be rendered for the distorted pixel corresponding to this pixel, thereby obtaining a virtual distorted image.

[0053] Based on this, when applying the solution provided by the embodiments of the present invention, during the process of the virtual camera collecting images of the three-dimensional simulation scene, the principle of various types of camera distortion occurring in a real camera can be used for distortion simulation, and a virtual distorted image capable of simulating the distortion effects of various types of camera distortion can be directly generated. In this way, the situation of edge blurring of the virtual distorted image caused by post-processing the images collected by the virtual camera can be avoided, thereby avoiding pixel loss in the virtual machine distorted image and improving the image quality. Furthermore, the obtained virtual distorted image can be made closer to the distortion effect of the image collected by the real camera, improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other embodiments based on these drawings.

[0055] Figure 1 FIG. 12 is a schematic diagram of the simulation imaging principle of a virtual camera based on the pinhole imaging principle;

[0056] FIG. 2(a) is a schematic diagram of pincushion distortion;

[0057] FIG. 2(b) is a schematic diagram of barrel distortion;

[0058] FIG. 2(c) is a lens assembly method of a camera with tangential distortion;

[0059] FIG. 2(d) is a schematic diagram of tangential distortion in the lens assembly method of FIG. 2(c);

[0060] Figure 3 Schematic flow diagram of a method for generating a virtual distorted image provided by an embodiment of the present invention;

[0061] Fig. 4(a) is the original image of a virtual image in an embodiment;

[0062] Fig. 4(b) is the virtual distorted image of the barrel distortion of Fig. 4(a);

[0063] Fig. 4(c) is the virtual distorted image of the pincushion distortion of Fig. 4(a);

[0064] Fig. 4(d) is the virtual distorted image of the tangential distortion of Fig. 4(a);

[0065] Figure 5 is Figure 3 Schematic flow diagram of a specific implementation manner of S301 in;

[0066] Figure 6 Schematic flow diagram of a specific implementation manner of determining the coloring result after collision by using the determined respective color parameters;

[0067] Figure 7 Schematic structural diagram of a virtual distorted image generating device provided by an embodiment of the present invention;

[0068] Figure 8 Schematic structural diagram of an electronic device provided by an embodiment of the present invention. Specific embodiments

[0069] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art based on this application belong to the scope of protection of the present invention.

[0070] In the related art, the generation method of virtual distorted images is as follows: first, obtain the image without distortion collected by the virtual camera, then determine the corresponding distortion parameters according to the preset distortion requirements, and use the distortion parameters to perform an affine transformation on the image without distortion to obtain the distorted image, that is, obtain the virtual distorted image. However, in the above-mentioned related art, performing post-processing operations on the image collected by the virtual camera can cause the edges of the obtained virtual distorted image to be blurred, thereby causing pixel loss in the virtual machine distorted image and poor image quality.

[0071] To solve the above technical problems, an embodiment of the present invention provides a method for generating a virtual distorted image.

[0072] Among them, the generation method can be applicable to various application scenarios that require generating virtual distorted images, such as, for example, simulated driving, virtual surgery, museum cloud exhibition platforms, etc. Moreover, the generation method can be applied to various electronic devices such as VR (Virtual Reality) devices, servers, etc. And the execution entity of the generation method can be a single electronic device or a cluster of electronic devices, hereinafter referred to as electronic devices.

[0073] A virtual distorted image generation method provided by an embodiment of the present invention may include the following steps:

[0074] Determine the color parameters to be rendered for each pixel point in the image plane of the virtual camera; wherein, the color parameters to be rendered are obtained by mapping each virtual object in the three-dimensional simulation scene to the image plane in the case of no camera distortion.

[0075] Based on the distortion parameters corresponding to the camera distortion type, determine the distorted pixel points corresponding to each pixel point in the image plane.

[0076] Render the color parameter of each pixel point as the color parameter to be rendered of the distorted pixel point corresponding to each pixel point, to obtain a virtual distorted image.

[0077] As can be seen above, when applying the solution provided by the embodiment of the present invention to collect an image of a three-dimensional simulation scene by using a virtual camera to obtain a virtual distorted image, the color parameters to be rendered for each pixel point in the image plane of the virtual camera can be determined first; wherein, the color parameter to be rendered for each pixel point is the color parameter of each pixel point in the non-distorted image obtained when each virtual object in the three-dimensional simulation scene is mapped to the image plane of the virtual camera in the case of no camera distortion. In this way, the color parameters of each pixel point in the non-distorted image can be directly obtained without generating the non-distorted image, as the color parameters to be rendered for each pixel point. Furthermore, based on the distortion parameters corresponding to the camera distortion type, the distorted pixel points corresponding to each pixel point in the above-mentioned image plane can be determined. Then, the color parameter of each pixel point in the image plane can be rendered as the color parameter to be rendered of the distorted pixel point corresponding to the pixel point, thereby obtaining a virtual distorted image.

[0078] Based on this, by applying the solution provided in the embodiments of the present invention, during the process of image acquisition of a three-dimensional simulation scene by a virtual camera, the principle of various types of camera distortion occurring in a real camera can be used for distortion simulation, and a virtual distorted image capable of simulating the distortion effects of various types of cameras can be directly generated. In this way, the situation of blurred edges of the virtual distorted image caused by post-processing operations on the images captured by the virtual camera can be avoided. Thus, pixel loss in the virtual distorted image can be avoided, and the image quality can be improved. Furthermore, the obtained virtual distorted image can be made closer to the distortion effect of the image captured by the real camera, improving the user experience.

[0079] To better understand a virtual distorted image generation method provided in the embodiments of the present invention, before describing a virtual distorted image generation method provided in the embodiments of the present invention, first, the relevant technical content involved in the embodiments of the present invention will be introduced.

[0080] (1) Three-dimensional simulation scene: A virtual scene generated through three-dimensional modeling that can simulate various real scenes, such as a room, a square, a park, etc.

[0081] (2) Virtual camera: A camera obtained by simulating a real camera that can perform image acquisition on a three-dimensional simulation scene. The coordinate transformation relationship between its image plane, camera coordinate system, and the world coordinate system corresponding to the three-dimensional simulation scene where it is located is the same as that of the real camera.

[0082] Generally, the principle of pinhole imaging can be used to simulate the generation of a virtual camera. As Figure 1 shown, it is the simulated imaging principle of a virtual camera obtained using the pinhole imaging principle.

[0083] Among them, the coordinate system with the camera optical center as the origin and the optical axis as the Z-axis is called the camera coordinate system. The origin of the camera coordinate system is the center point of the camera, that is, the small hole in pinhole imaging. Furthermore, the transformation relationship between the camera coordinate system and the world coordinate system is as follows:

[0084]

[0085] Among them, R and t respectively represent a preset rotation matrix and a preset translation matrix. [X c Y c Z c 1] T is the coordinate of point m i in the camera coordinate system, and [X w Y w Z w 1] T is the coordinate of point M i in the world coordinate system.

[0086] The conversion from the camera coordinate system to the image coordinate system of the virtual camera's image plane is essentially the projection of the camera space where the camera coordinate system is located onto the image plane. Furthermore, the resulting projection is the imaging result of the three-dimensional simulation scene on the image plane. The conversion relationship between the camera coordinate system and the image coordinate system is as follows:

[0087]

[0088] Normally, under ideal circumstances, there is also the following equation:

[0089]

[0090]

[0091]

[0092] Among them, W and H are the resolutions of the image plane in the width and height directions respectively, and w and h are the actual sizes of the virtual film of the virtual camera in the width and height directions respectively, that is, the sizes of the captured images in the width and height directions. For example, taking a full-frame DSLR virtual camera as an example, in its virtual film, w is 36mm and h is 24mm.

[0093] The origin of the image coordinate system is the upper left corner of the virtual film to facilitate the translation operation of coordinate points during the coordinate transformation process.

[0094] (3) Ray tracing: Also known as beam projection method, it is a method to present three-dimensional images on a two-dimensional screen. Taking Figure 1 as an example, a ray tracing process can include the following steps:

[0095] 1. Determine a pixel point p(x, y) in the image plane. Among them, the result of ray tracing for p(x, y) is to color and render this point in the image plane.

[0096] 2. Normalize the pixel coordinates of the pixel point p(x, y) according to the actual sizes of the virtual film of the virtual camera in the width and height directions to obtain p s (x s ,y s ), where

[0097] p s (x s ,y s ) = (p(x, y) / (W, H)) * 2.0 - 1.0.

[0098] Among them, the essence of the above step 2 is to convert the pixel coordinates of each pixel point p(x, y) on the virtual negative of the virtual camera to the range of [-1, 1].

[0099] 3. Combine with the focal length parameter f of the virtual camera to convert the above normalized pixel coordinates into the camera coordinate system, and obtain the corresponding point p in the camera coordinate system c (x c , y c , z c ), where

[0100] p c (x c , y c , z c ) = (x s w, y c h, -f)

[0101] Among them, in the above step 3, the virtual negative of the virtual camera can be used as a plane perpendicular to the Z-axis of the camera coordinate system and at a distance of the focal length parameter f from the origin of the camera coordinate system. Thus, for each pixel point p(x, y) in the image plane, its normalized coordinates p s (x s , y s ) can be restored according to the size of the virtual negative of the virtual camera. Thus, the coordinates of the pixel point in the XOY plane of the camera coordinate system can be obtained, and -f is used as the coordinate of the pixel point in the Z-axis direction of the camera coordinate system.

[0102] 4. Determine the light parameters in the camera coordinate system, that is, the starting point of the light is p c (x c , y c , z c ), and the direction of the light is

[0103] 5. According to the transformation relationship between the camera coordinate system and the world coordinate system, convert the above light starting point and direction into the world coordinate system to obtain the light parameters in the world coordinate system.

[0104] 6. The light corresponding to the light parameters in the world coordinate system propagates in the three-dimensional simulation scene, and it collides and colors with the objects in the three-dimensional simulation scene. Then, the finally obtained collision coloring result is the color value of the pixel point p(x, y) in the image screen.

[0105] (4) Image distortion: Due to mechanical accuracy and other problems, real cameras usually have various types of camera distortions, which in turn cause the real images collected to be distorted. Among them, camera distortion is mainly divided into two types: radial distortion and tangential distortion.

[0106] Radial distortion is the distortion caused by the shape of the camera lens, including barrel distortion and pincushion distortion. In the pinhole model, a straight line projects onto the pixel plane as a straight line. However, in reality, the camera lens often causes a straight line in the real environment to project as a curve in the image, and the distortion becomes more obvious towards the edge of the image. Furthermore, since the camera lens is often centrosymmetric, the distortion is usually radially symmetric. As shown in Fig. 2(a), it is a schematic diagram of pincushion distortion, and as shown in Fig. 2(b), it is a schematic diagram of barrel distortion.

[0107] Tangential distortion is caused by the fact that the camera lens and the image plane are not strictly parallel during the camera assembly process. As shown in Fig. 2(c), it is a lens assembly method of a camera with tangential distortion, and as shown in Fig. 2(d), it is a schematic diagram of tangential distortion under the lens assembly method of Fig. 2(c).

[0108] Next, in conjunction with the accompanying drawings, a method for generating a virtual distorted image provided by an embodiment of the present invention will be described.

[0109] Figure 3 It is a schematic flowchart of a method for generating a virtual distorted image provided by an embodiment of the present invention. As Figure 3 shown, the method may include the following steps:

[0110] S301: Determine the color parameters to be rendered for each pixel point in the image plane of the virtual camera;

[0111] Among them, the color parameters to be rendered are obtained by mapping each virtual object in the three-dimensional simulation scene onto the image plane in the absence of camera distortion;

[0112] When the virtual camera captures an image of the three-dimensional simulation scene, each pixel point in the finally obtained image is the projection of each virtual object in the three-dimensional simulation scene onto the image plane of the virtual camera.

[0113] Furthermore, since the three-dimensional simulation scene is a scene obtained by simulating the real scene through modeling, various virtual objects may exist in the three-dimensional simulation scene according to the simulated real scene, and each virtual object in the three-dimensional simulation scene may have color parameters.

[0114] Among them, the color parameters of each virtual object in the three-dimensional simulation scene may be color parameters in any color mode, for example, RGB mode, HSV model, etc. Of course, it is not limited thereto.

[0115] In this way, in the absence of camera distortion, for each pixel point in the image plane of the virtual camera, based on the color parameters of each virtual object in the three-dimensional simulation scene and the imaging principle of the virtual camera, when each virtual object in the three-dimensional simulation scene is mapped to the image plane, the color parameters to be rendered for each pixel point in the image plane of the virtual camera can be obtained.

[0116] Based on this, when generating a virtual distorted image of the virtual camera for the three-dimensional simulation scene, the color parameters to be rendered for each pixel point in the image plane of the virtual camera can be determined first. Among them, the color parameter to be rendered for each pixel point is: in the absence of camera distortion, when each virtual object in the three-dimensional simulation scene is mapped to this pixel point in the image plane, the color parameter of this pixel point.

[0117] S302: Based on the distortion parameters corresponding to the camera distortion type, determine the distorted pixel point corresponding to each pixel point in the image plane;

[0118] Different types of camera distortion can correspond to different distortion parameters. For each pixel point in the image plane of the virtual camera, the distorted pixel point corresponding to this pixel point can be determined based on the distortion parameters corresponding to the camera distortion type.

[0119] That is to say, for each pixel point in the image plane of the virtual camera, the distorted pixel point corresponding to this pixel point is: another pixel point in the image plane of the virtual camera.

[0120] Optionally, the distortion parameters corresponding to the camera distortion type can be pre-recorded in the electronic device. Thus, the distorted pixel point corresponding to each pixel point in the image plane can be directly determined based on the distortion parameters corresponding to the camera distortion type.

[0121] Optionally, the correspondence between the camera distortion type and the distortion parameters can be pre-set in the electronic device. Then, each time a virtual distorted image is generated, the electronic device can determine the camera distortion type used for generating this virtual distorted image according to operations such as user input and point selection, or according to the pre-set transformation rules of the camera distortion type, etc. Furthermore, among the above correspondences, the distortion parameters corresponding to the camera distortion type used this time can be selected. Thus, based on the selected distortion parameters, the distorted pixel point corresponding to each pixel point in the image plane can be determined.

[0122] S303: Render the color parameter of each pixel point as the color parameter to be rendered for the distorted pixel point corresponding to this pixel point, and obtain the virtual distorted image.

[0123] For each pixel point in the obtained virtual distorted image, which is distorted from a certain pixel point in the image plane of the virtual camera. For the sake of clarity in writing, the pixel points in the image plane of the virtual camera will be referred to as the first pixel points hereinafter, and the pixel points in the obtained virtual distorted image will be referred to as the second pixel points.

[0124] That is to say: For each second pixel point, the first pixel point with the same pixel coordinates as this second pixel point in the image plane of the virtual camera can be obtained. Furthermore, the distorted pixel point corresponding to this first pixel point can be obtained. In this way, when generating the virtual distorted image, the above-obtained distorted pixel points move from their original positions in the image plane of the virtual camera to the positions of these second pixel points in the virtual distorted image.

[0125] Based on this, for the image plane of the virtual camera, the color parameter of each pixel point can be rendered as the color parameter to be rendered of the distorted pixel point corresponding to this pixel point.

[0126] Among them, since the distorted pixel point corresponding to each pixel point is another pixel point in the image plane of the virtual camera, therefore, according to the color parameters to be rendered of each pixel point determined above, the color parameters to be rendered of the distorted pixel point corresponding to each pixel point can be directly obtained.

[0127] For example, determine the pixel point A0 in the image plane, and use the distortion parameters corresponding to the camera distortion type to determine the distorted pixel point A2 corresponding to the pixel point A0. Furthermore, in the finally obtained virtual distorted image, the pixel point A0 is distorted from the pixel point A2 in the image plane. In this way, in the obtained virtual distorted image, the color parameter of the pixel point A0 is the color parameter to be rendered of the determined pixel point A2.

[0128] In this way, for each pixel point in the image plane, the color parameter of this pixel point can be rendered as the color parameter to be rendered of the distorted pixel point corresponding to this pixel point. That is to say, for each point in the image plane, this pixel point can be colored using the color parameter to be rendered of the distorted pixel point corresponding to this pixel point.

[0129] Furthermore, after all the pixel points belonging to the virtual negative film of the virtual camera in the image plane are rendered as the color parameters to be rendered of the corresponding distorted pixel points, the virtual distorted image can be obtained.

[0130] For example, Fig. 4(a) is the original image of a certain virtual image, then Fig. 4(b) is the virtual distorted image of the barrel distortion of Fig. 4(a), Fig. 4(c) is the virtual distorted image of the pincushion distortion of Fig. 4(a), and Fig. 4(d) is the virtual distorted image of the tangential distortion of Fig. 4(a).

[0131] In many cases, the real cameras installed in the real scenario have camera distortion. As a result, the real images captured are not in line with the requirements of subsequent tasks, leading to large deviations in the results of subsequent tasks based on the real images. Even worse, subsequent tasks cannot be executed based on the real images.

[0132] To reduce the camera distortion of real cameras and obtain more accurate task results, a virtual camera can be used to simulate the real camera. Then, the obtained virtual distorted images can be used to analyze the real scenario, and the real camera can be adjusted according to the analysis results. Moreover, due to the influence of the performance of real cameras, in many cases where image-based scene analysis of the scenario is required, it is impossible to obtain enough real images of the real scenario captured by real cameras. Therefore, the real camera and the real scenario can also be simulated separately to construct a virtual camera and a three-dimensional simulation scenario. The virtual camera is used to collect virtual images of the three-dimensional simulation scenario to obtain virtual distorted images, and then the obtained virtual distorted images are used to perform scene analysis on the real scenario.

[0133] Based on this, optionally, in a specific implementation manner, a method for generating training distorted images provided in the above embodiments of the present invention may further include the following step 11:

[0134] Step 11: Use the virtual distorted images to perform a scene analysis task on the real scenario corresponding to the virtual camera to obtain a scene analysis result of the real scenario.

[0135] In this specific implementation manner, the virtual camera simulates the real camera. Therefore, the shooting angles and camera parameter-related information of the virtual camera and the simulated real camera are the same. As a result, the distortion type and degree of the virtual distorted images obtained by the virtual camera are the same as those of the real images captured by the real camera.

[0136] For example, three-dimensional modeling technology can be used to model the real scenario where the real camera is installed to obtain a three-dimensional simulation scenario with a virtual camera installed. That is to say, the three-dimensional simulation scenario with a virtual camera installed is a virtual version of the above real scenario with a real camera installed. This three-dimensional simulation scenario can simulate various events occurring in the real scenario, and the virtual camera can simulate the real camera to collect images, and obtain virtual distorted images with the same distortion type and degree as the real images captured by the real camera.

[0137] In this way, a scene analysis task for the real scene corresponding to the virtual camera can be preset, where the real scene corresponding to the virtual camera is: the real scene where the real camera simulated by the virtual camera is located. Further, after obtaining the virtual distorted image generated by using the above steps S301 - S303, the preset scene analysis task can be executed by using the virtual distorted image, and a scene analysis result for the above real scene can be obtained.

[0138] For example, the generated virtual distorted image can be used to analyze the camera distortion of the real camera. According to the scene analysis result, it can be determined whether the real camera is available. Exemplarily, a scene analysis model is trained with the virtual distorted image, and the pictures taken by the real camera in the real scene are input into the scene analysis model to analyze the camera distortion of the real camera and obtain the scene analysis result. When the scene analysis result indicates that the camera distortion of the real camera exceeds the distortion threshold, it can be determined that the real camera is not available; also exemplarily, when the scene analysis result indicates that the camera distortion of the real camera is a preset distortion type, it can be determined that the real camera is not available, etc.

[0139] Also for example, the virtual camera can simulate the lens rotation situation of the real camera to obtain multiple virtual distorted images at different shooting angles. Thus, the scene analysis of the real scene corresponding to the virtual camera can be performed based on the multiple virtual distorted images, and relevant information about the real scene can be determined according to the scene analysis result. Exemplarily, it can be determined whether the environment of the real scene is safe, whether there are objects such as specified types of vehicles and people in the real scene, whether the position of the real camera installed in the real scene is appropriate, etc.

[0140] That is to say, in this case, the multiple generated virtual distorted images can be used to replace the real scene images collected by the real camera to implement operations such as monitoring and analyzing the real scene.

[0141] Of course, the above scene analysis task can also include other types of tasks, and in this regard, the embodiments of the present invention do not make specific actions. The above scene analysis task can be implemented by using a neural network model or other analysis algorithms.

[0142] Based on this, by applying the solution provided in the embodiments of the present invention, during the process of the virtual camera collecting images of the three - dimensional simulation scene, the principle of various types of camera distortions occurring in the real camera can be used for distortion simulation to directly generate virtual distorted images that can simulate the distortion effects of various types of camera distortions. In this way, the situation of blurred edges of the virtual distorted image caused by post - processing operations on the images collected by the virtual camera can be avoided, thereby avoiding pixel loss in the virtual distorted image and improving the image quality. Further, the obtained virtual distorted image can be made closer to the distortion effect of the image collected by the real camera, improving the user experience.

[0143] Optionally, in a specific implementation, as Figure 5 shown, the above step S301 of determining the color parameters to be rendered for each pixel point in the image plane of the virtual camera may include the following steps:

[0144] S3011: For each pixel point, determine a path with the coordinate point corresponding to this pixel point in the camera coordinate system of the virtual camera as the starting point and the direction from the coordinate point to the origin of the camera coordinate system as the propagation direction, as the first light propagation path corresponding to this pixel point;

[0145] S3012: Utilize the conversion relationship between the camera coordinate system and the world coordinate system corresponding to the three-dimensional simulation scene to map the first light propagation path corresponding to each pixel point into the world coordinate system, obtaining the second light propagation path corresponding to each pixel point;

[0146] S3013: Determine the coloring result after the second light propagation path corresponding to each pixel point collides with each virtual object during propagation in the three-dimensional simulation scene, as the color parameter to be rendered for each pixel point.

[0147] In this specific implementation, based on the coordinate conversion relationship between the plane coordinate system of the image plane of the virtual camera and the camera coordinate system of the virtual camera as described above, for each pixel point in the image plane of the virtual camera, this pixel point can be converted into the coordinate point in the camera coordinate system of the virtual camera, that is, the coordinate point corresponding to this pixel point in the camera coordinate system of the virtual camera is obtained.

[0148] Furthermore, based on the ray tracing principle as described above, for each pixel point in the image plane of the virtual camera, the coordinate point corresponding to this pixel point in the camera coordinate system of the virtual camera can be used as the starting point of light propagation. Thus, the direction from the coordinate point corresponding to this pixel point to the origin of the above camera coordinate system can be determined, and this direction can be used as the light propagation direction.

[0149] In this way, the first light propagation path corresponding to this pixel point can be obtained, where the starting point of this first light propagation path is: the coordinate point corresponding to this pixel point in the camera coordinate system of the virtual camera, and the propagation direction of this first light propagation path is: the direction from the coordinate point corresponding to this pixel point in the camera coordinate system of the virtual camera to the origin of the camera coordinate system.

[0150] Furthermore, when the virtual camera captures images of the three-dimensional simulation scene, each pixel in the finally obtained image is the projection of each virtual object in the three-dimensional simulation scene on the image plane of the virtual camera. Therefore, the first light propagation path corresponding to each obtained pixel can be converted into the second light propagation path corresponding to each pixel in the above world coordinate system through the coordinate conversion relationship between the camera coordinate system of the virtual camera and the world coordinate system corresponding to the three-dimensional simulation scene.

[0151] In this way, according to the theory of light propagation, for example, light travels in a straight line, light can be reflected and / or refracted when it collides with various objects, and when light collides with various objects, the light of a certain color carried by it can be absorbed by the object, etc.; for each pixel, the light propagating in the reverse direction along the second light propagation path corresponding to the pixel can collide with each virtual object in the three-dimensional virtual scene and can undergo various changes such as refraction, reflection, and light absorption.

[0152] Based on this, according to the theory of light propagation, for each pixel in the image plane of the virtual camera, the coloring result after the second light propagation path corresponding to the pixel collides with each virtual object during propagation in the three-dimensional simulation scene can be determined, and the determined coloring result is the color parameter to be rendered for the pixel.

[0153] Optionally, in a specific implementation, step S3013, determining the coloring result after the second light propagation path corresponding to each pixel collides with each virtual object during propagation in the three-dimensional simulation scene, may include the following steps 21:

[0154] Step 21: For each pixel, determine the color parameter of the intersection point generated when the second light propagation path corresponding to the pixel collides with each virtual object during propagation in the three-dimensional simulation scene; and use the determined color parameters to determine the coloring result after the collision.

[0155] In this specific implementation, for each pixel, according to the starting point and direction of the second light propagation path corresponding to the pixel, the entire path from the light source in the three-dimensional simulation scene to the above starting point during the propagation of the second light propagation path corresponding to the pixel in the three-dimensional simulation scene can be determined. Thus, each virtual object collided with by the above entire path in the three-dimensional simulation scene, and the intersection point of the above entire path and the virtual object when colliding with each virtual object can be determined. Thus, the color parameter of the intersection point can be obtained.

[0156] That is to say, for each pixel, when the second light propagation path corresponding to this pixel propagates in the three-dimensional simulation scene, the color parameters of the intersection points generated by the collision with each virtual object can be determined. In this way, according to the light propagation principle and the determined color parameters, the coloring result after the collision can be determined.

[0157] Optionally, in a specific implementation, as Figure 6 shown, in the above step 21, using the determined color parameters to determine the coloring result after the collision may include the following steps:

[0158] S601: Determine the arrangement order of each intersection point according to the second light propagation path corresponding to this pixel;

[0159] S602: According to the color parameter of the last intersection point and the transformation method of the second light propagation path corresponding to this pixel at the last intersection point, determine the color parameter carried by the light connecting the last intersection point and the previous intersection point of the last intersection point;

[0160] Among them, the above transformation method can be any one or more of the three changes of refraction, reflection, and partial absorption of light

[0161] S603: For each intersection point except the last intersection point and the first intersection point, according to the color parameter of this intersection point, the color parameter carried by the light connecting this intersection point and the previous intersection point of this intersection point, and the transformation method of the second light propagation path corresponding to this pixel at this intersection point, determine the color parameter carried by the light connecting this intersection point and the previous intersection point of this intersection point;

[0162] S604: For the first intersection point, according to the color parameter of the first intersection point, the color parameter carried by the light connecting the first intersection point and the previous intersection point of the first intersection point, and the transformation method of the second light propagation path corresponding to this pixel at the first intersection point, determine the color parameter carried by the light connecting the first intersection point and the starting point of the second light propagation path corresponding to this pixel as the coloring result after the collision.

[0163] In this specific implementation, based on the ray tracing principle as described above, for each pixel, the light emitted by the light source in the three-dimensional simulation scene can be transmitted along the path opposite to the second light propagation path corresponding to this pixel to the point corresponding to this pixel in the above world coordinate system.

[0164] Furthermore, for each pixel point, when the light rays emitted by the light source in the three-dimensional simulation scene are transmitted along the path opposite to the second light ray propagation path corresponding to this pixel point to the point corresponding to this pixel point in the above world coordinate system, the order of the intersection points generated by colliding with each virtual object is opposite to the order of the intersection points generated by colliding with each virtual object when the second light ray propagation path corresponding to this pixel point propagates in the three-dimensional simulation scene.

[0165] In this way, for each pixel point, the arrangement order of each intersection point can be determined according to the second light ray propagation path corresponding to this pixel point.

[0166] Furthermore, the last intersection point in the above arrangement order is the first intersection point generated by colliding with each virtual object when the light rays emitted by the light source in the three-dimensional simulation scene are transmitted along the path opposite to the second light ray propagation path corresponding to this pixel point to the point corresponding to this pixel point in the above world coordinate system; correspondingly, the first intersection point in the above arrangement order is the last intersection point generated by colliding with each virtual object when the light rays emitted by the light source in the three-dimensional simulation scene are transmitted along the path opposite to the second light ray propagation path corresponding to this pixel point to the point corresponding to this pixel point in the above world coordinate system.

[0167] According to the optical principle, when light rays collide with an object, they can be completely or partially absorbed by the object, or reflected or refracted by the object. Therefore, when the light rays propagate out again after colliding with the object, their colors can change, that is, when the light rays propagate out again after colliding with the object, the color parameters carried by the light rays can change.

[0168] In this way, for the last intersection point in the above arrangement order, the color parameters carried by the light ray connecting this last intersection point and the previous intersection point of this last intersection point can be determined according to the light source in the three-dimensional simulation scene, the color parameters of this last intersection point, and the transformation method of the second light ray propagation path corresponding to this pixel point at this last intersection point.

[0169] Furthermore, for each intersection point in the above arrangement order except the last intersection point and the first intersection point, the color parameters carried by the light ray connecting this intersection point and the previous intersection point of this intersection point can be determined according to the color parameters of this intersection point, the color parameters carried by the light ray connecting this intersection point and the previous intersection point of this intersection point, and the transformation method of the second light ray propagation path corresponding to this pixel point at this intersection point.

[0170] For the first intersection point in the above arrangement order, based on the color parameter of the first intersection point, the color parameter carried by the light connecting the first intersection point and the previous intersection point of the first intersection point, and the transformation method of the second light propagation path corresponding to the pixel point at the first intersection point, determine the color parameter carried by the light connecting the first intersection point and the starting point of the second light propagation path corresponding to the pixel point, as the coloring result after the collision.

[0171] Among them, the transformation methods in the above steps S602 - 604 can be at least one of: refraction, reflection, and partial absorption.

[0172] Optionally, in a specific implementation manner, the above step S302, based on the distortion parameters corresponding to the camera distortion type, determines the distorted pixel points corresponding to each pixel point in the image plane, and may include the following steps 31 - 32:

[0173] Step 31: Based on the distortion parameters corresponding to the camera distortion type, determine the distorted coordinate points corresponding to each pixel point in the camera coordinate system, and obtain the distorted coordinate points of each pixel point.

[0174] In this specific implementation manner, when determining the distorted pixel points corresponding to each pixel point in the image plane, based on the distortion parameters corresponding to the camera distortion type, determine the distorted coordinate points corresponding to each pixel point in the camera coordinate system, and obtain the distorted coordinate points of each pixel point.

[0175] Among them, optionally, when the virtual camera is simulated based on the principle of pinhole imaging, for example, for the virtual camera shown as Figure 1 the above step 31 may include the following step 311:

[0176] Step 311: Use a preset formula to calculate the distorted coordinate points of each pixel point; where the preset formula is:

[0177]

[0178]

[0179] Among them, is the first coordinate of the i-th pixel point in the camera coordinate system, is the second coordinate of the distorted coordinate point of the i-th pixel point in the camera coordinate system, f0 is the focal length of the virtual camera, k1, k2, k3, k4, k5, k6, p1, and p2 are the distortion parameters corresponding to the camera distortion type, and r is a parameter determined based on the pixel coordinates of the i-th pixel point in the image plane.

[0180] Among them, the distortion parameters corresponding to different camera distortion types can be empirical values determined by technicians based on experience, or can be determined by technicians through analyzing the real distortion images collected by real cameras, and both are reasonable.

[0181] Step 32: Determine the pixel points in the image plane corresponding to the distorted coordinate points of each pixel point, as the distorted pixel points corresponding to each pixel point in the image plane.

[0182] In this way, for each pixel point in the image plane of the virtual camera, after obtaining the distorted coordinate point of this pixel point, the distorted coordinate point can be converted into the corresponding pixel point in the image plane of the virtual camera, and the converted pixel point is the distorted pixel point corresponding to this pixel point in the image plane of the virtual camera.

[0183] Corresponding to the virtual distortion image generation method provided in the above embodiment of the present invention, the embodiment of the present invention also provides a virtual distortion image generation device.

[0184] Figure 7 Shown in the structural schematic diagram of a virtual distortion image generation device provided by the embodiment of the present invention, as Figure 7 shown, the device may include the following modules:

[0185] A color parameter determination module 701, configured to determine the color parameters to be rendered of each pixel point in the image plane of the virtual camera; among them, the color parameters to be rendered are obtained by mapping each virtual object in the three-dimensional simulation scene to the image plane in the case of no camera distortion;

[0186] A distorted pixel point determination module 702, configured to determine the distorted pixel point corresponding to each pixel point in the image plane based on the distortion parameters corresponding to the camera distortion type;

[0187] A pixel point rendering module 703, configured to render the color parameter of each pixel point as the color parameter to be rendered of the distorted pixel point corresponding to this pixel point, to obtain a virtual distortion image.

[0188] As can be seen above, when applying the solution provided by the embodiments of the present invention to collect images of a three-dimensional simulation scene using a virtual camera to obtain a virtual distorted image, the color parameters to be rendered for each pixel point in the image plane of the virtual camera can be determined first; among them, the color parameter to be rendered for each pixel point is the color parameter of each pixel point in the non-distorted image obtained when each virtual object in the three-dimensional simulation scene is mapped to the image plane of the virtual camera in the absence of camera distortion. In this way, the color parameters of each pixel point in the non-distorted image can be directly obtained as the color parameters to be rendered for each pixel point without generating a non-distorted image. Furthermore, based on the distortion parameters corresponding to the camera distortion type, the distorted pixel point corresponding to each pixel point in the image plane can be determined. Then, the color parameter of each pixel point in the image plane can be rendered as the color parameter to be rendered for the distorted pixel point corresponding to this pixel point, thereby obtaining a virtual distorted image.

[0189] Based on this, when applying the solution provided by the embodiments of the present invention, during the process of collecting images of a three-dimensional simulation scene using a virtual camera, pixel simulation can be performed by using the principle of various types of camera pixels occurring in a real camera, and a virtual pixel image capable of simulating the pixel effects of various types of camera pixels can be directly generated. In this way, the situation of blurred edges of the virtual pixel image caused by post-processing the image collected by the virtual camera can be avoided, thereby avoiding pixel loss in the virtual machine distorted image and improving the image quality. Furthermore, the obtained virtual pixel image can be made closer to the pixel effect of the image collected by the real camera, improving the user experience.

[0190] Optionally, in a specific implementation manner, the device further includes:

[0191] A task execution module, configured to use the virtual distorted image to execute a scene analysis task on the real scene corresponding to the virtual camera, and obtain a scene analysis result of the real scene.

[0192] Optionally, in a specific implementation manner, the color parameter determination module 701 includes:

[0193] A first path determination sub-module, configured to, for each pixel point, determine a path starting from the coordinate point corresponding to this pixel point in the camera coordinate system of the virtual camera and with the direction from the coordinate point to the origin of the camera coordinate system as the propagation direction as the first light propagation path corresponding to this pixel point;

[0194] The second path determination sub-module is configured to map the first light propagation path corresponding to each pixel point into the world coordinate system by using the conversion relationship between the camera coordinate system and the world coordinate system corresponding to the three-dimensional simulation scene, so as to obtain the second light propagation path corresponding to each pixel point;

[0195] The color parameter determination sub-module is configured to determine the coloring result after collision with each virtual object when the second light propagation path corresponding to each pixel point propagates in the three-dimensional simulation scene, as the color parameter to be rendered for each pixel point.

[0196] Optionally, in a specific implementation manner, the color parameter determination sub-module includes:

[0197] The color parameter determination unit is configured to, for each pixel point, determine the color parameter of the intersection point generated by the collision between the second light propagation path corresponding to the pixel point and each virtual object when propagating in the three-dimensional simulation scene; and use the determined color parameters to determine the coloring result after collision.

[0198] Optionally, in a specific implementation manner, the color parameter determination unit is specifically configured to:

[0199] Determine the arrangement order of each intersection point according to the second light propagation path corresponding to the pixel point;

[0200] According to the color parameter of the last intersection point and the transformation method of the second light propagation path corresponding to the pixel point at the last intersection point, determine the color parameter carried by the light connecting the last intersection point and the previous intersection point of the last intersection point;

[0201] For each intersection point except the last intersection point and the first intersection point, according to the color parameter of the intersection point, the color parameter carried by the light connecting the intersection point and the previous intersection point of the intersection point, and the transformation method of the second light propagation path corresponding to the pixel point at the intersection point, determine the color parameter carried by the light connecting the intersection point and the previous intersection point of the intersection point;

[0202] For the first intersection point, according to the color parameter of the first intersection point, the color parameter carried by the light connecting the first intersection point and the previous intersection point of the first intersection point, and the transformation method of the second light propagation path corresponding to the pixel point at the first intersection point, determine the color parameter carried by the light connecting the first intersection point and the starting point of the second light propagation path corresponding to the pixel point, as the coloring result after collision.

[0203] Optionally, in a specific implementation manner, the distorted pixel point determination module 702 is specifically configured to:

[0204] A coordinate point determination sub-module, configured to determine the distorted coordinate points corresponding to each pixel point in the camera coordinate system based on the distortion parameters corresponding to the camera distortion type, so as to obtain the distorted coordinate points of each pixel point;

[0205] A distorted point determination sub-module, configured to determine the pixel point corresponding to the distorted coordinate point of each pixel point in the image plane as the distorted pixel point corresponding to each pixel point in the image plane.

[0206] Corresponding to the virtual distorted image generation method provided in the above embodiments of the present invention, embodiments of the present invention further provide an electronic device, as Figure 8 shown, including a processor 801, a communication interface 802, a memory 803, and a communication bus 804. Among them, the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.

[0207] The memory 803 is used to store a computer program;

[0208] The processor 801, when executing the program stored on the memory 803, implements the steps of any virtual distorted image generation method provided in the above embodiments of the present invention.

[0209] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is used in the figure to represent it, but it does not mean that there is only one bus or one type of bus.

[0210] The communication interface is used for communication between the above electronic device and other devices.

[0211] The memory may include a Random Access Memory (RAM), and may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0212] The above-mentioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0213] In another embodiment provided by the present invention, a computer-readable storage medium is further provided. A computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the steps of any virtual distortion image generation method provided by the above-mentioned embodiments of the present invention are implemented.

[0214] In another embodiment provided by the present invention, a computer program product containing instructions is further provided. When it runs on a computer, the computer is enabled to execute the steps of any virtual distortion image generation method provided by the above-mentioned embodiments of the present invention.

[0215] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, a computer, a server, or a data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a Solid State Disk (SSD)).

[0216] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0217] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the apparatus embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference can be made to the corresponding parts of the method embodiments for the relevant content.

[0218] The above are only the preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for generating a virtual distorted image, characterized in that, The method includes: Determining the color parameters to be rendered for each pixel point in the image plane of the virtual camera; wherein, the color parameters to be rendered are obtained by mapping each virtual object in the three-dimensional simulation scene to the image plane in the case of no camera distortion; Determining, based on the distortion parameters corresponding to the camera distortion type, the distorted pixel points corresponding to each pixel point in the image plane; Rendering the color parameters of each pixel point as the color parameters to be rendered for the distorted pixel point corresponding to the pixel point, to obtain a virtual distorted image; The determining the color parameters to be rendered for each pixel point in the image plane of the virtual camera includes: For each pixel point, determining the color parameters of the intersection points generated by the collision of the second light propagation path corresponding to the pixel point with each virtual object when propagating in the three-dimensional simulation scene; Determining the arrangement order of each intersection point according to the second light propagation path corresponding to the pixel point; According to the color parameters of the last intersection point and the transformation mode of the second light propagation path corresponding to the pixel point at the last intersection point, determining the color parameters carried by the light connecting the last intersection point and the previous intersection point of the last intersection point; For each intersection point except the last intersection point and the first intersection point, according to the color parameters of the intersection point, the color parameters carried by the light connecting the intersection point and the previous intersection point of the intersection point, and the transformation mode of the second light propagation path corresponding to the pixel point at the intersection point, determining the color parameters carried by the light connecting the intersection point and the previous intersection point of the intersection point; For the first intersection point, according to the color parameters of the first intersection point, the color parameters carried by the light connecting the first intersection point and the previous intersection point of the first intersection point, and the transformation mode of the second light propagation path corresponding to the pixel point at the first intersection point, determining the color parameters carried by the light connecting the first intersection point and the starting point of the second light propagation path corresponding to the pixel point as the coloring result after the collision, wherein the previous intersection point of the first intersection point is the starting point of the second light propagation path corresponding to the pixel point; Wherein, the determining method of the second light propagation path corresponding to the pixel point includes: For each pixel point, determining the path starting from the coordinate point corresponding to the pixel point in the camera coordinate system of the virtual camera and with the direction from the coordinate point to the origin of the camera coordinate system as the propagation direction as the first light propagation path corresponding to the pixel point; using the conversion relationship between the camera coordinate system and the world coordinate system corresponding to the three-dimensional simulation scene, mapping the first light propagation path corresponding to each pixel point to the world coordinate system to obtain the second light propagation path corresponding to each pixel point.

2. The method according to claim 1, characterized in that, The method further includes: Using the virtual distorted image to perform a scene analysis task on the real scene corresponding to the virtual camera, to obtain a scene analysis result regarding the real scene.

3. The method according to any one of claims 1-2, characterized in that, Determining the distorted pixel points corresponding to each pixel point in the image plane based on the distortion parameters corresponding to the camera distortion type includes: Determining the distorted coordinate points corresponding to each pixel point in the camera coordinate system based on the distortion parameters corresponding to the camera distortion type, to obtain the distorted coordinate points of each pixel point; Determining the pixel points corresponding to the distorted coordinate points of each pixel point in the image plane as the distorted pixel points corresponding to each pixel point in the image plane.

4. A virtual distorted image generation device, characterized in that The device includes: A color parameter determination module, configured to determine the color parameters to be rendered for each pixel point in the image plane of the virtual camera; wherein, the color parameters to be rendered are obtained by mapping each virtual object in the three-dimensional simulation scene to the image plane in the case of no camera distortion; A distorted pixel point determination module, configured to determine the distorted pixel points corresponding to each pixel point in the image plane based on the distortion parameters corresponding to the camera distortion type; A pixel point rendering module, configured to render the color parameters of each pixel point as the color parameters to be rendered for the distorted pixel points corresponding to the pixel points, to obtain a virtual distorted image; The color parameter determination module includes: A color parameter determination sub-module, configured to, for each pixel point, determine the color parameters of the intersection points generated by the collision of the second light propagation path corresponding to the pixel point with each virtual object during propagation in the three-dimensional simulation scene; determine the arrangement order of each intersection point according to the second light propagation path corresponding to the pixel point; according to the color parameters of the last intersection point and the transformation manner of the second light propagation path corresponding to the pixel point at the last intersection point, determine the color parameters carried by the light connecting the last intersection point and the previous intersection point of the last intersection point; for each intersection point except the last intersection point and the first intersection point, according to the color parameters of the intersection point, the color parameters carried by the light connecting the intersection point and the previous intersection point of the intersection point, and the transformation manner of the second light propagation path corresponding to the pixel point at the intersection point, determine the color parameters carried by the light connecting the intersection point and the previous intersection point of the intersection point; for the first intersection point, according to the color parameters of the first intersection point, the color parameters carried by the light connecting the first intersection point and the previous intersection point of the first intersection point, and the transformation manner of the second light propagation path corresponding to the pixel point at the first intersection point, determine the color parameters carried by the light connecting the first intersection point and the starting point of the second light propagation path corresponding to the pixel point as the coloring result after the collision, wherein the previous intersection point of the first intersection point is the starting point of the second light propagation path corresponding to the pixel point; Wherein, the color parameter determination module further includes: The first path determination sub-module is used to determine, for each pixel point, a path with the coordinate point corresponding to the pixel point in the camera coordinate system of the virtual camera as the starting point and the direction from the coordinate point to the origin of the camera coordinate system as the propagation direction, as the first light propagation path corresponding to the pixel point; The second path determination sub-module is used to map the first light propagation path corresponding to each pixel point into the world coordinate system by using the conversion relationship between the camera coordinate system and the world coordinate system corresponding to the three-dimensional simulation scene, to obtain the second light propagation path corresponding to each pixel point.

Images