Camera distortion simulation method, device and electronic equipment

By deploying multiple virtual perspective cameras in 3D rendering, using the field angle information of distortion-free images and distorted images to generate distorted simulated images, the problem of blurred image central area in real camera distortion simulation is solved, and a clearer simulation effect is achieved .

CN118840520BActive Publication Date: 2024-12-06HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411330360.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-23
Publication Date
2024-12-06
Estimated Expiration
2044-09-23

AI Technical Summary

Technical Problem

When simulating real camera distortion, the image distortions collected by wide-angle lenses and fish-eye lenses are more obvious, resulting in too blurred the central area of ​​the image, affecting the distortion simulation effect.

Method used

By deploying multiple virtual perspective cameras in the 3D rendering field, rendering distortion-free images, and sampling and generating distortion-simulated images from distortion-free images based on the field angle of one distortion image and the virtual perspective camera to avoid direct distortion-free images The image is processed.

Benefits of technology

It effectively avoids blurring of image areas during distortion simulation, ensures the clarity and accuracy of the simulation effect, and eliminates the problem of blurring of the center area caused by existing methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118840520B_ABST
    Figure CN118840520B_ABST
Patent Text Reader

Abstract

The present application provides a camera distortion simulation method, device and electronic device. In the present application, by using N virtual perspective cameras in the 3D rendering field to render undistorted images for the same 3D scene at the same time, and based on a distorted image and the field of view of the N virtual perspective cameras, simulated pixel points in the distorted simulation image are sampled from the N undistorted images, rather than directly processing the existing undistorted images. Compared with the existing conventional image post-processing methods, this can avoid blurring of the image area with obvious distortion during the camera distortion simulation process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to image processing, and in particular to a camera distortion simulation method, device and electronic device. Background Art

[0002] The images captured by real cameras are often distorted due to the inconsistency between the magnification of the central area of ​​the lens and the edge area. This distortion is more obvious in images captured by wide-angle lenses or fisheye lenses.

[0003] At present, the common method for simulating real camera distortion is the image post-processing method, specifically: a distortion-free image taken by a normal perspective camera is distorted to obtain a distorted image for simulation. However, when this method simulates images with obvious (severe) distortion, such as those collected by the wide-angle lens and fisheye lens mentioned above, if the distortion-free image is distorted so that the central area of ​​the image has obvious (severe) distortion, it may cause the central area of ​​the entire image to be too blurred, affecting the image distortion simulation effect. Summary of the invention

[0004] The present application provides a camera distortion simulation method, device and electronic device to avoid blurring of image areas with obvious distortion during the camera distortion simulation process.

[0005] The present application provides a camera distortion simulation method, the method comprising:

[0006] Obtaining N distortion-free images; the N distortion-free images include distortion-free images respectively rendered by N virtual perspective cameras at the same time for the same 3D scene; N virtual perspective cameras are deployed in the 3D scene, N is greater than 1, the N virtual perspective cameras are deployed at the same position and the orientations of the lenses of the N virtual perspective cameras meet the set similarity requirements; the N virtual perspective cameras have different field of view angles;

[0007] Obtain a distorted image;

[0008] Based on the N undistorted images and the distorted images, a distorted simulated image is generated; each simulated pixel point in the distorted simulated image is based on a mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and the field of view angles of the N virtual perspective cameras are sampled from the N undistorted images.

[0009] The embodiment of the present application also provides a camera distortion simulation device, which includes:

[0010] An obtaining unit is used to obtain N distortion-free images; the N distortion-free images include distortion-free images respectively rendered by N virtual perspective cameras for the same 3D scene at the same time; N virtual perspective cameras are deployed in the 3D scene, N is greater than 1, the N virtual perspective cameras are deployed at the same position and the orientations of the lenses of the N virtual perspective cameras meet the set similarity requirements; the N virtual perspective cameras have different field of view angles; and,

[0011] Obtain a distorted image;

[0012] A distortion simulation unit is used to generate a distortion simulation image based on the N undistorted images and the distorted image; each simulated pixel point in the distortion simulation image is based on a mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and the field of view angles of the N virtual perspective cameras are sampled from the N undistorted images.

[0013] The embodiment of the present application also provides an electronic device. The electronic device includes: a processor and a machine-readable storage medium;

[0014] The machine-readable storage medium stores machine-executable instructions that can be executed by the processor;

[0015] The processor is used to execute machine executable instructions to implement the steps of the above disclosed method.

[0016] It can be seen from the above technical solutions that in the present application, by using N virtual perspective cameras in the 3D rendering field to render undistorted images for the same 3D scene at the same time, and based on a distorted image and the field of view of the N virtual perspective cameras, simulated pixel points in the distorted simulated image are sampled from the N undistorted images, rather than directly processing the existing undistorted images. Compared with the existing conventional image post-processing methods, this can avoid blurring of image areas with obvious distortion during camera distortion simulation;

[0017] Furthermore, in this embodiment, based on a distorted image and the field of view of the above-mentioned N virtual perspective cameras, simulated pixel points in the distorted simulation image are sampled from N undistorted images, which is equivalent to splicing and fusing the pixel points in N undistorted images in the real-time rendering process, which can completely eliminate the blurring of the central area of ​​the distorted simulation image caused by the existing post-processing of a ready-made image. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] Figure 1A flow chart of the method provided in the embodiment of the present application;

[0020] Figure 2 A schematic diagram of a distortion-free image provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of a distorted image provided in an embodiment of the present application;

[0022] Figure 4 A flow chart of distortion simulation image generation provided in an embodiment of the present application;

[0023] Figure 5 A hybrid mask map provided by an embodiment of the present application;

[0024] Figure 6 , Figure 7 A simulation effect diagram provided for an embodiment of the present application;

[0025] Figure 8 A structural diagram of a device provided in an embodiment of the present application;

[0026] Fig. 9 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings.

[0028] See also Figure 1 , Figure 1 This is a flow chart of a method provided in an embodiment of the present application. The method is applied to electronic devices such as servers running 3D scenes, etc., and is not specifically limited in this embodiment. The process can be applied to camera distortion simulation in the field of real-time rendering.

[0029] As an embodiment, this embodiment is applied to a 3D scene. In the 3D scene, this embodiment will create N virtual perspective cameras, where N is greater than 1. For example, N is 2.

[0030] In one embodiment, the N virtual perspective cameras are deployed at the same location and the orientations of the lenses of the N virtual perspective cameras meet a set similarity requirement (such as consistent orientation, etc.).

[0031] In one embodiment, the N virtual perspective cameras have different field of view angles.

[0032] Based on the above description, Figure 1 As shown, the process may include the following steps:

[0033] Step 101, obtaining N distortion-free images.

[0034] In this embodiment, the N undistorted images include undistorted images rendered by the N virtual perspective cameras at the same time for the same 3D scene image. Figure 2 Taking N as 2 as an example, two undistorted images are shown. As described above, the N virtual perspective cameras have different field of view angles. When N is 2, the field of view angles of the two virtual perspective cameras are also different, for example, one of the field of view angles is 90° and the other field of view angle is 60°. Since the field of view angles of the above-mentioned N virtual perspective cameras are different, the field of view angles of the above-mentioned N virtual perspective cameras will definitely be different in size. For ease of description, when N is 2, the undistorted image rendered by one of the two virtual perspective cameras with a larger field of view angle, such as 90°, can be called a wide field of view angle image. Correspondingly, the undistorted image rendered by the other virtual perspective camera can be called a narrow field of view angle image. Figure 2 An example of a wide-viewing-angle image (referred to as wide-viewing-angle image) and a narrow-viewing-angle image (referred to as narrow-viewing-angle image) is shown.

[0035] It should be noted that, in this embodiment, the image resolution of the undistorted images rendered by the N virtual perspective cameras is the same. Based on the different field of view angles of the N virtual perspective cameras, relatively speaking, the undistorted image rendered by the virtual perspective camera with a larger field of view has a wider scene range, which may include all or part of the undistorted image rendered by the virtual perspective camera with a smaller field of view. For example Figure 2 The wide FOV image shown contains the content of the entire narrow FOV image.

[0036] However, at the same image resolution, the undistorted image rendered by a virtual perspective camera with a smaller field of view has richer scene details than that rendered by a virtual perspective camera with a larger field of view.

[0037] Based on the above description, when generating the distortion simulation image later, the distortion-free image rendered by the virtual perspective camera at each field of view angle will be sampled adaptively. For details, see step 103, which will not be described here.

[0038] Step 102: Obtain a distorted image.

[0039] As an embodiment, the distorted image is generated by transforming a normal undistorted image according to a set distortion algorithm such as an OpenCV algorithm. Figure 3 A distorted image is shown as an example.

[0040] Optionally, the image resolution of the distorted image is the same as the image resolution of the N undistorted images.

[0041] Alternatively, the image resolution of the distorted image is in a multiple relationship with the image resolution of the N undistorted images (but the aspect ratio of the distorted image here is the same as the aspect ratio of the N undistorted images).

[0042] Step 103, based on the N undistorted images and the distorted images, a distorted simulated image is generated, wherein each simulated pixel in the distorted simulated image is based on a mapped pixel in the distorted image that has a mapping relationship with the simulated pixel, and the field of view of the N virtual perspective cameras is sampled from the N undistorted images.

[0043] As described above, the characteristics of the undistorted image rendered by the virtual perspective camera with a large field of view and the undistorted image rendered by the virtual perspective camera with a small field of view, when generating the distorted simulation image, this embodiment triggers the generation of the distorted simulation image from an angle with higher pixel accuracy and clearer distortion effect as much as possible. For example, for the overlapping area in N undistorted images, each simulated pixel point in the simulated area having a mapping relationship with the overlapping area in the distorted simulation image is sampled from the undistorted image rendered by the virtual perspective camera with the smallest field of view, for example, specifically, the pixel information of the mapped pixel point having a mapping relationship with the simulated pixel point is sampled from the undistorted image rendered by the virtual perspective camera with the smallest field of view as the pixel information of the simulated pixel point.

[0044] That is, when there is an overlapping area in N undistorted images, each simulated pixel point in the simulated area in the distorted simulated image that has a mapping relationship with the overlapping area is sampled from the undistorted image rendered by the virtual perspective camera with the smallest field of view. Based on the characteristics of the undistorted image rendered by the virtual perspective camera with a small field of view, such as richer scene details, the accuracy of the simulated pixel points in the simulated area can be made higher, and the distorted effect can be made clearer. The following describes step 103 by way of example, which will not be described here.

[0045] So far, completed Figure 1 The process shown.

[0046] pass Figure 1 It can be seen from the process shown that this embodiment uses the non-distorted images rendered by N virtual perspective cameras in the 3D rendering field at the same time for the same 3D scene screen, and samples simulated pixel points in the distorted simulation image from the N non-distorted images based on a distorted image and the field of view of the N virtual perspective cameras, rather than directly processing the existing non-distorted images. Compared with the existing conventional image post-processing method, this can avoid blurring of the image area with obvious distortion during the camera distortion simulation process;

[0047] Furthermore, in this embodiment, based on a distorted image and the field of view of the above-mentioned N virtual perspective cameras, simulated pixel points in the distorted simulation image are sampled from N undistorted images, which is equivalent to splicing and fusing the pixel points in N undistorted images in the real-time rendering process, which can completely eliminate the blurring of the central area of ​​the distorted simulation image caused by the existing post-processing of a ready-made image.

[0048] The above step 103 is described below:

[0049] As an embodiment, in the above step 103, based on the N undistorted images and the distorted images, generating the distorted simulated image includes: performing the following step a for each simulated pixel point in the simulated image template to obtain the distorted simulated image:

[0050] Step a: Based on the mapped pixel points in the distorted image that have a mapping relationship with the simulated pixel points and the field of view of N virtual perspective cameras, a sampling image is determined from N undistorted images; based on the pixel parameters of the mapped pixel points in the sampling image that have a mapping relationship with the simulated pixel points, the pixel parameters of the simulated pixel points in the simulated image template are corrected.

[0051] As an embodiment, the pixel parameter here is, for example, a pixel value, etc., which is not specifically limited in this embodiment.

[0052] To facilitate understanding of step 103, Figure 4 The flowchart for generating the distortion simulation image provided in the embodiment of the present application is shown by way of example.

[0053] See also Figure 4 , Figure 4 This is a flow chart of the distortion simulation image generation provided in the embodiment of the present application. Figure 4 As shown, the process may include the following steps:

[0054] Step 401, traverse each untraversed simulated pixel point in the simulated image template in order, and use the traversed simulated pixel point as the current pixel point.

[0055] For example, in this embodiment, each untraversed simulated pixel point in the simulated image template is first traversed in order from front to back and from left to right, and the traversed simulated pixel point is used as the current pixel point.

[0056] Step 402 : determining a sampled image from the N undistorted images based on the mapped pixel points in the distorted image that have a mapping relationship with the current simulated pixel point and the field of view of the N virtual perspective cameras.

[0057] As an embodiment, the pixel information of any pixel in the distorted image includes a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute. Here, for example, the first specified attribute corresponds to a blue channel, and the second specified attribute corresponds to an alpha channel. Correspondingly, the first attribute value corresponding to the first specified attribute is a blue channel value; the second attribute value corresponding to the second specified attribute is an alpha channel value.

[0058] Based on this, this embodiment can determine the sampling image from the above-mentioned N undistorted images with the help of the coordinate information of the mapping pixel point in the distorted image that has a mapping relationship with the simulated pixel point, the first attribute value corresponding to the first specified attribute and the second attribute value corresponding to the second specified attribute in the pixel information of the mapping pixel point, and the field of view of the N virtual perspective cameras.

[0059] For example, based on the coordinate information of the mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and the first attribute value corresponding to the first specified attribute and the second attribute value corresponding to the second specified attribute in the pixel information of the mapped pixel point, a reference decision coefficient is determined. The reference decision coefficient here is used to determine the target sampling decision coefficient for selecting the sampled image, as described below. Then, based on the reference decision coefficient and the field of view of the N virtual perspective cameras, the target sampling decision coefficient is determined; finally, based on the target sampling decision coefficient, the sampled image is determined from the N undistorted images.

[0060] As an embodiment, the reference decision coefficient is determined based on the coordinate information of a mapping pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and a vector consisting of a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute in the pixel information of the mapping pixel point.

[0061] For example, the reference decision coefficient is expressed by the following formula: .

[0062] in, represents the reference decision coefficient, represents the coordinate information of the mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, A vector representing a first attribute value corresponding to a first specified attribute in the pixel information of the mapped pixel, such as the above-mentioned blue channel value, and a second attribute value corresponding to a second specified attribute, such as the above-mentioned opaque channel value.

[0063] As an embodiment, there are many ways to implement the above-mentioned determination of the target sampling decision coefficient based on the reference decision coefficient and the field of view angles of N virtual perspective cameras, such as: determining the trigonometric function value corresponding to the specified trigonometric function of the field of view angle of each virtual perspective camera; determining the target sampling decision coefficient based on the reference decision coefficient and each sampling decision sub-coefficient; any sampling decision sub-coefficient is obtained by performing a specified operation on the trigonometric function values ​​corresponding to the field of view angles of two virtual perspective cameras.

[0064] Optionally, the trigonometric function value corresponding to the specified trigonometric function for determining the field of view angle of each virtual perspective camera may be, for example, a tangent value for determining the field of view angle of each virtual perspective camera.

[0065] Optionally, the above-mentioned determination of the target sampling decision coefficient based on the reference decision coefficient and each sampling decision sub-coefficient may be, for example, as follows: first input each sampling decision sub-coefficient and the reference decision coefficient into a specified algorithm to determine the target sampling decision coefficient. The specified algorithm may be set according to actual needs, and the present embodiment does not specifically limit it. For example, each sampling decision sub-coefficient is multiplied by its corresponding proportional weight to obtain a first result, the first result obtained is added to the reference decision coefficient to obtain a second result, the second result is divided by the sum of each sampling decision sub-coefficient, and the final result is determined as the target sampling decision coefficient. Here, the proportional weight corresponding to any sampling decision sub-coefficient may be the same, for example, both may be 1 / N; or, the proportional weights corresponding to at least two sampling decision sub-coefficients are different, for example, they may be set according to actual needs, and the present embodiment does not specifically limit it.

[0066] For ease of understanding, the above-mentioned specified algorithm is illustrated below by taking N as 2 as an example.

[0067] For example, optionally, the target sampling decision coefficient may be specifically expressed by the following specified algorithm:

[0068] ;

[0069] in, is the horizontal field of view of the virtual perspective camera with the smallest field of view, It is the horizontal field of view of the virtual perspective camera with the largest field of view. Represents the target sampling decision coefficient. represents the sampling decision sub-coefficient, represents the above reference decision coefficient.

[0070] It should be noted that the above description is only an example description using N as 2. The method of specifying the algorithm when N is other values ​​is the same as the description of the specified algorithm above, which will not be repeated here.

[0071] As an embodiment, based on the target sampling decision coefficient, a sampling image is determined from N undistorted images, for example: from the decision coefficient range currently corresponding to the field of view angle of each virtual perspective camera that has been obtained, the target decision coefficient range in which the target sampling decision coefficient is located is determined; and the undistorted image rendered by the virtual perspective camera that currently has a field of view angle corresponding to the target decision coefficient range is determined as the sampling image.

[0072] For example, taking N as 2, the sampling image can be determined from N undistorted images based on the target sampling decision coefficient: if the target sampling decision coefficient is within the decision coefficient range corresponding to the narrow field of view, such as the range of 0 to 1, then the undistorted image rendered by the virtual perspective camera with the smallest field of view is determined as the above sampling image; otherwise, the undistorted image rendered by the virtual perspective camera with the largest field of view is determined as the above sampling image.

[0073] Step 403, based on the pixel parameters of the mapped pixel points in the sampled image that have a mapping relationship with the current simulated pixel point, correct the pixel parameters of the current simulated pixel point in the simulated image template; if the current simulated pixel point is the last simulated pixel point traversed in the simulated image template, the simulated image template at this time is determined as the above-mentioned distorted simulated image, otherwise, return to the step of traversing each untraversed simulated pixel point in the simulated image template in sequence.

[0074] As an embodiment, the above-mentioned correction of the pixel parameters of the simulated pixel point in the simulated image template based on the pixel parameters of the mapped pixel point in the sampled image that has a mapping relationship with the simulated pixel point can specifically be: modifying the pixel value of the simulated pixel point in the simulated image template to the pixel value of the mapped pixel point in the sampled image that has a mapping relationship with the simulated pixel point.

[0075] So far, completed Figure 4 The process shown.

[0076] pass Figure 4 The process shown finally realizes the above-mentioned generation of a distorted simulation image based on N undistorted images and distorted images.

[0077] In this embodiment, in the distorted simulated image, in order to avoid gaps caused by stitching, for the area to be adjusted starting from the boundary line and along the direction toward the center area of ​​the image, the simulated pixel points in the area to be adjusted can be adjusted based on the undistorted image rendered by each virtual perspective camera to avoid the above-mentioned gap. Here, the boundary line is determined based on adjacent boundary simulated pixel points in the distorted simulated image. The adjacent boundary simulated pixel points are sampled from the undistorted images rendered by two different virtual perspective cameras.

[0078] For example, taking N as 2, the boundary line is a boundary line between pixel points in the undistorted image with a wide field of view and pixel points in the undistorted image with a narrow field of view.

[0079] As an embodiment, the above-mentioned adjustment of the simulated pixel points in the area to be adjusted may be, for example: for each simulated pixel point in the area to be adjusted, a mapping pixel point mapped by the simulated pixel point is selected from the undistorted image rendered by each virtual perspective camera, the pixel parameters of each mapping pixel point, such as pixel values, are fused, and the fusion result is determined as the pixel parameter, such as pixel value, of the simulated pixel point.

[0080] Still taking N as 2 as an example, the above-mentioned area to be adjusted is the area from the boundary line to the center of the image in the distorted simulation image. Of course, if N is greater than 2, the above-mentioned area to be adjusted is the area from the boundary line to another boundary line in the direction toward the center of the image in the distorted simulation image.

[0081] As an embodiment, for each simulated pixel point in the above-mentioned area to be adjusted, a mapping pixel point mapped by the simulated pixel point can be selected from the undistorted images rendered by each virtual perspective camera, and the pixel parameters, such as pixel values, of each mapping pixel point are linearly fused, and the fusion result is determined as the pixel parameter, such as pixel value, of the simulated pixel point. This step can be performed as follows Figure 5 The hybrid mask map shown in . Figure 5 In the mixed mask image shown, pure white indicates that the pixels in the narrow field of view image are fully used, black indicates that the pixels in the wide field of view image are fully used, and the gray transition part at the junction is the fusion result after linear fusion of the pixels at the same position in the narrow field of view image and the wide field of view image.

[0082] The above describes the method provided in the embodiment of the present application. In order to make the effect of the above method more obvious, Figure 6 , Figure 7 Examples are shown of clear images after distortion simulation of narrow viewing angle images, and clear images after distortion simulation of wide viewing angle images and narrow viewing angle images. By combining narrow viewing angle images or wide viewing angle images and narrow viewing angle images for distortion simulation, it is possible to avoid blurring of image areas with obvious distortion during camera distortion simulation, ensuring clarity of the entire image.

[0083] The following is a description of the device provided in the embodiment of the present application:

[0084] See also Figure 8 , Figure 8 This is a diagram of the device structure provided in the embodiment of the present application. Figure 8 As shown, the device may include:

[0085] An obtaining unit is used to obtain N distortion-free images; the N distortion-free images include distortion-free images respectively rendered by N virtual perspective cameras for the same 3D scene at the same time; N virtual perspective cameras are deployed in the 3D scene, N is greater than 1, the N virtual perspective cameras are deployed at the same position and the orientations of the lenses of the N virtual perspective cameras meet the set similarity requirements; the N virtual perspective cameras have different field of view angles; and,

[0086] Obtain a distorted image;

[0087] A distortion simulation unit is used to generate a distortion simulation image based on the N undistorted images and the distorted image; each simulated pixel point in the distortion simulation image is based on a mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and the field of view angles of the N virtual perspective cameras are sampled from the N undistorted images.

[0088] As an embodiment, the N distortion-free images have the same resolution;

[0089] The distorted image is generated according to a set distortion algorithm;

[0090] The image resolution of the distorted image is the same as the image resolution of the N undistorted images, or the image resolution of the distorted image is a multiple of the image resolution of the N undistorted images.

[0091] As an embodiment, generating a distortion simulation image based on the N distortion-free images and the distortion image includes:

[0092] The following steps are performed for each simulated pixel point in the simulated image template to obtain a distorted simulated image: pixel information of any pixel point in the distorted image includes a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute;

[0093] The generating a distortion simulation image based on the N distortion-free images and the distortion image comprises:

[0094] For each simulated pixel point in the simulated image template, a reference decision coefficient is determined based on coordinate information of a mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute in the pixel information of the mapped pixel point; the reference decision coefficient is used to determine a target sampling decision coefficient for selecting a sampling image;

[0095] Determine a target sampling decision coefficient based on the reference decision coefficient and the field of view angles of N virtual perspective cameras;

[0096] Based on the target sampling decision coefficient, determining a sampling image from the N undistorted images;

[0097] The pixel parameters of the simulated pixel point in the simulated image template are corrected based on the pixel parameters of the mapped pixel point in the sampled image that has a mapping relationship with the simulated pixel point.

[0098] As an embodiment, the first specified attribute corresponds to a blue channel, and the first attribute value corresponding to the first specified attribute is a blue channel value;

[0099] The second specified attribute corresponds to an opaque alpha channel, and the second attribute value corresponding to the second specified attribute is an opaque channel value;

[0100] The reference decision coefficient is determined based on the coordinate information of a mapping pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and a vector consisting of a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute in the pixel information of the mapping pixel point.

[0101] As an embodiment, the reference decision coefficient is expressed by the following formula: ;

[0102] in, represents the reference decision coefficient, represents the coordinate information of the mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, A vector consisting of a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute in the pixel information of the mapped pixel.

[0103] As an embodiment, determining the target sampling decision coefficient based on the reference decision coefficient and the field of view angles of N virtual perspective cameras includes:

[0104] Determine trigonometric function values ​​corresponding to the specified trigonometric functions of the field of view angles of each virtual perspective camera;

[0105] Based on the reference decision coefficient and each sampling decision sub-coefficient, a target sampling decision coefficient is determined; any sampling decision sub-coefficient is obtained by performing a specified operation on the trigonometric function values ​​corresponding to the field of view angles of any two virtual perspective cameras.

[0106] As an embodiment, determining a sampling image from the N distortion-free images based on the target sampling decision coefficient includes:

[0107] The target decision coefficient range in which the target sampling decision coefficient is located is determined from the decision coefficient range currently corresponding to the field of view angle of each virtual perspective camera that has been obtained; and the distortion-free image rendered by the virtual perspective camera that currently has a field of view angle corresponding to the target decision coefficient range is determined as the sampling image.

[0108] As an embodiment, the distortion simulation unit further determines the area to be adjusted in the distortion simulation image starting from the boundary line and along the direction toward the center area of ​​the image; the boundary line is determined based on adjacent boundary simulation pixel points in the distortion simulation image, and the adjacent boundary simulation pixel points are sampled from the undistorted images rendered by two different virtual perspective cameras respectively;

[0109] For each simulated pixel point in the area to be adjusted, a mapping pixel point mapped by the simulated pixel point is selected from the undistorted images rendered by each virtual perspective camera, pixel parameters of each mapping pixel point are fused, and the fusion result is determined as the pixel parameter of the simulated pixel point.

[0110] As an embodiment, if N is 2, the area to be adjusted is the area from the boundary line to the center of the image in the distorted simulation image.

[0111] So far, completed Figure 8 Structural description of the device shown.

[0112] The present application also provides Figure 8 The hardware structure of the device shown. Fig. 9 , Fig. 9 This is a structural diagram of an electronic device provided in an embodiment of the present application. Fig. 9 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0113] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0114] Exemplarily, the above-mentioned machine-readable storage medium can be any electronic, magnetic, optical or other physical storage device, which can contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium can be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as optical disk, DVD, etc.), or similar storage medium, or a combination thereof.

[0115] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A camera distortion simulation method, It is characterized in that The method includes: Obtaining N distortion-free images; the N distortion-free images include distortion-free images respectively rendered by N virtual perspective cameras at the same time for the same 3D scene; N virtual perspective cameras are deployed in the 3D scene, N is greater than 1, the N virtual perspective cameras are deployed at the same position and the orientations of the lenses of the N virtual perspective cameras meet the set similarity requirements; the N virtual perspective cameras have different field of view angles; Obtain a distorted image; Based on the N undistorted images and the distorted images, a distorted simulated image is generated; each simulated pixel point in the distorted simulated image is sampled from the N undistorted images based on a mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and the field of view angles of the N virtual perspective cameras; wherein the pixel information of any pixel point in the distorted image includes a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute; the generating of the distorted simulated image based on the N undistorted images and the distorted images includes: for each simulated pixel point in the simulated image template, based on a mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, A reference decision coefficient is determined based on coordinate information of a mapped pixel point having a mapping relationship, and a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute in the pixel information of the mapped pixel point; the reference decision coefficient is used to determine a target sampling decision coefficient for selecting a sampling image; based on the reference decision coefficient and the field of view of N virtual perspective cameras, the target sampling decision coefficient is determined; based on the target sampling decision coefficient, a sampling image is determined from the N undistorted images; and based on the pixel parameters of the mapped pixel point in the sampling image that has a mapping relationship with the simulation pixel point, the pixel parameters of the simulation pixel point in the simulation image template are corrected.

2. The method according to claim 1, It is characterized in that The N distortion-free images have the same resolution; The distorted image is generated according to a set distortion algorithm; The image resolution of the distorted image is the same as the image resolution of the N undistorted images, or the image resolution of the distorted image is a multiple of the image resolution of the N undistorted images.

3. The method according to claim 1, It is characterized in that The first specified attribute corresponds to a blue channel, and the first attribute value corresponding to the first specified attribute is a blue channel value; The second specified attribute corresponds to an opaque alpha channel, and the second attribute value corresponding to the second specified attribute is an opaque channel value; The reference decision coefficient is determined based on the coordinate information of a mapping pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and a vector consisting of a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute in the pixel information of the mapping pixel point.

4. The method according to claim 1 or 3, It is characterized in that The reference decision coefficient is expressed by the following formula: ; in, represents the reference decision coefficient, represents the coordinate information of the mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, A vector consisting of a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute in the pixel information of the mapped pixel.

5. The method according to claim 1, It is characterized in that The step of determining the target sampling decision coefficient based on the reference decision coefficient and the field of view angles of the N virtual perspective cameras includes: Determine trigonometric function values ​​corresponding to the specified trigonometric functions of the field of view angles of each virtual perspective camera; Based on the reference decision coefficient and each sampling decision sub-coefficient, a target sampling decision coefficient is determined; any sampling decision sub-coefficient is obtained by performing a specified operation on the trigonometric function values ​​corresponding to the field of view angles of any two virtual perspective cameras.

6. The method according to claim 1, It is characterized in that The step of determining a sampling image from the N distortion-free images based on the target sampling decision coefficient comprises: The target decision coefficient range in which the target sampling decision coefficient is located is determined from the decision coefficient range currently corresponding to the field of view angle of each virtual perspective camera that has been obtained; and the distortion-free image rendered by the virtual perspective camera that currently has a field of view angle corresponding to the target decision coefficient range is determined as the sampling image.

7. The method according to claim 1, It is characterized in that The method further comprises: Determining an area to be adjusted in the distorted simulated image starting from a boundary line and along a direction toward a central area of ​​the image; the boundary line is determined based on adjacent boundary simulated pixel points in the distorted simulated image, and the adjacent boundary simulated pixel points are sampled from undistorted images rendered by two different virtual perspective cameras; For each simulated pixel point in the area to be adjusted, a mapping pixel point mapped by the simulated pixel point is selected from the undistorted images rendered by each virtual perspective camera, pixel parameters of each mapping pixel point are fused, and the fusion result is determined as the pixel parameter of the simulated pixel point.

8. A camera distortion simulation device, It is characterized in that The device includes: An obtaining unit is used to obtain N distortion-free images; the N distortion-free images include distortion-free images respectively rendered by N virtual perspective cameras for the same 3D scene at the same time; N virtual perspective cameras are deployed in the 3D scene, N is greater than 1, the N virtual perspective cameras are deployed at the same position and the orientations of the lenses of the N virtual perspective cameras meet the set similarity requirements; the N virtual perspective cameras have different field of view angles; and, Obtain a distorted image; A distortion simulation unit is used to generate a distortion simulation image based on the N undistorted images and the distorted image; each simulated pixel point in the distorted simulation image is sampled from the N undistorted images based on a mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and the field of view angles of the N virtual perspective cameras; wherein the pixel information of any pixel point in the distorted image includes a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute; the generating of the distortion simulation image based on the N undistorted images and the distorted image includes: for each simulated pixel point in the simulation image template, based on the pixel point in the distorted image that has a mapping relationship with the simulated pixel point, A reference decision coefficient is determined based on coordinate information of a mapped pixel point with a mapping relationship between the pixel points, and a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute in the pixel information of the mapped pixel point; the reference decision coefficient is used to determine a target sampling decision coefficient for selecting a sampling image; based on the reference decision coefficient and the field of view of N virtual perspective cameras, the target sampling decision coefficient is determined; based on the target sampling decision coefficient, a sampling image is determined from the N undistorted images; and based on the pixel parameters of the mapped pixel point in the sampling image that has a mapping relationship with the simulated pixel point, the pixel parameters of the simulated pixel point in the simulated image template are corrected.

9. An electronic device, It is characterized in that The electronic device includes: a processor and a machine-readable storage medium; The machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is used to execute machine executable instructions to implement the method steps of any one of claims 1-7.

Citation Information

Patent Citations

  • Simulating multi-camera imaging systems

    CN107407554A

  • Image processing method, apparatus, apparatus and storage medium based on virtual reality

    CN109461213A

  • Virtual distortion image generation method and device

    CN113989467A

  • Virtual simulation method and device for camera distortion

    CN114332246A

  • Measurement method and device based on fringe projection measurement model and control equipment

    CN115200509A