Image simulation method and apparatus

Abstract

The application discloses an image simulation method and device. The method comprises the following steps: acquiring a first image height corresponding to a target pixel point in a simulation screen; inputting the first image height into a target distortion model; and outputting a second image height corresponding to the target pixel point in a target image, wherein the target image is obtained by collecting images of multiple directions through a first camera and projecting the images onto multiple surfaces of a cubic box; a value of a target parameter in the target distortion model is determined according to calibration distortion data of the first camera; a first pixel coordinate corresponding to the target pixel point in the target image is determined according to the second image height; a target color RGB value at the first pixel coordinate in the target image is determined; an RGB value of the target pixel point in the simulation screen is set as a target RGB value; and a simulation image is obtained. In this way, performance consumption caused by establishment of multiple cameras can be greatly reduced, the simulation accuracy is improved, a better simulation effect can be achieved, and the workload of staff is reduced.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to an image simulation method and device. BACKGROUND

[0002] With the development and progress of science and technology, people have higher and higher requirements for image simulation technology.

[0003] In the related art, when image simulation is performed, the mapping relationship between the original pixel points and the distorted pixel points is calculated by using fixed models such as equidistance projection, equal solid angle projection, stereoscopic projection, and orthogonal projection after the images captured by multiple cameras are spliced, so as to complete the image simulation.

[0004] However, the performance consumption caused by the image capturing by multiple cameras is large, and when the model is selected, the staff needs to test in sequence according to the used scene to select the most suitable model from the above models, which greatly increases the workload of the staff. In addition, since the above models are fixed models without parameters, the effect of each model is fixed, so the above models may not achieve good simulation effect. SUMMARY

[0005] The embodiments of the present application provide an image simulation method and device, which can greatly reduce the performance consumption caused by the establishment of multiple cameras, avoid the instability caused by different fixed models, improve the accuracy of simulation, achieve better simulation effect, and do not need the staff to select the model by testing, thereby reducing the workload of the staff.

[0006] In a first aspect, the embodiments of the present application provide an image simulation method, which comprises:

[0007] obtaining a first image height corresponding to a target pixel point in a simulation screen,

[0008] inputting the first image height into a target distortion model to output a second image height corresponding to the target pixel point in a target image, the target image being obtained by projecting images of multiple directions captured by a first camera onto multiple surfaces of a cube box, and a value of a target parameter in the target distortion model being determined according to calibration distortion data of the first camera,

[0009] determining a first pixel coordinate corresponding to the target pixel point in the target image according to the second image height,

[0010] determining a target color RGB value at the first pixel coordinate in the target image,

[0011] setting the RGB value of the target pixel point in the simulation screen as the target RGB value to obtain a simulation image.

[0012] In a second aspect, the embodiments of the present application provide an image simulation device, the device comprising:

[0013] an acquisition module configured to acquire a first image height corresponding to a target pixel point in a simulation screen,

[0014] an input module configured to input the first image height into a target distortion model, and output a second image height corresponding to the target pixel point in a target image, the target image being obtained by projecting images of a plurality of orientations captured by a first camera onto a plurality of surfaces of a cube box, a value of a target parameter in the target distortion model being determined according to calibration distortion data of the first camera,

[0015] a first determination module configured to determine a first pixel coordinate of the target pixel point in the target image according to the second image height,

[0016] a second determination module configured to determine a target color RGB value at the first pixel coordinate in the target image,

[0017] a setting module configured to set the RGB value of the target pixel point in the simulation screen as a target RGB value, and obtain a simulation image.

[0018] In a third aspect, the embodiments of the present application provide an electronic device, the device comprising: a processor and a memory storing computer program instructions,

[0019] the processor, when executing the computer program instructions, implements the image simulation method shown in any one of the embodiments of the first aspect.

[0020] In a fourth aspect, the embodiments of the present application provide a computer storage medium, the computer storage medium storing computer program instructions, the computer program instructions, when executed by a processor, implementing the image simulation method shown in any one of the embodiments of the first aspect.

[0021] In a fifth aspect, the embodiments of the present application provide a computer program product, instructions in the computer program product being executed by a processor of an electronic device, so that the electronic device executes the image simulation method shown in any one of the embodiments of the first aspect.

[0022] The image simulation method and apparatus of this application embodiment can obtain the first image height corresponding to the target pixel in the simulation screen, input the first image height into the target distortion model, output the second image height corresponding to the target pixel in the target image, then determine the first pixel coordinate of the target pixel in the target image based on the second image height, determine the target color RGB value at the first pixel coordinate in the target image, and then set the RGB value of the target pixel in the simulation screen as the target RGB value to obtain the simulation image. Since the target image is obtained by capturing images from multiple directions through a first camera and projecting them onto multiple faces of a cube, the performance consumption caused by setting up multiple cameras is greatly reduced. In addition, the values ​​of the target parameters in the target distortion model are determined based on the calibration distortion data of the first camera, thus avoiding the instability caused by different fixed models, improving the accuracy of the simulation, achieving better simulation results, and eliminating the need for staff to select models through testing, reducing the workload of staff. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is one of the flowcharts of an image simulation method provided in one embodiment of this application.

[0025] Figure 2 This is one of the schematic diagrams of an image simulation process provided in one embodiment of this application.

[0026] Figure 3 This is the second flowchart of an image simulation method provided in one embodiment of this application.

[0027] Figure 4 This is a second schematic diagram of an image simulation process provided in one embodiment of this application.

[0028] Figure 5 This is a schematic diagram of the structure of an image simulation device provided in one embodiment of this application.

[0029] Figure 6 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0030] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0031] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0032] Figure 1 The diagram illustrates one of the flowcharts of an image simulation method provided in one embodiment of this application. Specifically, it may be a method for determining the values ​​of target parameters, target focal length, and target distance required in the image simulation method provided in one embodiment of this application.

[0033] like Figure 1 As shown, the method for determining the values ​​of the target parameters, target focal length, and target distance required in the image simulation method provided in this application embodiment may include S110-S190, as follows:

[0034] S110: Project the target world coordinates onto a normalized sphere, establish a first coordinate system with the center of the normalized sphere as the first coordinate origin, and determine the first coordinates corresponding to the target world coordinates in the first coordinate system.

[0035] Here, the origin of the first coordinate system can be the center of the normalized sphere. The target world coordinates can be the coordinates of any world coordinate point. Projecting the world coordinate point onto the normalized sphere yields the first coordinate point corresponding to the world coordinate point on the normalized sphere. The first coordinate can be the coordinates of this first coordinate point.

[0036] In some implementations, to determine the first coordinates more accurately, S110 may include:

[0037] divide the coordinates of the X-axis, the Y-axis and the Z-axis of the target world coordinates (X, Y, Z) by obtain the first coordinates of the target world coordinates (X, Y, Z) as wherein,

[0038] Exemplarily, as shown in Figure 2 , the target world coordinates (X, Y, Z) of the world coordinate point x can be projected onto the normalized sphere to obtain the first coordinate point x s . A first coordinate system is established with the center C m of the normalized sphere as the first coordinate origin. The first coordinate of the first coordinate point x s is the coordinate point corresponding to the world coordinate point x in the first coordinate system. The coordinates of the X-axis, the Y-axis and the Z-axis of the target world coordinates (X, Y, Z) can be divided by to obtain the first coordinate of the first coordinate point x s as wherein,

[0039] In this way, the first coordinate corresponding to the target world coordinates in the first coordinate system can be more accurately determined through the above process.

[0040] S120: translating the first coordinate origin along the Z-axis direction of the first coordinate system by a target distance to obtain a second coordinate system, and determining the second coordinates corresponding to the first coordinates in the second coordinate system.

[0041] Here, the first coordinate origin is translated along the Z-axis direction of the first coordinate system by a target distance to obtain a second coordinate system, and then the second coordinates corresponding to the first coordinate point in the second coordinate system can be determined.

[0042] In some embodiments, in order to more accurately determine the second coordinates corresponding to the first coordinate point in the second coordinate system, the above S120 can include:

[0043] adding the target distance ξ to the coordinates of the X-axis, the Y-axis and the Z-axis of the first coordinates to obtain the second coordinates as

[0044] Exemplarily, as shown in Figure 2 , the first coordinate origin C m is translated along the Z-axis direction of the first coordinate system by a target distance ξ to obtain a second coordinate system with C p as the second coordinate origin, and then the first coordinates of the first coordinate point x s are added to the coordinates of the X-axis, the Y-axis and the Z-axis of the target distance ξ, so that the second coordinates of the first coordinate point x s in the second coordinate system can be obtained as ξ).

[0045] Thus, the second coordinate corresponding to the first coordinate in the second coordinate system can be determined more accurately through the above process.

[0046] S130: normalizing the second coordinate to determine a third coordinate corresponding to the second coordinate on a two-dimensional plane.

[0047] Here, projecting the first coordinate point onto a two-dimensional plane with a Z-axis coordinate of 1 can obtain a second coordinate point, and the third coordinate of the second coordinate point can be the coordinate corresponding to the second coordinate of the first coordinate point on the two-dimensional plane.

[0048] In some embodiments, in order to more accurately determine the third coordinate corresponding to the second coordinate on the two-dimensional plane, S130 described above can include:

[0049] dividing the X-axis, Y-axis and Z-axis coordinates of the second coordinate by the Z-axis coordinate of the second coordinate to obtain the third coordinate

[0050] Exemplarily, as shown in Figure 2 , the first coordinate point x s is projected onto a two-dimensional plane with Z = 1 to obtain a second coordinate point X m , and the second coordinate of the first coordinate point x s is divided by the Z-axis coordinate of the second coordinate to obtain the third coordinate of the second coordinate point X m on the two-dimensional plane .

[0051] Thus, the third coordinate corresponding to the second coordinate on the two-dimensional plane can be determined more accurately through the above process.

[0052] S140: projecting the third coordinate onto the imaging plane of the simulated fisheye camera to determine the fourth coordinate corresponding to the third coordinate on the imaging plane corresponding to the simulated fisheye camera.

[0053] The imaging plane of the simulated fisheye camera can be a simulated screen, and the simulated image obtained through the image simulation method provided in the present application can be displayed on the simulated screen. The simulated fisheye camera can be a simulated camera that obtains the simulated image through shooting and processing.

[0054] Here, projecting the second coordinate point onto the imaging plane of the simulated fisheye camera can obtain a second pixel point, and the coordinate of the second pixel point can be the fourth coordinate corresponding to the third coordinate on the imaging plane corresponding to the simulated fisheye camera.

[0055] In some implementations, in order to more accurately determine the fourth coordinate corresponding to the third coordinate on the imaging plane of the simulated fisheye camera, the above-mentioned S140 may include:

[0056] The third coordinate Multiply by the parameter matrix corresponding to the simulated fisheye camera Obtain the fourth coordinate

[0057] Here, the parameter matrix can be based on the target focal length f of the simulated fisheye camera. f It's confirmed.

[0058] For example, such as Figure 2 As shown, the second coordinate point X m Projecting this onto the camera plane (i.e., the imaging plane of a simulated fisheye camera), we can obtain the second pixel X. p Set the second coordinate point X m The third coordinate Multiply by the parameter matrix corresponding to the simulated fisheye camera The second pixel X can be obtained. p The fourth coordinate

[0059] Thus, the above process can more accurately determine the fourth coordinate corresponding to the third coordinate on the imaging plane of the simulated fisheye camera.

[0060] S150: Determine the expression for the target image height of the second pixel point corresponding to the fourth coordinate and the expression for the tangent of the target incident angle.

[0061] Here, the fourth coordinate can be used to represent the target image height and the tangent of the target incident angle of the second pixel, thus obtaining the expressions for the target image height and the tangent of the target incident angle.

[0062] In some implementations, to more accurately determine the target mapping relationship between the target image height and the target incident angle, the expression for the target image height r can be: The expression for the tangent of the target incident angle θ can be:

[0063] Specifically, the second pixel X p The fourth coordinate can be in, Based on this, the target image height can be obtained.

[0064] The target incident angle θ is the angle between the incident ray and the Z-axis. Or it can be written as Z2 • tan 2 θ = X 2 + Y 2 .

[0065] S160: Determine the target mapping relationship between the target image height and the target incident angle according to the expression of the target image height and the tangent expression of the target incident angle.

[0066] Here, according to the expression of the target image height and the tangent expression of the target incident angle, the target mapping relationship between the target image height and the target incident angle can be determined.

[0067] In some embodiments, the target mapping relationship can be

[0068] Specifically, substituting the tangent expression of the target incident angle into the expression of the target image height can obtain Since Therefore, the target mapping relationship between the target image height and the target incident angle can be

[0069] In this way, the target mapping relationship between the target image height and the target incident angle can be more accurately determined through the above process.

[0070] S170: Determine the target distortion model according to the target mapping relationship and the distortion model.

[0071] Here, substituting the above target mapping relationship into the distortion model can obtain the target distortion model.

[0072] In some embodiments, the mathematical formula of the distortion model can be r f = r * (1 + k1r 2 + k2r 4 + ··· + k n r 2n ), where r f may represent a distortion function, n can be a preset constant, and k can be a target parameter. The above S170 can include:

[0073] Substituting the target mapping relationship into the distortion model r f = r * (1 + k1r 2 + k2r 4 + ··· + k n r 2n ) can obtain the formula of the target distortion model as

[0074] Here, the distortion model can be a radtan distortion model, a formula of the radtan distortion model is a Taylor expansion, and the formula has to-be-calibrated parameters that increase with an increase in a power number, and with the increase in the power number, i.e., the increase in the to-be-calibrated parameters, the accuracy of the model is more accurate.

[0075] The target parameter k can be a to-be-calibrated parameter, and the preset constant n can be a power number. The specific value of n can be set according to actual needs. For example, n can be 9.

[0076] In this way, the formula of the target distortion model can be accurately obtained through the above process.

[0077] S180: According to the calibration distortion data and the target distortion model, the value of the target parameter in the distortion model is determined.

[0078] Here, when the parameters of the simulation fisheye camera are calibrated, the calibration can be performed according to the original factory distortion data table of the real camera in the real world. The original factory distortion data table can include an incident angle FOV of light and a corresponding object height and image height of the incident angle. The object height is the distance from the original pixel point to the coordinate origin, and the image height is the distance from the pixel point after distortion to the coordinate origin. The calibration distortion data can be the original factory distortion data table of the real camera. The calibration distortion data can include a mapping relationship between the image height and the incident angle. The image height in the calibration distortion data is substituted into r f The incident angle corresponding to the image height in the calibration distortion data is substituted into θ in the target distortion model, and the value of the target parameter k can be fitted.

[0079] S190: According to the distortion model and the value of the target parameter, the target distortion model is determined.

[0080] Here, the value of the target parameter k is substituted into the distortion model r f =r*(1+k1r 2 +k2r 4 +···+k n r 2n ), and the target distortion model can be obtained.

[0081] In some embodiments, in order to accurately obtain the value of the target focal length and the value of the target distance of the simulation fisheye camera, thereby facilitating more accurate image simulation, after the above S170, the method can further include:

[0082] According to the calibration distortion data and the target distortion model, the value of the target focal length and the value of the target distance of the simulation fisheye camera are determined.

[0083] Here, the image height in the calibration distortion data is substituted into r fThe incident angle corresponding to the image height in the calibration distortion data is substituted into the target distortion model, and the value of the target focal length f of the simulation fisheye camera and the value of the target distance ξ can be obtained in addition to the value of the target parameter k fitted. f

[0084] In this way, the value of the target focal length of the simulation fisheye camera and the value of the target distance can be accurately obtained through the above process, thereby facilitating more accurate image simulation.

[0085] In this embodiment, the value of the target parameter in the distortion model can be determined based on the calibration distortion data of the real camera, thereby obtaining the target distortion model. Using the target distortion model with parameters for image simulation can avoid the instability caused by using a fixed model without parameters, thereby improving the accuracy of simulation and achieving better simulation results. Moreover, there is no need for staff to select a model through testing, thereby reducing the workload of the staff.

[0086] Figure 3 A flowchart of an image simulation method provided by an embodiment of the present application is shown.

[0087] As shown in Figure 3 , the execution subject of the image simulation method can be an image simulation device, and the image simulation method can include S310-S350, which are specifically as follows.

[0088] S310: Obtain a first image height corresponding to a target pixel point in a simulation screen.

[0089] Here, the simulation screen can be a screen for displaying a simulation image. The pixel points on the simulation screen can be sampled to obtain the pixel coordinates of the target pixel point, and then the first image height corresponding to the target pixel point is calculated based on the pixel coordinates of the target pixel point. The target pixel point can be any pixel point on the simulation screen.

[0090] S320: Input the first image height into a target distortion model, and output a second image height corresponding to the target pixel point in a target image.

[0091] The target image can be obtained by capturing images of multiple orientations by the first camera and projecting them onto multiple faces of a cube.

[0092] Here, the second image height of the pixel point corresponding to the target pixel point in the target image can be found from the real target image based on the target distortion model according to the image height of the target pixel point on the simulation screen.

[0093] ​The value of the target parameter in the target distortion model can be determined according to the calibration distortion data of the first camera, and the value of the target parameter can be the value of k determined in the above embodiment. Since the target distortion model is a parameterized model, and the value of the target parameter is determined according to the calibration distortion data of the first camera, the simulation of the image based on the target distortion model can avoid the instability caused by using a fixed model without parameters, thereby improving the accuracy of the simulation and achieving a better simulation effect. Moreover, the staff does not need to select the model by testing, thereby reducing the workload of the staff.

[0094] S330: Determine the first pixel coordinate corresponding to the target pixel point in the target image according to the second image height.

[0095] Here, according to the second image height of the pixel point corresponding to the target pixel point in the target image, the first pixel coordinate of the pixel point corresponding to the target pixel point in the target image can be determined.

[0096] In some embodiments, in order to obtain a more accurate simulation image, the above S330 can include:

[0097] Determine the first pixel coordinate corresponding to the target pixel point in the target image according to the second image height, the value of the target focal length, and the value of the target distance.

[0098] Here, the second image height r f , wherein, The value of the target focal length f f and the value of the target distance ξ have been obtained in the above embodiment, the value of the second image height r f has been obtained through the first image height and the target distortion model, and according to the above embodiment, the fourth coordinate of the second pixel point X p is Therefore, Z = 1. Thus, the values of X, Y, and Z can be determined, thereby obtaining the first pixel coordinate (X, Y, Z) corresponding to the target pixel point in the target image.

[0099] In this way, the value of the target focal length and the value of the target distance are both determined based on the calibration distortion data of the first camera, thereby improving the accuracy of the simulation image.

[0100] S340: Determine the target color RGB value at the first pixel coordinate in the target image.

[0101] Here, according to the first pixel coordinate, the pixel point corresponding to the first pixel coordinate can be found in the target image, and then the target RGB value of the pixel point can be determined.

[0102] S350: set the RGB value of the target pixel point in the simulation screen to the target RGB value, to obtain a simulation image.

[0103] Here, the target RGB value is rendered to the target pixel point in the simulation screen, and a simulation image is obtained.

[0104] Exemplarily, as shown in the figure, the images in six directions of up, down, left, right, front and back are acquired by the first camera, recorded on the six faces of a cubemap established in the game engine Unity, and a target image is obtained. Unity can be a development tool providing solutions in game development, art, architecture, car design, film and television, etc. Figure 4

[0105] Then, the three-dimensional coordinate points of the image data on the six faces of the cubemap are calculated by the shader language used by Unity, wherein the shader is a step of inserting or changing computer graphics resources in the execution of a task through some instructions, and is a specific method of drawing an image. The formula used in the calculation is the target distortion model introduced in the above embodiment.

[0106] Then, the RGB value at the pixel coordinate on the two-dimensional image is collected and rendered to the corresponding pixel coordinate in the simulation screen.

[0107] Thus, the first image height corresponding to the target pixel point in the simulation screen can be acquired, and the first image height is input into the target distortion model to output the second image height corresponding to the target pixel point in the target image. Then, the first pixel coordinate corresponding to the target pixel point in the target image is determined according to the second image height, the target color RGB value at the first pixel coordinate in the target image is determined, and the RGB value of the target pixel point in the simulation screen is set to the target RGB value, so that a simulation image can be obtained. Since the target image is acquired by the first camera in multiple directions and projected onto multiple faces of the cubemap, the performance consumption caused by establishing multiple cameras is greatly reduced. In addition, the value of the target parameter in the target distortion model is determined according to the calibration distortion data of the first camera, so that the instability caused by different fixed models is avoided, the accuracy of the simulation is improved, a better simulation effect can be achieved, and the workload of the staff is reduced without the staff selecting the model through testing.

[0108] Based on the same inventive concept, the embodiment of the present application also provides an image simulation device. The image simulation device provided by the embodiment of the present application will be described in detail below. Figure 5 The image simulation device provided by the embodiment of the present application will be described in detail below.

[0109] Figure 5 ​A structural schematic diagram of an image simulation device is shown.

[0110] As shown in the figure, the image simulation device can include: Figure 5

[0111] The acquisition module 501 is configured to acquire a first image height corresponding to a target pixel point in a simulation screen,

[0112] The input module 502 is configured to input the first image height into a target distortion model, and output a second image height corresponding to the target pixel point in a target image, the target image being obtained by projecting images in multiple directions collected by a first camera onto multiple surfaces of a cubic box, and a value of a target parameter in the target distortion model being determined according to calibration distortion data of the first camera,

[0113] The first determination module 503 is configured to determine a first pixel coordinate corresponding to the target pixel point in the target image according to the second image height,

[0114] The second determination module 504 is configured to determine a target color RGB value at the first pixel coordinate in the target image,

[0115] The setting module 505 is configured to set the RGB value of the target pixel point in the simulation screen as a target RGB value, and obtain a simulation image.

[0116] Therefore, the first image height corresponding to the target pixel point in the simulation screen can be acquired, and the first image height can be input into the target distortion model, and the second image height corresponding to the target pixel point in the target image can be output, then the first pixel coordinate corresponding to the target pixel point in the target image can be determined according to the second image height, and the target color RGB value at the first pixel coordinate in the target image can be determined, and the RGB value of the target pixel point in the simulation screen can be set as the target RGB value, so that the simulation image can be obtained. Since the target image is obtained by projecting images in multiple directions collected by the first camera onto multiple surfaces of the cubic box, the performance consumption caused by establishing multiple cameras is greatly reduced. In addition, the value of the target parameter in the target distortion model is determined according to the calibration distortion data of the first camera, so that the instability caused by different fixed models is avoided, the accuracy of the simulation is improved, a better simulation effect can be achieved, and the workload of the staff is reduced without the staff selecting the model through testing.

[0117] In some embodiments, in order to avoid the instability caused by using a fixed model without parameters and reduce the workload of the staff, the device can further include:

[0118] ​The third determining module is configured to project the target world coordinate onto a cubic box before obtaining a first height of a target pixel point in the simulation screen, establish a first coordinate system with a center of the cubic box as a first coordinate origin, and determine a first coordinate corresponding to the target world coordinate in the first coordinate system.

[0119] The fourth determining module is configured to translate the first coordinate origin by a target distance along a Z-axis direction of the first coordinate system to obtain a second coordinate system, and determine a second coordinate corresponding to the first coordinate in the second coordinate system.

[0120] The fifth determining module is configured to normalize the second coordinate to determine a third coordinate corresponding to the second coordinate in a two-dimensional plane.

[0121] The sixth determining module is configured to project the third coordinate onto an imaging plane of the simulation fisheye camera to determine a fourth coordinate corresponding to the third coordinate on the imaging plane of the simulation fisheye camera.

[0122] The seventh determining module is configured to determine an expression of a target height of a second pixel point corresponding to the fourth coordinate and a tangent expression of a target incident angle.

[0123] The eighth determining module is configured to determine a target mapping relationship between the target height and the target incident angle according to the expression of the target height and the tangent expression of the target incident angle.

[0124] The ninth determining module is configured to determine a target distortion model according to the target mapping relationship and the distortion model.

[0125] The tenth determining module is configured to determine a value of a target parameter in the distortion model according to the calibration distortion data and the target distortion model, the calibration distortion data including a mapping relationship between a height and an incident angle.

[0126] The eleventh determining module is configured to determine the target distortion model according to the distortion model and the value of the target parameter.

[0127] In some embodiments, in order to obtain a more accurate simulation image, the device can further include:

[0128] The twelfth determining module is configured to determine a value of a target focal length and a value of a target distance of the simulation fisheye camera according to the calibration distortion data and the target distortion model after determining the target distortion model according to the target mapping relationship and the distortion model.

[0129] The first determining module 503 can include:

[0130] The determining sub-module is configured to determine a first pixel coordinate corresponding to the target pixel point in the target image according to the second height, the value of the target focal length and the value of the target distance.

[0131] In some embodiments, in order to more accurately determine the first coordinate, the third determining module can comprise:

[0132] a first calculating sub-module, configured to divide the coordinates of the X axis, the Y axis and the Z axis of the target world coordinate (X, Y, Z) by to obtain the first coordinate as wherein,

[0133] In some embodiments, in order to more accurately determine the second coordinate corresponding to the first coordinate point in the second coordinate system, the fourth determining module can comprise:

[0134] a second calculating sub-module, configured to add the target distance ξ to the coordinates of the X axis, the Y axis and the Z axis of the first coordinate to obtain the second coordinate as

[0135] In some embodiments, in order to more accurately determine the third coordinate corresponding to the second coordinate on the two-dimensional plane, the fifth determining module can comprise:

[0136] a third calculating sub-module, configured to divide the coordinates of the X axis, the Y axis and the Z axis of the second coordinate by the Z axis coordinate of the second coordinate to obtain the third coordinate as

[0137] In some embodiments, in order to more accurately determine the fourth coordinate corresponding to the third coordinate on the imaging plane corresponding to the simulated fisheye camera, the sixth determining module can comprise:

[0138] a fourth calculating sub-module, configured to multiply the third coordinate by the parameter matrix corresponding to the simulated fisheye camera to obtain the fourth coordinate The parameter matrix is determined according to the target focal length f f of the simulated fisheye camera.

[0139] In some embodiments, in order to more accurately determine the target mapping relationship between the target image height and the target incident angle, the expression of the target image height r is the tangent expression of the target incident angle θ is and the target mapping relationship is

[0140] In some embodiments, the mathematical formula of the distortion model is r f = r * (1 + k1r 2 + k2r 4 + ··· + k n r 2n ), wherein rf denotes a distortion function, n is a preset constant, k is a target parameter, in order to accurately derive the formula of the target distortion model, the ninth determination module can include:

[0141] The fifth calculation sub-module is configured to substitute the target mapping relationship into the distortion model r f = r * (1 + k1r 2 + k2r 4 +... + knr n r 2n ), and the formula of the target distortion model is

[0142] Figure 6 FIG. 1 shows a structural schematic diagram of an electronic device according to an embodiment of the present application.

[0143] As shown in Figure 6 FIG. 6, the electronic device 6 can implement an exemplary hardware architecture of the electronic device according to the image simulation method and the image simulation apparatus in the embodiments of the present application. The electronic device can refer to the electronic device in the embodiments of the present application.

[0144] The electronic device 6 can include a processor 601 and a memory 602 storing computer program instructions.

[0145] Specifically, the processor 601 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or can be configured to implement one or more integrated circuits of the embodiments of the present application.

[0146] The memory 602 can include mass storage for data or instructions. As an example and not by way of limitation, the memory 602 can include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc (e.g., a compact disc (CD) or a digital versatile disc (DVD)), a solid-state drive (SSD), a USB drive, or a combination of two or more of these. Where appropriate, the memory 602 can include removable or non-removable (or fixed) media, where appropriate. The memory 602 can be internal or external to the integrated gateway disaster recovery appliance. In particular embodiments, the memory 602 is non-volatile, solid-state memory. In particular embodiments, the memory 602 can include read-only memory (ROM), random-access memory (RAM), a magnetic disk storage medium, an optical storage medium, flash memory devices, electrical, optical, or other physically tangible storage

[0147] The processor 601 implements the image simulation method of any of the above embodiments by reading and executing computer program instructions stored in the memory 602.

[0148] In one example, the electronic device can further include a communication interface 603 and a bus 604. As shown, the processor 601, the memory 602, and the communication interface 603 are connected through the bus 604 and complete communication with each other. Figure 6

[0149] The communication interface 603 is mainly used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application.

[0150] The bus 604 includes hardware, software, or both, that couples components of the electronic device to each other. As an example and not by way of limitation, the bus can include an accelerated graphics port (AGP) or other graphics bus, a

[0151] ​Enhanced Industry Standard Architecture (EISA) bus, Front Side Bus (FSB), HyperTransport (HT) interconnect, Industry Standard Architecture (ISA) bus, InfiniBand® interconnect, Low Pin Count (LPC) bus, Memory Bus, Micro Channel Architecture (MCA) bus, Peripheral Component Interconnect (PCI) bus, PCI-Express (PCI-X) bus, Serial Advanced Technology Attachment (SATA) bus, Video Electronics Standards Association local (VLB) bus, or other suitable bus or combination of two or more of these. Where suitable, bus 504 can comprise one or more buses. Although particular buses are described and shown in the embodiments of the present application, the present application contemplates any suitable bus or interconnect.

[0152] The electronic device can execute the image simulation method in the embodiments of the present application, thereby realizing the image simulation method and device described in combination Figures 1 to 5 with the above-mentioned embodiments.

[0153] In addition, in combination with the image simulation method in the above-mentioned embodiments, the embodiments of the present application can provide a computer storage medium for implementation. The computer storage medium has computer program instructions stored thereon, which, when executed by a processor, implement any of the image simulation methods in the above-mentioned embodiments.

[0154] It needs to be made clear that the present application is not limited to the particular configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted herein. In the above-mentioned embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of the steps, after understanding the spirit of the present application.

[0155] The functional blocks shown in the structural block diagrams described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the present application are program or code segments used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. The "machine-readable medium" can include any medium capable of storing or transmitting information. Examples of the machine-readable medium include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segments can be downloaded via a computer network such as the Internet, an intranet, etc.

[0156] It should also be noted that the example embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in an order different from the embodiments, or several steps can be performed simultaneously.

[0157] The above generally describes aspects of the present application with reference to a flowchart and / or a block diagram of the method, the apparatus (system) and computer program product according to embodiments of the present application. It should be understood that each block of the flowchart and / or the block diagram, as well as combinations of blocks in the flowchart and / or the block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, so that the instructions executed via the processor of the computer or other programmable data processing apparatus implement the functions / acts specified in the flowchart and / or the block diagram block or blocks. The processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It should also be understood that each block of the flowchart and / or the block diagram, as well as combinations of blocks in the flowchart and / or the block diagram, can also be implemented by dedicated hardware, or a combination of computer instructions and dedicated hardware.

[0158] The above is only a specific implementation of the present application, and those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, module and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here. It should be understood that the protection scope of the present application is not limited to this, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed in the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. An image simulation method characterized by, The method comprises: acquiring a first image height corresponding to a target pixel point in a simulation screen, input the first image height into a target distortion model, and output a second image height corresponding to the target pixel point in a target image, the target image being obtained by projecting images of multiple orientations collected by a first camera onto multiple surfaces of a cubic box, a value of a target parameter in the target distortion model being determined according to calibration distortion data of the first camera, wherein a mathematical formula of the target distortion model is r f denotes a distortion function, n is a preset constant, k is the target parameter, θ is a target incident angle, ξ is a target distance, f f is a target focal length of the camera, and the calibration distortion data includes a mapping relationship between the image height and the incident angle. determining a first pixel coordinate corresponding to the target pixel point in the target image according to the second image height, determining a target color RGB value at the first pixel coordinate in the target image, setting an RGB value of the target pixel point in the simulation screen as the target RGB value to obtain a simulation image.

2. The method of claim 1, wherein, Before the acquiring a first image height corresponding to a target pixel point in a simulation screen, the method further comprises: projecting a target world coordinate onto a normalized sphere, establishing a first coordinate system with the center of the normalized sphere as a first coordinate origin, and determining a first coordinate corresponding to the target world coordinate in the first coordinate system, translating the first coordinate origin along the Z-axis direction of the first coordinate system by a target distance to obtain a second coordinate system, and determining a second coordinate corresponding to the first coordinate in the second coordinate system, normalizing the second coordinate to determine a third coordinate corresponding to the second coordinate on a two-dimensional plane, projecting the third coordinate onto an imaging plane of a simulation fisheye camera to determine a fourth coordinate corresponding to the third coordinate on the imaging plane of the simulation fisheye camera, determining an expression of a target image height of a second pixel corresponding to the fourth coordinate and a tangent expression of a target incident angle, determining a target mapping relationship between the target image height and the target incident angle according to the expression of the target image height and the tangent expression of the target incident angle, determining a target distortion model according to the target mapping relationship and a distortion model, determining a value of the target parameter in the distortion model according to the calibration distortion data and the target distortion model, the calibration distortion data comprising a mapping relationship between an image height and an incident angle, determining the target distortion model according to the distortion model and the value of the target parameter.

3. The method of claim 2, wherein, After the determining a target distortion model according to the target mapping relationship and a distortion model, the method further comprises: determining a value of a target focal length of the simulation fisheye camera and a value of the target distance according to the calibration distortion data and the target distortion model, the determining a first pixel coordinate corresponding to the target pixel point in the target image according to the second image height, the value of the target focal length and the value of the target distance, comprises: determining a first pixel coordinate corresponding to the target pixel point in the target image according to the second image height, the value of the target focal length and the value of the target distance.

4. The method of claim 2, wherein, the projecting a target world coordinate onto a normalized sphere, establishing a first coordinate system with the center of the normalized sphere as a first coordinate origin, and determining a first coordinate corresponding to the target world coordinate in the first coordinate system, comprises: Divide the coordinates of the X-axis, Y-axis and Z-axis of the target world coordinates (X, Y, Z) by to obtain the first coordinates wherein, 5. The method of claim 2, wherein, the translating the first coordinate origin along the Z-axis direction of the first coordinate system by a target distance to obtain a second coordinate system, and determining a second coordinate corresponding to the first coordinate in the second coordinate system, comprises: The first coordinate The second coordinate is obtained by adding the target distance ξ to the X-axis, Y-axis and Z-axis coordinates of the first coordinate 6. The method of claim 2, wherein, the normalizing the second coordinate to determine a third coordinate corresponding to the second coordinate on a two-dimensional plane, comprises: divide the X-axis, Y-axis and Z-axis coordinates of the second coordinate by the Z-axis coordinate of the second coordinate, and obtain the third coordinate as ​ 7. The method of claim 2, wherein, The third coordinate is projected onto an imaging plane of a simulated fisheye camera, and a fourth coordinate corresponding to the third coordinate on an imaging plane corresponding to the simulated fisheye camera is determined. multiplying the third coordinate by a parameter matrix corresponding to the simulated fisheye camera obtaining the fourth coordinate The parameter matrix is determined according to a target focal length f of the simulated fisheye camera f .​ 8. The method of claim 2, wherein, An expression of the target image height r is An expression of the tangent of the target incident angle θ is The target mapping relationship is 9. The method of claim 8, wherein, The mathematical formula of the distortion model is r f = r * (1 + k1r 2 + k2r 4 + ··· + knr n r 2n ), wherein r f represents a distortion function, n is a preset constant, and k is the target parameter. The target distortion model is determined according to the target mapping relationship and a distortion model, and the target mapping relationship is determined according to the first mapping relationship and the second mapping relationship. The target mapping relationship is obtained Substitute the distortion model r f = r * (1 + k1r 2 + k2r 4 + ··· + k n r 2n ), and a formula of the target distortion model is obtained 10. An image simulation apparatus characterized by comprising: The device comprises: An acquisition module is configured to acquire a first image height corresponding to a target pixel point in a simulation screen, an input module is configured to input the first image height to a target distortion model, and a second image height corresponding to the target pixel point in a target image is output, the target image being obtained by projecting images of multiple orientations collected by a first camera onto multiple surfaces of a cubic box, a value of a target parameter in the target distortion model being determined according to calibration distortion data of the first camera, wherein a mathematical formula of the target distortion model is r f denotes a distortion function, n is a preset constant, k is a target parameter, θ is a target incident angle, ξ is a target distance, f f is a target focal length of the camera, and the calibration distortion data includes a mapping relationship between the image height and the incident angle. A first determination module is configured to determine a first pixel coordinate corresponding to the target pixel point in the target image according to the second image height. A second determination module is configured to determine a target color RGB value at the first pixel coordinate in the target image. A setting module is configured to set the RGB value of the target pixel point in the simulated screen as the target RGB value to obtain a simulated image.

Images