Forward projection calculation method for rotary biprism imaging system based on Fermat principle

Abstract

The invention provides a forward projection calculation method for a rotating biprism imaging system based on the Fermat principle, and the method comprises the steps: obtaining a target function through the Fermat principle according to the characteristics of the rotating biprism imaging system, converting a forward projection problem under a refraction condition into a path optimization problem, and carrying out the calculation of the forward projection of the rotating biprism imaging system through a pinhole camera model. And forward projection calculation under the refraction condition of the rotary biprism is realized. Based on the Fermat principle, the method can accurately simulate the propagation path of light in a plurality of non-parallel refraction interfaces, especially in a dynamic environment of a rotating biprism, significantly improves the precision of forward projection calculation, and reduces errors caused by the fact that a conventional method is supposed to be too simplified.

Description

Technical Field

[0001] The invention belongs to the technical fields of image processing, imaging system and optical simulation, and in particular relates to a forward projection calculation method of a rotating double prism imaging system based on the Fermat principle. Background Art

[0002] The rotating dual prism imaging system is an imaging device based on optical principles and is widely used in 3D imaging, visual systems, laser scanning, spatial positioning and other fields. The system usually consists of two prisms with specific geometric shapes. By rotating and adjusting the position of the prisms, the propagation direction of light can be precisely controlled. The rotating dual prism affects the imaging effect by changing the propagation path of light, especially in high-precision 3D reconstruction and object tracking applications.

[0003] In a rotating dual-prism imaging system, forward projection calculation is a key step in analyzing and predicting the propagation path of light. By accurately calculating the path of light from the object to the imaging plane, the forward projection method provides the necessary data support for subsequent image correction, three-dimensional reconstruction and other tasks. However, some existing forward projection calculation methods usually only study a single refractive interface or multiple parallel refractive interfaces, and these methods are often not applicable under the conditions of multiple non-parallel refractive interfaces. Especially in a rotating dual-prism imaging system, light will experience multiple refractions when passing through the prism, and the normal direction of each refractive interface is usually non-parallel. In addition, the normal direction of the refractive interface and the light path will change continuously with the rotation of the prism, which makes it difficult for traditional forward projection methods to accurately describe the propagation path of light. Summary of the invention

[0004] In view of the problems existing in the above-mentioned prior art solutions, the present invention proposes a forward projection calculation method for a rotating dual-prism imaging system based on the Fermat principle.

[0005] The principle of the present invention is as follows: due to the introduction of the prism, the imaging system is highly nonlinear, and the traditional pinhole camera model cannot perform forward projection calculations. Based on the Fermat principle, the forward projection problem under refraction conditions is converted into a path optimization problem by utilizing the fact that the path of light is unique and minimum when it propagates in different media.

[0006] The specific implementation process is as follows: in a spatial rectangular coordinate system with the camera optical center as the origin, based on the camera's internal parameters and the three-dimensional coordinates of the object point in space, determine the intersection of the rotating double prism refraction interface and the camera's visual axis; calculate the plane equation of the refraction interface according to the prism rotation angle, take the object point as the starting point, based on the pinhole camera model, calculate the total optical path of the light according to the ray tracing method, use the shortest path optical path determined by the Fermat principle as the objective function, and use the numerical optimization method to determine the actual direction of the initial light, and finally obtain the pixel coordinates of the object point in the camera coordinate system, thereby realizing the forward projection of the object point of the rotating double prism imaging system.

[0007] The technical solution specifically adopted by the present invention to solve the technical problem is:

[0008] A forward projection calculation method for a rotating bi-prism imaging system based on the Fermat principle: According to the characteristics of the rotating bi-prism imaging system, the Fermat principle is used to obtain the objective function, the forward projection problem under refraction conditions is transformed into a path optimization problem, and combined with a pinhole camera model, the forward projection calculation under the refraction condition of the rotating bi-prism is realized.

[0009] Furthermore, a spatial rectangular coordinate system is established with the optical center of the camera as the origin and the system parameters are determined; the intersection of the Z axis corresponding to the camera visual axis and the refraction interface of the double prism is determined in the spatial rectangular coordinate system, which are respectively denoted as O 1 , O 2 , O 3 , O 4 ; According to the rotation angle of the prism (θ 1 ,θ 2 ) Determine the plane equation of the prism refraction interface; determine the intrinsic parameters of the camera and the three-dimensional coordinates of the object point in space, and calculate the direction of the light after refraction through the prism and the intersection of the ray and the prism refraction interface Where i∈[1,4]; taking the object point as the starting point, the total optical path of the light from the object point to the optical center of the camera is calculated, and the shortest path optical path determined by the Fermat principle is used as the objective function; the direction of the initial light when the total optical path is the shortest is calculated by the numerical optimization method as the actual direction of the initial light, and the pixel coordinates of the object point in the camera coordinate system are calculated in combination with the pinhole camera model to realize the forward projection of the object point of the rotating dual prism imaging system.

[0010] Furthermore, in the rotating dual-prism imaging system, the two prisms have the same parameters, and the camera visual axis is the rotation axis of the prism.

[0011] Furthermore, the establishment of a spatial rectangular coordinate system with the camera optical center as the origin and determination of system parameters are specifically as follows: 0Establish a spatial rectangular coordinate system as the origin, with the positive direction of the camera's visual axis as the positive direction of the Z axis, the positive direction of the Y axis vertically downward, and the positive direction of the X axis to the right; let the prism close to the camera be prism 1, and the prism far from the camera be prism 2, and prism 1 and prism 2 are respectively denoted as Π 1 and Π 2 ; The wedge angle of the prism is α, the refractive index is n, and the refractive index of air is n 0 ; When the main section of the prism is on the YOZ plane and the thick end faces the positive direction of the Y axis, the prism is in the zero position. When it rotates clockwise around the Z axis, the angle is positive. The angles of prism 1 and prism 2 are denoted by θ 1 and θ 2 ; Determine the intrinsic parameters of the camera and the three-dimensional coordinates of the object point P as needed.

[0012] Furthermore, the intrinsic parameters of the camera and the three-dimensional coordinates of the object point in space are determined to calculate the direction of the light after refraction through the prism. and the intersection of the ray and the prism refraction interface The specific process is:

[0013] Taking the object point P as the starting point, the initial direction of the light is recorded as Where η, γ, ξ represent the components of the vector in the X, Y, and Z directions respectively. The direction of the refracted light can be obtained according to the law of vector refraction:

[0014]

[0015] where k = 0, 1, 2 or 3, n k is the refractive index of light in the medium. When k is an even number, it represents the refractive index of light in the air. When k is an odd number, it represents the refractive index in the prism.

[0016] Determine the point where a ray intersects the prism surface by intersecting a line with a plane:

[0017]

[0018] Where i∈[0,4], correspond is the origin of the coordinate system; correspond For object points.

[0019] Furthermore, the total optical path of the light from the object point to the optical center of the camera is calculated, and the shortest path optical path determined according to the Fermat principle is used as the objective function:

[0020] Calculate the light from the object point P to the camera optical center O 0 The total optical path traveled, the objective function is expressed by the following formula:

[0021]

[0022] Where n j is the refractive index of the jth refractive medium;

[0023] The total optical path is about the initial light direction The direction vector is a unit vector with the following constraints:

[0024] η 2 +γ 2 +ξ 2 =1.

[0025] Furthermore, the direction of the initial light when the total optical path is the shortest calculated by the numerical optimization method is the actual direction of the initial light, and the pixel coordinates of the object point in the camera coordinate system are obtained in combination with the pinhole camera model as follows:

[0026] Combined with the above constraints, choose As a starting point for an iterative process;

[0027] During the iterative optimization process, several boundary conditions are set to limit the range of parameter changes during the iteration process, and the maximum range of the light is determined to be η, γ∈[-1,1], ξ∈[-1,0];

[0028] The convergence condition is to meet one of the following two conditions to end the iteration process: 1) The number of iterations I reaches the maximum number of iterations I max ; 2) The change of the objective function between two consecutive iterations is less than the preset tolerance;

[0029] Find Later brought into Calculate the light and prism π 1 The intersection of the inclined surface is projected into the camera using the pinhole camera model. The projection formula is:

[0030]

[0031] Where K is the intrinsic parameter of the camera, λ is the scale factor, is the pixel coordinate of the object point P in the camera after refraction through the prism, that is, the forward projection of the object point P in the rotating dual prism imaging system.

[0032] And, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of a forward projection calculation method for a rotating dual-prism imaging system based on the Fermat principle as described above are implemented.

[0033] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a forward projection calculation method for a rotating dual-prism imaging system based on the Fermat principle as described above.

[0034] Compared with the prior art, the present invention and its preferred embodiments have at least the following beneficial effects:

[0035] 1. Based on Fermat's principle, it can accurately simulate the propagation path of light in multiple non-parallel refractive interfaces, especially in the dynamic environment of rotating double prisms, which significantly improves the accuracy of forward projection calculation and reduces the errors caused by oversimplified assumptions of traditional methods.

[0036] 2. It can fully consider the multiple refraction processes of light in the rotating double prism system, and is suitable for scenes with multiple non-parallel refraction interfaces and dynamically changing normal directions, which expands the application scope of the forward projection calculation method.

[0037] 3. Through more accurate light path prediction, the present invention can provide more reliable data support for image correction and three-dimensional reconstruction, thereby improving the overall imaging quality and three-dimensional reconstruction accuracy of the rotating dual-prism imaging system.

[0038] 4. It provides an engineering-implementable forward projection calculation method that can be applied to laser scanning, three-dimensional vision, object positioning, and automated detection, providing theoretical support and technical solutions for the design and optimization of related optical systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0040] Figure 1 It is a schematic diagram of a rotating dual prism imaging system according to an embodiment of the present invention.

[0041] Figure 2 The present invention is a flowchart of a forward projection calculation method of a rotating dual prism imaging system based on the Fermat principle.

[0042] Figure 3 It is a schematic diagram of a forward projection calculation method of a rotating dual prism imaging system based on the Fermat principle according to an embodiment of the present invention.

[0043] Figure 4 This is a result example diagram of a forward projection calculation method for a rotating dual-prism imaging system based on the Fermat principle in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the features and advantages of the present invention more clearly understood, the following embodiments are specifically described in detail as follows:

[0045] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0046] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0047] The present invention provides a forward projection calculation method for a rotating dual-prism imaging system based on the Fermat principle. The method utilizes the unique and shortest path characteristics of light when propagating in different media, converts the forward projection problem under refraction conditions into a path optimization problem, and accurately calculates the pixel coordinates of the object point in the camera in combination with a pinhole camera model. Through the corresponding internal camera parameters, prism parameters, and the spatial position parameters of the prism relative to the camera and the three-dimensional coordinates of the object point P, the direction of the initial light of the object point P is assumed, and the actual initial light direction is calculated by iterative optimization, and the forward projection calculation is realized in combination with the pinhole camera model. The specific implementation steps for reference are as follows:

[0048] Step S1, establishing a spatial rectangular coordinate system with the camera optical center as the origin and determining system parameters;

[0049] Step S2, determine the intersection of the Z axis and the prism refraction interface in the spatial rectangular coordinate system, and record them as O and 1 , O 2 , O 3 , O 4 ;

[0050] Step S3, according to the rotation angle (θ 1 ,θ 2 ) determine the plane equation of the prism refraction interface;

[0051] Step S4, determine the intrinsic parameters of the camera and the three-dimensional coordinates of the object point in space, and calculate the direction of the light after refraction through the prism and the intersection of the ray and the prism refraction interface Where i∈[1,4];

[0052] Step S5, calculating the total optical path of the light from the object point to the optical center of the camera, and taking it as the objective function;

[0053] Step S6, using a numerical optimization method to calculate the direction of the initial light when the total optical path is the shortest, as the actual direction of the initial light, and combining the pinhole camera model to calculate the pixel coordinates of the object point.

[0054] The present invention is described in more detail below in conjunction with the accompanying drawings and embodiments. This embodiment is specifically implemented based on the technical solution of the present invention, and provides detailed implementation methods and operating steps, but the protection scope of the present invention is not limited to the following embodiments.

[0055] The embodiment of the present invention provides a forward projection calculation method for a rotating dual-prism imaging system based on the Fermat principle, wherein the rotating dual-prism imaging system is composed of two prisms with completely identical parameters and an RGB industrial camera. Figure 1 shown. Figure 1 In the figure, f represents the focal length of the camera, p 0 The main point of the camera.

[0056] In this embodiment, the parameters of the rotating dual prism imaging system are set as follows: the prism wedge angle is α=11.35°, the camera resolution is set to 3072×2048 pixels, and the pixel size is d x =d y =2.4um, the focal length of the camera is f=12mm, d 1 =26.99mm, d 2 =d 4 =5.51mm,d 3 =2mm.

[0057] refer to Figure 2 This embodiment provides a forward projection calculation method for a rotating dual prism imaging system based on the Fermat principle, comprising the following steps:

[0058] Step S1, establishing a spatial rectangular coordinate system with the camera optical center as the origin and determining system parameters;

[0059] Step S2, determine the intersection of the Z axis and the prism refraction interface in the spatial rectangular coordinate system, and record them as O and 1 , O 2 , O 3 , O 4 ;

[0060] Step S3, according to the rotation angle (θ 1 ,θ 2 ) determine the plane equation of the prism refraction interface;

[0061] Step S4, determine the intrinsic parameters of the camera and the three-dimensional coordinates of the object point in space, and calculate the direction of the light after refraction through the prism and the intersection of the ray and the prism refraction interface Where i∈[1,4];

[0062] Step S5, calculating the total optical path of the light from the object point to the optical center of the camera, and taking it as the objective function;

[0063] Step S6, using a numerical optimization method to calculate the direction of the initial light when the total optical path is the shortest, as the actual direction of the initial light, and combining the pinhole camera model to calculate the pixel coordinates of the object point.

[0064] In this embodiment, step S1 is specifically as follows:

[0065] Step S11, with the camera optical center O 0 Establish a spatial rectangular coordinate system for the origin, with the positive direction of the camera's visual axis as the positive direction of the Z axis, the positive direction of the Y axis vertically downward, and the positive direction of the X axis to the right. The three axes of this coordinate system follow the right-hand rule;

[0066] Step S12: the prism close to the camera is prism 1, and the prism far from the camera is prism 2. Prism 1 and prism 2 are respectively denoted as π 1 and Π2; the wedge angle of the prism is α, the refractive index is n, and the refractive index of air is n 0 ; When the main section of the prism is on the YOZ plane and the thick end faces the positive direction of the Y axis, the prism is in the zero position. When it rotates clockwise around the Z axis, the angle is positive. The angles of prism 1 and prism 2 are denoted by θ 1 and θ 2 ; Determine the intrinsic parameters of the camera and the three-dimensional coordinates of the object point P as needed;

[0067] Step S2 is specifically as follows:

[0068] Determine the intersection of the Z axis and the prism refraction interface in the spatial rectangular coordinate system, and denote them as O 1 , O 2 , O 3 , O 4 ;

[0069] Step S3 is specifically as follows:

[0070] According to the rotation angle of the prism (θ 1 ,θ 2 ) determine the plane equation of the prism refraction interface;

[0071] Step S4 is specifically as follows:

[0072] Step S41, taking the object point P as the starting point, the initial direction of the light is recorded as According to the law of vector refraction, the direction of the light after refraction can be calculated. The formula is as follows:

[0073]

[0074] where i = 0, 1, 2 or 3, ni is the refractive index of light in the medium. When i is an even number, it represents the refractive index of light in the air. When i is an odd number, it represents the refractive index in the prism.

[0075] Step S42, determining the intersection point of the light ray and the prism surface by intersecting the straight line with the plane, the formula is as follows:

[0076]

[0077] Where i∈[0,4], Right now is the origin of the coordinate system; Right now For object points;

[0078] Step S5 is specifically as follows:

[0079] Step S51, calculate the light from the object point P to the camera optical center O 0 The total optical path length is Figure 3 As shown, the objective function can be expressed by the following formula:

[0080]

[0081] Where n j is the refractive index of the jth refractive medium;

[0082] Step S52, the total optical path is about the initial light direction The direction vector is a unit vector with the following constraints:

[0083] η 2 +γ 2 +ξ 2 =1

[0084] Step S6 is specifically as follows:

[0085] Step S61, first determine an initial value, combined with the constraints in step S52, to facilitate calculation, select As a starting point for an iterative process;

[0086] Step S62, in the iterative optimization method, several boundary conditions are set to limit the range of parameter changes during the iteration process to ensure the rationality and stability of the optimization results. According to the actual situation, the maximum range of the light is determined to be η, γ∈[-1,1], ξ∈[-1,0];

[0087] Step S63, determine the convergence condition, and the iterative process ends if one of the following two conditions is met: 1) The number of iterations I reaches the maximum number of iterations I max ; 2) The change of the objective function between two consecutive iterations is less than the preset tolerance;

[0088] Step S64, find Then put its value into Calculate the light and prism π 1 The intersection of the inclined surface is projected into the camera using the pinhole camera model. The projection formula is:

[0089]

[0090] Where K is the intrinsic parameter of the camera, λ is the scale factor, and p is the pixel coordinate of the object point P in the camera after refraction through the prism, that is, the forward projection of the object point P in the rotating dual prism imaging system.

[0091] In this embodiment, steps S4-S6 construct an objective function based on the Fermat principle, find the optimal solution through a numerical optimization algorithm, and finally perform forward projection calculation in combination with a pinhole camera. To verify the robustness of the method, 171 object points are forward projected under multiple prism rotation angles. The object point is 2500 mm from the optical center of the camera, and the prism rotation angle (θ 1 ,θ 2 ) are (0,120), (60,180), (120,240), (180,300) respectively. The results are as follows Figure 4 As shown in Figure 3, the image point distribution at different prism rotation angles demonstrates that this method has strong robustness and can adapt to the complex situation of non-parallel refractive interfaces.

[0092] The above embodiments show that the forward projection calculation method of a rotating dual-prism imaging system based on the Fermat principle proposed in the present invention can calculate the forward projection of the object point under the above steps, providing more reliable data support for image correction and three-dimensional reconstruction, thereby improving the overall imaging quality and three-dimensional reconstruction accuracy of the rotating dual-prism imaging system.

[0093] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0094] It needs to be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to execute the above method. The storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0095] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0096] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

[0097] The present invention is not limited to the above-mentioned optimal implementation mode. Anyone can derive other various forms of a forward projection calculation method for a rotating dual-prism imaging system based on the Fermat principle under the inspiration of the present invention. All equal changes and modifications made within the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. A forward projection calculation method for a rotating dual-prism imaging system based on the Fermat principle, characterized in that: According to the characteristics of the rotating biprism imaging system, the Fermat principle is used to obtain the objective function, and the forward projection problem under refraction conditions is transformed into a path optimization problem. Combined with the pinhole camera model, the forward projection calculation under the refraction of the rotating biprism is realized.

2. The forward projection calculation method of a rotating dual prism imaging system based on the Fermat principle according to claim 1, characterized in that: Establish a spatial rectangular coordinate system with the camera optical center as the origin and determine the system parameters; determine the intersection points of the Z axis corresponding to the camera visual axis and the double prism refraction interface in the spatial rectangular coordinate system, which are recorded as O1, O2, O3, and O4 respectively; determine the plane equation of the prism refraction interface according to the rotation angle of the prism (θ1, θ2); determine the intrinsic parameters of the camera and the three-dimensional coordinates of the object point in space, and calculate the direction of the light after refraction through the prism and the intersection of the ray and the prism refraction interface Where i∈[1,4]; taking the object point as the starting point, the total optical path of the light from the object point to the optical center of the camera is calculated, and the shortest path optical path determined by the Fermat principle is used as the objective function; the direction of the initial light when the total optical path is the shortest is calculated by the numerical optimization method as the actual direction of the initial light, and the pixel coordinates of the object point in the camera coordinate system are calculated in combination with the pinhole camera model to realize the forward projection of the object point of the rotating dual prism imaging system.

3. The forward projection calculation method of a rotating dual prism imaging system based on the Fermat principle according to claim 1, characterized in that: In the rotating dual-prism imaging system, the two prisms have the same parameters, and the camera visual axis is the rotation axis of the prism.

4. The forward projection calculation method of a rotating dual prism imaging system based on the Fermat principle according to claim 2, characterized in that: The method of establishing a spatial rectangular coordinate system with the optical center of the camera as the origin and determining the system parameters is specifically as follows: establishing a spatial rectangular coordinate system with the optical center of the camera O0 as the origin, with the positive direction of the camera's visual axis as the positive direction of the Z axis, the positive direction of the Y axis vertically downward, and the positive direction of the X axis as the right; assuming that the prism close to the camera is prism 1, and the prism far from the camera is prism 2, prism 1 and prism 2 are respectively denoted as Π1 and Π2; the wedge angles of the prisms are all α, the refractive indexes are all n, and the refractive index of air is n0; when the main cross-section of the prism is on the YOZ plane and the thick end faces the positive direction of the Y axis, it is the zero position of the prism, and the angle is positive when rotating clockwise around the Z axis, and the angles of prism 1 and prism 2 are respectively denoted as θ1 and θ2; the intrinsic parameters of the camera and the three-dimensional coordinates of the object point P are determined as needed.

5. The forward projection calculation method of a rotating dual prism imaging system based on the Fermat principle according to claim 4, characterized in that: The method determines the intrinsic parameters of the camera and the three-dimensional coordinates of the object point in space, and calculates the direction of the light after refraction through the prism and the intersection of the ray and the prism refraction interface The specific process is: Taking the object point P as the starting point, the initial direction of the light is recorded as Where η, γ, ξ represent the components of the vector in the X, Y, and Z directions respectively. The direction of the refracted light can be obtained according to the law of vector refraction: where k = 0, 1, 2 or 3, n k is the refractive index of light in the medium. When k is an even number, it represents the refractive index of light in the air. When k is an odd number, it represents the refractive index in the prism. Determine the point where a ray intersects the prism surface by intersecting a line with a plane: Where i∈[0,4], correspond is the origin of the coordinate system; correspond For object points.

6. The forward projection calculation method of a rotating dual prism imaging system based on the Fermat principle according to claim 5, characterized in that: The total optical path of the light from the object point to the optical center of the camera is calculated, and the shortest path optical path determined by Fermat's principle is used as the objective function: Calculate the total optical path of the light from the object point P to the camera optical center O0. The objective function is expressed by the following formula: Where n j is the refractive index of the jth refractive medium; The total optical path is about the initial light direction The direction vector is a unit vector with the following constraints: or 2 +g 2 +ξ 2 =1.

7. The forward projection calculation method of a rotating dual prism imaging system based on the Fermat principle according to claim 6, characterized in that: The direction of the initial light when the total optical path is the shortest calculated by the numerical optimization method is the actual direction of the initial light. The pixel coordinates of the object point in the camera coordinate system are obtained by combining the pinhole camera model as follows: Combined with the above constraints, choose As a starting point for an iterative process; During the iterative optimization process, several boundary conditions are set to limit the range of parameter changes during the iteration process, and the maximum range of the light is determined to be η, γ∈[-1,1], ξ∈[-1,0]; The convergence condition is to meet one of the following two conditions to end the iteration process: 1) The number of iterations I reaches the maximum number of iterations I max ; 2) The change of the objective function between two consecutive iterations is less than the preset tolerance; Find Later brought into The intersection of the light ray and the inclined surface of the prism Π1 is calculated and projected into the camera using the pinhole camera model. The projection formula is: Where K is the intrinsic parameter of the camera, λ is the scale factor, is the pixel coordinate of the object point P in the camera after refraction through the prism, that is, the forward projection of the object point P in the rotating dual prism imaging system.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of a forward projection calculation method for a rotating dual-prism imaging system based on the Fermat principle as described in any one of claims 1 to 7 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a forward projection calculation method for a rotating dual-prism imaging system based on the Fermat principle as described in any one of claims 1 to 7.

Images