A non-fully symmetrical pentamirror single-viewpoint catadioptric infrared panoramic system and method

Abstract

This invention relates to an asymmetric pentamirror single-viewpoint catadioptric infrared panoramic imaging system and method. The method includes: selecting appropriate infrared imaging components and their lens parameters based on the temperature difference between the target and the background in a specific task, according to spatial resolution and range requirements; determining the precise position of the infrared imaging components; determining the specific dimensions of the asymmetric pentamirror reflector; analyzing the field of view for obstruction based on the obtained structural parameters; and optimizing the structure based on the unobstructed field of view analysis results until an unobstructed asymmetric pentamirror reflector is obtained. This invention can achieve unobstructed, seamless, and blind-zone-free infrared imaging of a 360° horizontal and ±29° pitch field of view, providing omnidirectional, high-pitch real-time panoramic infrared images for vehicles, and can be used in military and civilian fields such as intelligent transportation, autonomous driving, and military reconnaissance.

Description

Technical Field

[0001] This invention relates to the field of vehicle-mounted panoramic infrared imaging applications, and in particular to a non-fully asymmetric pentamirror single-viewpoint catadioptric infrared panoramic system and method. Background Technology

[0002] Infrared imaging is unaffected by changes in ambient light, glare, backlight, smoke, and dust. It also possesses a certain degree of resistance to light rain, snow, fog, and haze, making it a primary technology for all-weather scene perception in low-visibility conditions at night, significantly enhancing vehicle safety. In recent years, my country's infrared focal plane array detectors have made significant progress, with pixel size, dimensions, and thermal sensitivity reaching international advanced levels and holding an important position in the international market. Large-scale application of my country's low-cost infrared focal plane array detectors in vehicle driving is imminent. Infrared panoramic systems can provide 360° infrared images of the surrounding scene, eliminating blind spots for drivers. Through algorithms, they can achieve functions such as positioning, mapping, obstacle avoidance, path planning, and navigation, meeting the urgent needs of intelligent and autonomous driving.

[0003] Based on the technical approach, infrared panoramic systems can be divided into three categories:

[0004] ① Time-division multi-view infrared panoramic system. An infrared optical system and an infrared focal plane detector (referred to as infrared imaging components in this application) rotate 360° around a fixed axis perpendicular to the optical axis, or an optical scanner rotates 360° to reflect incident radiation to the infrared imaging components. Video sequences with partially overlapping content are then stitched together to obtain a panoramic image. Its advantages include low cost and good imaging quality, but low temporal resolution. Typical examples include infrared search and track systems such as the French VAMPIR system, the Israeli SPIR-TAS system, and the Dutch SIRIUS system.

[0005] ② Multi-aperture infrared panoramic system. This system integrates or distributes multiple infrared imaging components to cover a 360° azimuth field of view. The video images output by these imaging components are stitched together. Its advantages include high spatial resolution, but parallax exists for close-range targets, and ghosting occurs during stitching. A typical example is a vehicle-mounted panoramic situational awareness system, such as Leonardo DRS's DVE WIDE, which compactly integrates three infrared imaging components to achieve a large field of view of 321° (107°×3)×30°.

[0006] ③ Single-viewpoint catadioptric infrared panoramic system. This system combines a quadric mirror with a conventional refractive infrared optical system and an infrared focal plane detector. It eliminates the need for a moving scanning mechanism and achieves 360° azimuth field of view staring imaging using only a single area array detector. Its key features are a single-viewpoint structural constraint, a simple object-image mapping relationship within the panoramic field of view, and accurate target direction finding. A typical example is the infrared panoramic periscope, such as the one developed by the U.S. Naval Research Laboratory using a single 2048×2048 mid-wave infrared focal plane detector, which provides a 360° horizontal field of view and a -10° to +30° elevation field of view. However, due to the nonlinearity of the quadric mirror, the detection range of such systems varies with the elevation angle, and the detection range is closely related to the size of the single focal plane detector.

[0007] Currently, researchers abroad have developed single-viewpoint constrained catadioptric systems operating in the visible light band using tetrahedrons, pentahedrons, and decahedrons. These systems share the following characteristics: the multifaceted mirrors form a symmetrical regular polyhedron with a base angle of 45°; each mirror corresponds to the same focal length of the camera lens (i.e., equal field of view); and each camera viewpoint has the same vertical distance from the base and the same horizontal distance to the central axis. The camera viewpoint is either the center of the lens (when the lens is thin) or the object-side principal point of the lens (when the lens is a lens group). The essence of the symmetrical multifaceted single-viewpoint constrained catadioptric structure is to use regular polyhedron mirrors to superimpose the virtual images formed by multiple camera viewpoints with the same focal length onto a single point.

[0008] Taking a fully symmetrical pentamirror as an example, such as Figure 1 As shown in (a), P1, P2, P3, P4, and P5 are simplified camera viewpoints using a pinhole imaging model, with the viewpoint direction vertically downwards. Each facet corresponding to a viewpoint is an outward-facing plane mirror, forming a 45° angle with the horizontal plane. All viewpoints lie on the intersection of the horizontal plane containing the vertex of the pyramid and the vertical plane containing the angle bisectors of the corresponding facets. The virtual viewpoints obtained by imaging viewpoints in different directions through the plane mirrors coincide at the same point, making point P' the unique viewpoint of the system. Assuming the observer is located at point P', they can observe a seamless panoramic image of the surroundings from that point through multiple cameras with different imaging directions. Figure 1 (b) is Figure 1 (a) is a frontal projection of two adjacent plane mirrors. The virtual image points of the two viewpoints P1 and P2, which are symmetrical about the mirror axis, are located on the vertical line OO' between the vertex O' and the bottom surface, and coincide at point P'. Single viewpoint constraint can be achieved by constructing the constraint relationship between the plane mirror and the camera.

[0009] Tonbo Imaging, Inc. of the United States developed the WolfPack, a dual-band segmented aperture multi-view panoramic system using nine sets of low-light CMOS components and uncooled infrared imaging components. Compared with this segmented aperture, multi-view panoramic system, the segmented aperture, single-view panoramic system constructed by introducing a reflective surface has the following advantages:

[0010] (1) It can effectively eliminate parallax between different cameras;

[0011] (2) It can directly obtain a 360° horizontal field of view image without image stitching;

[0012] (3) It can ensure that the objects and images in the horizontal 360° field of view correspond one-to-one, and the image of the same object is unique. There is no "cross-view tracking" problem, which improves the credibility of the target in the panoramic image. It is not only conducive to the identification and tracking of the target of interest, but also conducive to the quantitative measurement of the target's direction and motion state.

[0013] Based on the aforementioned existing technologies, it is worthwhile to study how to combine the advantages of multi-viewpoint panoramic vision and single-viewpoint catadioptric panoramic vision, and to improve the panoramic vision system in light of the actual situation in vehicle applications where the requirements for forward detection distance are high and the requirements for rear detection distance are relatively low. Summary of the Invention

[0014] To address the aforementioned issues, this invention is designed for vehicle-mounted panoramic infrared imaging applications. It combines the advantages of multi-aperture panoramic imaging and single-aperture catadioptric panoramic imaging. Considering the practical requirements of high forward detection distance and relatively low rear detection distance in vehicle applications, this invention proposes a non-fully asymmetric pentamirror single-aperture catadioptric infrared panoramic imaging system and method. This is a design scheme for a multi-aperture, single-aperture, non-fully asymmetric pentamirror infrared panoramic imaging system.

[0015] The specific technical solution of the present invention is as follows:

[0016] A design method for an asymmetric pentagonal frustum reflector includes the following steps:

[0017] Step (1) Select appropriate infrared imaging components and their lens parameters based on the temperature difference between the target and the background in a specific task, according to the requirements of spatial resolution and effective distance;

[0018] Step (2) Based on the height m of the virtual viewpoint P', the distance d from the center point of the bottom surface to the side length, the height s of the trimmable mirror, the height h of the equivalent viewpoint P of the imaging component, the horizontal distance l from the structure vertex O', the vertical field of view angle 2ε of the imaging component, and the lens diameter k, select a suitable combination of viewpoint height m and mirror tilt angle θ. By solving the height h of the equivalent viewpoint P of the infrared imaging component and the horizontal distance l from the structure vertex O', determine the precise position of the infrared imaging component. Then, based on the reflected field of view, determine the distance d from the bottom center point of the structure to the side length and the height s of the trimmable mirror, thereby determining the specific dimensions of the non-fully symmetrical pentagonal reflector.

[0019] Step (3) Analyze whether the field of view is obstructed based on the structural parameters obtained in step (2);

[0020] Step (4) Optimize the structure based on the unobstructed field of view analysis results until an unobstructed asymmetric pentagonal frustum reflector is obtained.

[0021] Further, in step (2), based on different combinations of mirror tilt angle θ and virtual viewpoint height m, the horizontal distance l from the central axis and the vertical height h from the bottom surface of the infrared imaging component are determined, expressed as:

[0022] l = 2(dtanθ - m)sinθcosθ;

[0023] h=2dsinθcosθ+(sin 2 θ-cos 2 θ)m;

[0024] The expression for the cuttable mirror height s is:

[0025]

[0026] Furthermore, in step (3), the unobstructed analysis includes:

[0027] When calculating the current parameters, the infrared imaging component does not compare the vertical field of view ε1' and ε2' during its own imaging with the actual vertical field of view ε. If ε1' < ε < ε2', then unobstructed imaging can be achieved under this structure; where:

[0028]

[0029] in:

[0030]

[0031] This invention also relates to the application of an asymmetric pentaprism reflector, which uses the aforementioned asymmetric pentaprism reflector satisfying the single-viewpoint constraint and infrared lenses with different focal lengths to reflect and converge scene radiation from five directions onto five sets of vertically placed infrared focal plane detectors. The optical center of the infrared lens coincides with the center of the infrared focal plane detector, so the image center point (C) is... x C y )satisfy

[0032]

[0033] When the image coordinates (x, y) are projected onto a cylindrical surface, let the coordinates be (x', y').

[0034] Conversion formulas for front and left / right field of view focal length f1 = 5.8mm:

[0035]

[0036] Conversion formulas for left and right rear field of view focal length f2 = 4.1mm:

[0037]

[0038] The present invention also relates to an asymmetric pentagonal frustum reflector, which is obtained according to the above-described design method.

[0039] Furthermore, it includes a disk platform, a mirror reflection area, an infrared imaging component, and a central column axis;

[0040] Several infrared imaging components are fixed on a disc platform, maintaining a certain horizontal and vertical distance from the central column axis, and the specular reflection areas are arranged around the central column axis;

[0041] The mirror reflection area is separate, including the mirror support and the mirror surface; or, the mirror reflection area is integral, which is processed by integral molding and then coated with a high reflectivity film.

[0042] Several reflective mirrors correspond to several infrared imaging components and are at a certain angle; each infrared imaging component collects the infrared radiation reflected by the corresponding reflective mirror, which together form a 360° infrared panoramic image.

[0043] Furthermore, different numbers of first adjustment rings are placed on the central column axis to adjust the height of several infrared imaging components; different numbers of second adjustment rings are placed at the connection point between a single infrared imaging component and the disk platform to adjust the vertical height of the single infrared imaging component.

[0044] The present invention also relates to an asymmetric pentahedral mirror single-viewpoint catadioptric infrared panoramic system, comprising the aforementioned asymmetric pentagonal reflector.

[0045] This invention also relates to the application of an asymmetric pentagonal mirror single-viewpoint catadioptric infrared panoramic system, which uses the aforementioned asymmetric pentagonal mirror to obtain a 360° panoramic field of view.

[0046] The present invention also relates to an image processing method, which obtains an image based on the above-mentioned asymmetric pentamirror single-viewpoint catadioptric infrared panoramic system, comprising the following steps:

[0047] First, the image at one focal length is scaled, preserving the front, left, and right field-of-view information. After cylindrical projection, the image at one focal length is scaled and sampled. The scaling factor γ is determined by the vertical field-of-view angles ε1 and ε2 of the infrared imaging component corresponding to the first and second focal lengths, as shown in the formula:

[0048]

[0049] To obtain a seamless panoramic infrared image, the image at the second focal length is magnified and then aligned with the image at the first focal length. The redundant field of view is then matched and cut to obtain a seamless panoramic infrared image.

[0050] Perform uniform grayscale balance on the panoramic infrared images; take infrared images acquired simultaneously in five directions and calculate their mean μ for each. i (i = 1, 2, 3, 4, 5) and variance σ i (i = 1, 2, 3, 4, 5, to obtain the average mean μ of these 5 infrared images) average Mean variance σ average The expression is as follows:

[0051]

[0052] For each of the 5 frames I i Perform grayscale balancing on (i = 1, 2, 3, 4, 5), and the output result is O. i (i = 1, 2, 3, 4, 5), the expression is:

[0053]

[0054] Drivers mostly ensure safe driving by observing the road ahead and to the left and right. Therefore, the forward-looking and left and right-looking cameras in the panoramic system are required to have a long working distance so that they can perceive road information as early as possible and thus quickly perform obstacle avoidance operations. The working distance requirement for the rear-view camera is relatively low.

[0055] This invention combines the advantages of multi-aperture panoramic vision and single-aperture catadioptric panoramic vision. Considering the practical requirements of high forward detection distance and relatively low rearward detection distance in vehicle applications, it proposes a multi-aperture, single-aperture, asymmetric pentamirror infrared panoramic vision system design. The forward and left / right side fields of view are both 64°, while the rear has two 84° fields of view, collectively forming a 360° horizontal and ±29° pitch panoramic field of view. For uncooled infrared imaging components with different focal lengths, the structural design of the asymmetric pentamirror is completed, and a theoretical model of the asymmetric catadioptric panoramic vision structure satisfying single-aperture constraints is established. An adjustable, viewpoint-aligning system mechanical structure is designed, and the system's projection conversion and image processing elements are analyzed to improve its comprehensiveness, realism, and reliability. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of a single viewpoint constraint for a fully symmetrical pentamirror mirror in the prior art; where (a) represents a fully symmetrical pentamirror mirror; and (b) is a front projection view of an adjacent plane mirror.

[0057] Figure 2 These are the MRTD calculation results of this invention embodiment; where (a) represents the detection distance of infrared lenses with focal lengths of 4.1mm, 5.8mm, and 9.1mm at a 50% detection probability; and (b) represents the horizontal field of view corresponding to infrared lenses with focal lengths of 4.1mm, 5.8mm, and 9.1mm.

[0058] Figure 3 This is a schematic diagram of the field of view configuration of the system according to an embodiment of the present invention;

[0059] Figure 4 This is a schematic diagram of the geometric structure of the non-fully asymmetric pentahedron according to an embodiment of the present invention;

[0060] Figure 5 This is a schematic diagram of the parameter definition of the non-fully symmetrical pentamirror structure according to an embodiment of the present invention; wherein, (a) is a three-dimensional view; and (b) is a two-dimensional side projection view.

[0061] Figure 6 This is an example of unobstructed imaging analysis in an embodiment of the present invention;

[0062] Figure 7 This is a design flowchart of the non-fully asymmetric pentamirror structure according to an embodiment of the present invention;

[0063] Figure 8 is a schematic diagram of the transformation from planar projection to circumferential view image according to an embodiment of the present invention; wherein, (a) is a planar image in five directions; and (b) is a circumferential view image after cylindrical projection.

[0064] Figure 9 This is a schematic diagram of the structure of one type of non-symmetrical pentamirror according to an embodiment of the present invention;

[0065] Figure 10 This is a partial structural schematic diagram of one embodiment of the non-symmetrical pentamirror of the present invention;

[0066] Figure 11 This is a diagram showing the positional relationship between the central axis of a non-symmetrical pentamirror and the mirror support in one embodiment of the present invention.

[0067] Figure 12 This is a schematic diagram of the central principal axis of one embodiment of a non-fully symmetrical pentamirror mirror according to the present invention;

[0068] Figure 13 This is a schematic diagram of the imaging results of an embodiment of the present invention. Detailed Implementation

[0069] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] Unless otherwise defined, the technical or scientific terms used in the embodiments of this application shall have the ordinary meaning understood by one of ordinary skill in the art. The terms "first," "second," and similar terms used in this embodiment do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Installed," "connected," and "linked" should be interpreted broadly; for example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two elements. Terms such as "upper," "lower," "left," "right," "horizontal," and "vertical" are used only relative to the orientation of the components in the accompanying drawings. These directional terms are relative concepts used for relative description and clarification, and they may change accordingly depending on the orientation of the components in the accompanying drawings.

[0071] Example 1

[0072] The asymmetric pentamirror single-viewpoint catadioptric infrared panoramic system design of this embodiment is as follows:

[0073] First, select an appropriate infrared camera based on the pedestrian detection distance while the vehicle is driving. Assume the pedestrian's dimensions are 0.5 × 1.7 m. 2With a ambient temperature of 333K, a background temperature of 298K, an infrared detector pixel size of 640×512, a pixel size of 12μm, and a noise equivalent temperature difference (NETD) of 40mK, the curves of the equivalent blackbody temperature difference ΔT' between the pedestrian and the background and the distance R, calculated using the minimum resolvable temperature difference (MRTD) formula, are as follows: Figure 2 As shown in (a), the detection distances of the 4.1mm, 5.8mm, and 9.1mm focal length infrared lenses are 145m, 200m, and 320m, respectively, with corresponding horizontal field of view angles of 86°, 67°, and 46°. Figure 2 As shown in (b). The MRTD calculation process is as follows:

[0074] Infrared imaging technology can effectively detect surrounding objects when vehicles are driving at night or in low visibility conditions. Using pedestrians as the detection target, the detection range of infrared imaging components is analyzed, and appropriate infrared imaging component parameters are designed or selected based on actual detection needs.

[0075] In infrared imaging systems, MRTD is used to evaluate the system's temperature resolution and spatial resolution. The expression for MRTD is:

[0076]

[0077] In equation (1), NETD is the noise equivalent temperature difference, and SNR is... TH The threshold signal-to-noise ratio at which the observer can distinguish the strips, α and β are the instantaneous field of view angles in the horizontal and vertical directions, and t is the threshold signal-to-noise ratio at which the observer can distinguish the strips. e Human eye integration time, f p For frame rate, τ d Let Δf be the dwell time and Δf be the noise equivalent bandwidth. MTF is the modulation transfer function of the infrared optoelectronic system. Here, we mainly consider the diffraction effect in the optical system, so the MTF of the system can be expressed as:

[0078]

[0079] Where f c =D / λ, representing the spatial cutoff frequency of the optical system, where D is the effective aperture of the optical system, and λ is the average operating wavelength. Let the radiative exitance M represent the radiant power emitted per unit surface area of ​​the radiating target into the hemispherical space. Then the radiative exitance of a gray body with emissivity ε and temperature T is:

[0080]

[0081] In equation (3), c1 = 3.7418 × 10 8 W·μm 4 / m 2 c2 = 1.4388 × 10 4μm·K. Infrared radiation is affected by atmospheric absorption and scattering. Changes in radiant energy cause temperature changes, which are functions of distance R. The transmittance of infrared radiation through the atmosphere can be expressed as:

[0082] τ(R)=exp(-p·R) (4)

[0083] Where p is the atmospheric attenuation coefficient, which is related to the wavelength of infrared radiation, geographical location, atmospheric pressure, temperature, humidity, season, and climate conditions. The radiant exitance M'(T') of the radiation target after atmospheric transmission is:

[0084] M'(T')=τ(R)·M(T) (5)

[0085] In practical applications, the background temperature is usually taken in the range of 220K to 320K. For the infrared band of 8 to 12μm, c2 >> λT, and formula (5) can be expressed as:

[0086]

[0087] Assume the target temperature is T t The background temperature is T. b Then the equivalent blackbody temperature difference ΔT' between the target and the background reaching the detector is:

[0088] ΔT'=T t '-T b (7)

[0089] Among them, T t '、T b ' represents the equivalent blackbody temperature of the target and background radiation after atmospheric transmission, respectively. When the target width is W and the height is H, the number of equivalent line pairs n can be determined according to the Johnson criterion. e The target is divided into sections with a width of H / 2n. e The line pairs correct the target aspect ratio in the MRTD formula for range performance, resulting in the corrected MRTD. target for:

[0090]

[0091] Where γ is the aspect ratio of the target equivalent strip, expressed as:

[0092]

[0093] At a detection range R, the target's limiting spatial frequency f T It can be represented as:

[0094]

[0095] Let the effective range of the infrared system be R. The energy radiated by the target and the background will be affected by atmospheric attenuation, and there is a relationship curve ΔT'(R) between the equivalent temperature difference between the two and the effective range R. The MRTD is corrected based on the detection probability, observation level, and actual target size. target (f) can be used to determine the corresponding limiting spatial frequency f. T Thus, a function of R is obtained, which, combined with ΔT'(R), can be used to solve for the effective range of the infrared system.

[0096] like Figure 2 As shown, based on the above calculations, to obtain a 360° panoramic field of view using five asymmetric mirrors, the forward and left / right views are both 64° (focal length 5.8mm), and the back view consists of two 84° fields of view (focal length 4.1mm). The field of view configuration is as follows: Figure 3 As shown.

[0097] Traditional vehicle vision systems have blind spot detection areas, primarily the A-pillar blind spot caused by the obstruction of the windshield by the tilted A-pillars on both sides (see...). Figure 3 Blind Spot I and blind spots in rearview mirrors (see Figure 3 The system designed in this embodiment can effectively eliminate these blind spots (Blind Spot II). The parameters of the infrared detector components selected for the system are shown in Table 1.

[0098] Table 1 Performance parameters of the infrared imaging components used in this embodiment.

[0099]

[0100]

[0101] Based on detection distance analysis and spatial resolution requirements, an infrared imaging component was selected, and a non-fully symmetrical pentamirror structure was designed according to single-viewpoint constraints. Three sets of equivalent viewpoints with focal length f1 = 5.8mm are denoted as P1, P2, and P3, with corresponding facets forming an angle α with the horizontal plane – the mirror tilt angle. Two sets of equivalent viewpoints with focal length f2 = 4.1mm are denoted as P4 and P5, with corresponding facets forming a mirror tilt angle β with the horizontal plane. The different focal length lens combinations result in the bottom of the pentamirror structure exhibiting a non-fully symmetrical pentagonal shape with equal distances from the vertices to the center, but with side lengths and interior angles not entirely equal. Furthermore, the infrared imaging component is positioned at different heights in the vertical direction, such as… Figure 4 As shown. The key design point of the single-viewpoint constrained non-symmetrical pentamirror structure is to adjust the spatial position of different mirror tilt angle combinations (α, β) and infrared imaging components so that the virtual images formed by viewpoints P1, P2, P3, P4, and P5 coincide at point P'.

[0102] To facilitate the calculation of the structural parameters of the non-fully symmetric pentahedron under single-viewpoint constraints, we selected... Figure 4We will analyze an infrared imaging component in one direction. Without loss of generality, let the equivalent viewpoint of this imaging component be P, the corresponding mirror tilt angle be θ, and other structural parameters be defined as follows: Figure 4 , 5 As shown, and listed in Table 2, including: the height m of the virtual viewpoint P', the distance d from the center point of the bottom surface to the side length, and the height s of the cuttable mirror surface (…). Figure 5 (b) The height above the red dashed line, the height h of the equivalent viewpoint P of the imaging component, the horizontal distance l from the structural vertex O′, the vertical field of view 2ε of the imaging component, and the lens diameter k.

[0103] Table 2 Structural parameters of the asymmetric pentamirror

[0104]

[0105] The overall dimensions of the pentamirror structure are determined by the mirror tilt angle θ, the distance d from the bottom center point to the side length, and the height m of the virtual viewpoint. Assume the left boundary ray of the vertical field of view angle 2ε is located exactly at point C at the bottom of the structure, and the mirror prism corresponding to viewpoint P is... Figure 5 In (a), the plane ABO', the side projection of the reflecting surface is... Figure 5 CO' in (b). Based on different combinations of mirror tilt angle θ and virtual viewpoint height m, the horizontal distance of the infrared imaging component from the central axis (approximately equal to the horizontal distance l between the equivalent viewpoint P and the structural vertex O') and the vertical height from the bottom surface (approximately equal to the height h of the equivalent viewpoint P) can be determined, expressed as:

[0106] l=2(dtanθ-m)sinθcosθ (11)

[0107] h=2dsinθcosθ+(sin 2 θ-cos 2 θ)m (12)

[0108] Assuming that when equations (1) and (2) are satisfied, the right boundary ray of the vertical field of view intersects the reflecting surface at point Q. The mirror region above the horizontal plane where point Q is located, i.e. Figure 5 (b) The area above the red dashed line does not participate in imaging and can be cropped in actual use. Therefore, the practically usable structure is a non-fully symmetrical pentagonal frustum, while the plane mirror is an isosceles trapezoid. The expression for the cropped mirror height s is:

[0109]

[0110] If the mirror tilt angle θ is too small or the virtual viewpoint height m is too high, the image of the reflecting prism imaging component itself will obstruct the imaging of the surrounding scene. If the mirror tilt angle θ is too large or the virtual viewpoint height m is too low, the size of the prism needs to be increased to ensure the integrity of the imaging. Therefore, it is necessary to perform lens-free imaging analysis to compromise and adjust the values ​​of θ and m.

[0111] Based on the analysis of light reflection in geometric optics, the critical angle of the field of view when the infrared imaging component just does not image itself is as follows: Figure 6 As shown, let the diameter k of the infrared imaging component lens corresponding to the equivalent viewpoint P be a line segment MN, and the critical incident ray I ( Figure 6 Ray I just passes through the lens boundary point M, and is reflected at point Q at an incident angle φ1. The reflected ray passes through the lens boundary point N and enters the infrared imaging component; the critical incident ray II ( Figure 6 Ray II is reflected at point C at an incident angle φ2. The reflected ray passes through the lens boundary point M and enters the infrared imaging component. Under these conditions, let the angle ∠MPN = 2ε'.

[0112] Based on geometric relationships, we have φ1 = θ - ε' and φ2 = θ + ε'. At this point, the following geometric relationship also holds:

[0113]

[0114] According to formula (4), we can solve for two values ​​of ε', let them be ε1' and ε2' respectively, and get:

[0115]

[0116] in

[0117]

[0118] When calculating the current parameters according to formulas (5) and (6), the infrared imaging component does not compare the vertical field of view ε1' and ε2' when it is imaging itself with the actual vertical field of view ε (values ​​are shown in Table 1). If ε1' < ε < ε2', then the system under this structure can achieve unobstructed imaging.

[0119] In summary, such as Figure 7 As shown, the main steps for designing a non-fully symmetrical pentamirror structure based on single-viewpoint constraints include:

[0120] (1) For the temperature difference between the target and the background in a specific task, select appropriate infrared imaging components and their lens parameters according to the requirements of spatial resolution and effective distance, as shown in Table 1.

[0121] (2) Select a suitable combination of viewpoint height m and mirror tilt angle θ based on the vertical field of view 2ε, lens diameter k, etc. Determine the precise position of the infrared imaging component by solving the height h of the equivalent viewpoint P of the infrared imaging component and the horizontal distance l from the top of the structure O'. Then, determine the distance d from the bottom center point of the structure to the side length and the height s of the cuttable mirror based on the reflected field of view, thereby determining the specific dimensions of the non-fully symmetrical pentagonal reflector.

[0122] (3) Analyze whether the field of view is obstructed based on the system structure parameters obtained in the previous step;

[0123] (4) Optimize the system structure based on the unobstructed field of view analysis results until an unobstructed, small-sized, and easy-to-manufacture asymmetric pentagonal reflector is obtained.

[0124] The non-symmetrical pentamirror and infrared lenses with different focal lengths satisfy the single viewpoint constraint radiate and reflect the scene in five directions and converge it to five sets of vertically placed infrared focal plane detectors, as shown in Figure 8. Since the axis where the virtual viewpoint P' is located can be regarded as the rotation axis, cylindrical projection is used to convert the planar images in the five directions of Figure 8(a) into seamless panoramic images, as shown in Figure 8(b).

[0125] In Figure 8, the solid lines and dashed lines represent infrared imaging components with focal lengths of f1 = 5.8 mm and f2 = 4.1 mm, respectively.

[0126] In addition, to preserve as much frontal and left / right field of view information as possible, a focal length of f1 = 5.8 mm was used as the cylindrical projection radius for circumferential image projection. Simultaneously, three images with a focal length of f1 = 5.8 mm and two images with a focal length of f2 = 4.1 mm can also be saved as large field-of-view images and made available to users.

[0127] To simplify the description, let the width of one image in Figure 8(a) be W and the height be H, and let the optical center of the infrared lens coincide with the center of the infrared focal plane detector. Then the center point of the image (C) x C y )satisfy

[0128]

[0129] When the image coordinates (x, y) are projected onto the cylinder in Figure 8(b), let the coordinates be (x', y'). Then, the transformation formula from coordinates (x, y) to coordinates (x', y') needs to be discussed in two cases:

[0130] (1) Conversion formulas for front and left / right field of view focal length f1 = 5.8mm:

[0131]

[0132] (2) Conversion formulas for left and right rear field of view focal length f2 = 4.1mm:

[0133]

[0134] As a specific implementation method, this embodiment utilizes ProE software to simulate and design the system, proposing an adjustable and alignable mechanical structure scheme to ensure that the viewpoints of infrared imaging components with different focal lengths can accurately coincide at the same point. Figure 9 , 10 As shown in Figures 11 and 12, the panoramic system includes a disc platform 1, a mirror reflection area 6, an infrared imaging component 3, and a central column axis 2.

[0135] The infrared imaging component 3 is fixed to the circular platform 1 at a certain horizontal and vertical distance from the central column axis 2. The mirror reflection areas 6 are arranged around the central column axis and are separate, mainly composed of a reflector bracket and a reflector surface. Planar brackets are added in the horizontal and vertical directions to ensure the stability and accuracy of the tilt angle, and grooves are engraved on the bracket surface to ensure proper fit between the two. The reflector surface is composed of float glass with a certain thickness, uniform internal structure, and a smooth, flat front surface, which is beneficial for the mirror reflection of infrared radiation. Each infrared imaging component 3 collects the infrared radiation reflected by its corresponding reflector surface, collectively forming a 360° infrared circumferential image.

[0136] To ensure single-viewpoint constraint by aligning multiple viewpoints with a single virtual viewpoint, a mechanical structure with adjustable spatial position of the infrared imaging component can be designed.

[0137] On one hand, different numbers of first adjustment rings 4 are placed on the central column shaft 2 to adjust the height, and are fixed by locking rings; on the other hand, the infrared imaging component 3 is positioned on the disk using a fixed base assembly (e.g., bolts, screws, etc.), which can move within a certain range to adjust the horizontal distance of the infrared imaging component, and different numbers of second adjustment rings 5 ​​are placed below to adjust the vertical height. The adjustment mechanism of the central column shaft 2 and the fixed base adjustment mechanism ensure that the spatial position of the infrared imaging component can be manually adjusted, avoiding viewpoint position offset caused by structural processing and manual assembly errors. The final structural parameters of the system after mechanical design in this embodiment are shown in Table 3.

[0138] Table 3. Theoretical and actual values ​​of system structural parameters in this embodiment.

[0139]

[0140]

[0141] After processing, assembly, and debugging, the physical object is obtained. When the system is working, the five infrared imaging components simultaneously acquire video images according to the external synchronization signal, and the size of one frame image is 640×480.

[0142] Since this embodiment uses infrared lenses with two focal lengths (f1 = 5.8mm and f2 = 4.1mm), the image at one of the focal lengths first needs to be scaled. To preserve as much frontal and left / right field-of-view information as possible, the image at f2 = 4.1mm is magnified and upsampled after cylindrical projection. The magnification factor γ is determined by the vertical field-of-view angles ε1 and ε2 of the infrared imaging components corresponding to the focal lengths f1 = 5.8mm and f2 = 4.1mm, and the formula is:

[0143]

[0144] From formula (10), the amplification factor γ = 1.41 is obtained.

[0145] To obtain a seamless panoramic infrared image, the redundant fields of view in the horizontal direction of the video images with focal lengths f1 = 5.8mm and f2 = 4.1mm are 6.5% (640 × 6.5% ≈ 42 pixels) and 8.6% (640 × 8.6% ≈ 55 pixels), respectively. Therefore, after the image with focal length f2 = 4.1mm is magnified, it is aligned with the image with focal length f1 = 5.8mm, and then the redundant fields of view are matched and cut to obtain a seamless panoramic infrared image.

[0146] Due to variations in exposure and automatic gain, a uniform grayscale balance needs to be applied to the panoramic infrared images. Infrared images acquired simultaneously from five directions are used, and their mean values ​​(μ) are calculated for each. i (i = 1, 2, 3, 4, 5) and variance σ i (i = 1, 2, 3, 4, 5), the average mean μ of these 5 infrared images is obtained. average Mean variance σ average The expression is as follows:

[0147]

[0148] Using this as a reference, five frames of images I were analyzed respectively. i (When i = 1, 2, 3, 4, perform grayscale balancing; the output result is O) i (i = 1, 2, 3, 4, the expression is:

[0149]

[0150] In summary, the image processing steps required to acquire the infrared circumferential image in this system mainly include: cylindrical projection, scaling, center alignment, redundant part cutting, and grayscale balancing, ultimately obtaining a complete and seamless infrared circumferential image, such as... Figure 13As shown, the infrared imaging components in front and on the left and right have a long working distance, expanding the driver's field of vision and meeting the need for all-day, blind-spot-free observation.

[0151] Therefore, this embodiment addresses the different pedestrian detection distance requirements in vehicle driving applications (200m and 145m respectively) for the front, left, right, and rear views. It proposes and implements a non-fully asymmetric pentamirror single-viewpoint catadioptric infrared panoramic system with a 64° field of view for the front, left, and right views, and two 84° fields of view for the rear view. By using the non-fully asymmetric pentamirror to coincide the virtual viewpoints of three sets of infrared imaging components with focal lengths of f1=5.8mm and two sets of f2=4.1mm into the same point, a design process for a single-viewpoint constrained non-fully asymmetric pentamirror structure is established. That is, after selecting appropriate infrared imaging components and their lens parameters according to the spatial resolution and effective distance requirements, the specific dimensions of the non-fully asymmetric pentagonal reflector are determined according to the single-viewpoint structural constraints. After conducting field-of-view occlusion analysis based on the system structural parameters, the system structure is further optimized until an unobstructed, small-sized, and easily manufactured non-symmetric pentagonal reflector is obtained. After completing the fabrication and assembly of the asymmetric five-mirror catadioptric infrared panoramic prototype system, a panoramic infrared image processing workflow was proposed, including steps such as cylindrical projection, scaling, center alignment, redundant part cutting, and grayscale balancing. This ultimately achieves unobstructed, seamless, and blind-spot-free infrared imaging of a 360° horizontal field of view and a ±29° elevation field of view. The system acquires comprehensive, realistic, and reliable infrared images of the scene, helping to eliminate blind spots for vehicle drivers, improve intelligent driving capabilities, and has broad application prospects in both civilian and military fields.

[0152] Example 2

[0153] In this embodiment, the specular reflection area of ​​the panoramic system is integral, manufactured using a monolithic molding method and then coated with a high-reflectivity thin film. The rest is the same as in Embodiment 1.

[0154] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A design method for a single-viewpoint constrained asymmetric pentagonal frustum reflector, characterized in that: Includes the following steps: Step (1) Constructing the constraint relationship between the plane mirror and the imaging components to achieve single-viewpoint constraint: For the temperature difference between the target and the background in a specific task, five infrared imaging devices and two lenses with two different focal lengths are selected to form five sets of infrared imaging components with two focal lengths, based on the requirements of spatial resolution and effective distance. The equivalent viewpoint of all infrared imaging components is determined. P 1 、P 2 、P 3 、P 4 、P 5. The virtual images formed by corresponding reflecting surfaces of the asymmetric pentagonal mirror coincide at the same virtual viewpoint. This satisfies the single-viewpoint constraint. To avoid loss of generality, we will select one infrared imaging component from one direction for analysis, and let the equivalent viewpoint of this imaging component be... P The corresponding mirror tilt angle is θ Other structural parameters are defined as virtual viewpoint height. m virtual viewpoint The height of the asymmetric pentagonal reflector from the bottom surface and the distance from the center point of the bottom surface to the side length. d Adjustable mirror height s That is, the top of the pentagonal pyramid does not participate in the actual imaging and can be clipped to become the height of the pentagonal frustum and the equivalent viewpoint of the imaging component. P Horizontal distance from the vertex of the pentagonal pyramid l Vertical field of view of the imaging component Lens diameter k； Step (2) Based on the virtual viewpoint high m Distance from the center point of the base to the side length d Adjustable mirror height s Equivalent viewpoint of imaging components P height h and structural vertices horizontal distance l Vertical field of view of the imaging component Lens diameter k Select an appropriate viewpoint height m Inclination of the mirror θ The combination of these is achieved by solving the equivalent viewpoint of the infrared imaging component. P height h and structural vertices horizontal distance l This is used to determine the precise location of the infrared imaging component, and then the distance from the center point of the bottom of the structure to the side length is determined based on the reflected field of view. d Adjustable mirror height s This allows for the determination of the specific dimensions of the non-fully asymmetric pentagonal frustum reflector. Step (3) Analyze whether the field of view is obstructed based on the structural parameters obtained in step (2); Step (4) Optimize the structure based on the unobstructed field of view analysis results until an unobstructed asymmetric pentagonal frustum reflector is obtained.

2. The design method according to claim 1, characterized in that: In step (2), according to different mirror tilt angles θ and virtual viewpoint height m Combined, determine the horizontal distance of the infrared imaging component from the central axis. l and vertical height from the bottom surface h The expression is: ； ； Adjustable mirror height s The expression is: ； In step (3), the unobstructed analysis includes: The vertical field of view when the infrared imaging component is not imaging itself during the calculation of the current parameters. , , and the actual vertical field of view To make a comparison, if This structure enables unobstructed imaging; where: ； in: 。 3. An application of a single-viewpoint constrained asymmetric pentagonal frustum reflector, characterized in that: The asymmetric pentaprism reflector and infrared lenses with different focal lengths, satisfying the single-viewpoint constraint as described in any one of claims 1-2, radiate and reflect the scene from five directions and converge it onto five sets of vertically placed infrared focal plane detectors. The optical center of the infrared lens coincides with the center of the infrared focal plane detector, thus the image center point... satisfy Image coordinates When the projection is a cylindrical surface, let the coordinates be... W and H are the width and height of the image, respectively; Front and left and right field of view focal length Time conversion formula: ； Left rear and right rear field of view focal length Time conversion formula: 。 4. A single-viewpoint constrained asymmetric pentagonal frustum reflector, characterized in that: Obtained by the design method according to any one of claims 1-2.

5. The single-viewpoint constrained asymmetric pentagonal frustum reflector according to claim 4, characterized in that: Includes a disc platform, a mirror reflection area, an infrared imaging component, and a central column axis; Several infrared imaging components are fixed on a disc platform, maintaining a certain horizontal and vertical distance from the central column axis, and the specular reflection areas are arranged around the central column axis; The mirror reflection area is separate, including the mirror support and the mirror surface; or, the mirror reflection area is integral, which is processed by integral molding and then coated with a high reflectivity film. Several reflective mirrors correspond to several infrared imaging components and are at a certain angle; each infrared imaging component collects the infrared radiation reflected by the corresponding reflective mirror, which together form a 360° infrared panoramic image.

6. The single-viewpoint constrained asymmetric pentagonal frustum reflector according to claim 5, characterized in that: Different numbers of first adjustment rings are placed on the central column axis to adjust the height of several infrared imaging components; different numbers of second adjustment rings are placed at the connection between a single infrared imaging component and the disk platform to adjust the vertical height of the single infrared imaging component.

7. A non-fully symmetrical pentamirror single-viewpoint catadioptric infrared panoramic system, characterized in that: Including the single-viewpoint constrained asymmetric pentagonal frustum reflector as described in any one of claims 4-6.

8. An application of a non-fully symmetrical pentamirror single-viewpoint catadioptric infrared panoramic system, characterized in that: A 360° panoramic field of view can be obtained by using the single-viewpoint constrained asymmetric pentagonal reflector as described in any one of claims 4-6.

9. An image processing method, characterized in that: The image acquisition method of the asymmetric pentamirror single-viewpoint catadioptric infrared panoramic system according to claim 8 includes the following steps: First, the image for one focal length is scaled, preserving the front and left / right field of view information. After cylindrical projection, the image for one focal length is scaled and sampled, with a scaling factor... Vertical field of view of the infrared imaging component corresponding to the first and second focal lengths , The decision is made using the following formula: ； To obtain a seamless panoramic infrared image, the image at the second focal length is magnified and then aligned with the image at the first focal length. The redundant field of view is then matched and cut to obtain a seamless panoramic infrared image. Perform uniform grayscale balance on the panoramic infrared images; take infrared images acquired simultaneously in five directions and calculate their mean values ​​for each. With variance The average value of these 5 infrared images was obtained. Mean variance The expression is as follows: Five frames of images respectively Perform grayscale balance, and the output result is: The expression is: 。

Images