A surround view observation method based on wide-angle imaging microelements

Abstract

This invention discloses a panoramic observation method based on wide-angle imaging micro-elements, comprising: designing the number, width, longitudinal and transverse curvature, and tilt angle of the wide-angle imaging micro-elements; designing the radius, thickness, and curvature range of the lens based on the reflected light path and object distance to ensure effective light enters the lens; selecting a set of lens parameters, determining the optimal observation position in conjunction with the camera focal length, and acquiring the clearest micro-element image; mapping, segmenting, and correcting geometric distortion of the image; finally, sorting the images according to the segmentation angle and playing them continuously according to a preset panoramic time to achieve highly continuous three-dimensional panoramic observation of the target; this invention can achieve parameterized matching between the optical system and observation requirements, ensuring the stability and continuity of the panoramic imaging acquisition, and outputting a visually continuous high-fidelity three-dimensional panoramic dynamic video.

Description

Technical Field

[0001] This invention relates to the fields of geometric optics, computer vision, and panoramic observation, and particularly to a panoramic observation method based on wide-angle imaging micro-elements. Background Technology

[0002] With the development of intelligent sensing technology, space exploration technology and complex physical field observation technology, the demand for three-dimensional panoramic observation of target objects with large field of view, high continuity and high spatial resolution is increasing. Especially in complex space target observation scenarios, such as plasma morphology observation such as electric arc, analysis of discharge channel evolution process, and spatial reconstruction of complex curved surface targets, higher requirements are placed on the three-dimensional spatial structure, dynamic evolution process and spatial continuity expression ability of the target.

[0003] Existing panoramic observation technologies mainly include ultra-wide-angle lens imaging, fisheye lens imaging, and multi-camera stitching imaging. Most of these technologies are based on two-dimensional imaging models and focus on expanding the field of view coverage, but lack a systematic design for continuously acquiring the three-dimensional spatial structure of the target object.

[0004] While imaging schemes based on ultra-wide-angle or fisheye lenses can achieve large field-of-view coverage on a single imaging unit, they are essentially two-dimensional projection imaging models. In complex environments, they lack the ability to represent spatial depth information and suffer from severe geometric distortion, resulting in complex spatial structure mapping relationships and making it difficult to accurately reflect the true three-dimensional spatial distribution characteristics of the target. These problems are particularly prominent in complex plasma target observation scenarios such as electric arcs, making it difficult to continuously observe the three-dimensional morphology, spatial expansion structure, and evolution process of the arc channel.

[0005] While multi-camera stitching solutions can acquire spatial information from multiple perspectives, the system structure is complex, costly, and difficult to calibrate. Multi-view data is prone to error accumulation during spatial reconstruction, and it is difficult to guarantee the time synchronization accuracy and spatial consistency in dynamically evolving targets, thus affecting the accuracy and stability of the 3D reconstruction results.

[0006] In summary, the existing panoramic observation technology system is still centered on two-dimensional imaging in terms of optical design principles, and has not been systematically designed from the fundamental perspective of three-dimensional spatial information acquisition mechanisms. As a result, it has inherent defects in terms of depth information, spatiotemporal consistency and system robustness when facing complex three-dimensional dynamic targets. Summary of the Invention

[0007] This invention provides a panoramic observation method based on wide-angle imaging micro-elements to overcome the above-mentioned technical problems.

[0008] 1. A panoramic observation method based on wide-angle imaging micro-elements, comprising: S1: Design the micro-element parameters for wide-angle imaging, which include the number of micro-elements, the width of micro-elements, the longitudinal curvature of micro-elements, the lateral curvature of micro-elements, and the tilt angle of micro-elements; S2: Based on the laws of rectilinear propagation and reflection, the reflected light path is obtained. Combining the parameters of the wide-angle imaging element and the distance between the observed object and the wide-angle imaging element, the structural parameters of the lens are designed to ensure that there are effective light rays in the reflected light path that meet the preset imaging quality requirements. The structural parameters include the radius, thickness and curvature of the lens; one side of the lens is a curved surface and the other side is a plane. S3: Select any set of structural parameters within the range of structural parameters, and design the camera observation position according to the camera's focal length so that the wide-angle imaging micro-element image in the lens is clearest under the structural parameters, and then acquire the wide-angle imaging micro-element image. S4: Process the wide-angle imaging micro-element image, including performing coordinate mapping from a ring to a plane, image segmentation, and geometric distortion correction on the wide-angle imaging micro-element image to obtain a corrected segmented image; S5: Sort the corrected images according to the segmentation angle, and continuously play the sorted images according to the preset panoramic viewing time requirements to realize panoramic observation of wide-angle imaging micro-elements.

[0009] Furthermore, the design of the number of infinitesimal elements, the width of the infinitesimal elements, the longitudinal curvature of the infinitesimal elements, and the transverse curvature of the infinitesimal elements includes: S11. Determine the number and width of the micro-elements to meet the preset requirements for circumferential viewing speed and visual continuity. The formulas for the number and width are as follows:

[0010]

[0011] in, Indicates the number of micro-elements in wide-angle imaging. This represents the width of the wide-angle imaging element. This represents the average angular velocity, measured in degrees per second. This represents the minimum effective frame rate to ensure visual continuity, measured in frames per second. This represents the distance between the wide-angle imaging element and the axis of symmetry of the observed object during panoramic viewing. S12. Determine the longitudinal curvature and lateral curvature of each wide-angle imaging micro-element to ensure that each wide-angle imaging micro-element can obtain the complete field of view of the measured object. The expressions for the longitudinal curvature and lateral curvature of the micro-element are as follows:

[0012]

[0013] in, Let the transverse curvature of the infinitesimal element be denoted as . The longitudinal curvature of the infinitesimal element; This is the margin coefficient; The longitudinal span of the wide-angle reflective micro-element; The height of the observed object; To observe the width of the object.

[0014] Furthermore, the range of structural parameters for designing the lens includes: S21. Let the radius of the annular region be... The relationship between the radial position of the lens (i.e., the vertical distance between the lens and the observed object) and the annular region of the target is expressed as follows:

[0015] in, This indicates the vertical distance between the lens and the observed object. and Indicates the margin coefficient; The radius of a lens is represented by its physical radius, based on the law of reflection of light. The above expression should be satisfied to ensure that all effective light rays from the target annular region can enter the lens and participate in imaging. This represents the radial distance from the boundary of the target area to the system's optical axis. This represents the radial distance from the outer boundary of the target area to the system's optical axis; For the design of the micro-element tilt angle, The angle of incidence of the light ray onto the horizontal plane of the lens; This indicates an angle of 180 degrees; S22. Suppose that after the target area is modulated by the lens, the target projection range on the camera imaging plane is... , and These represent the radial positions of the inner and outer boundaries of the target region on the imaging plane, respectively. The average radial magnification required for this region, based on the target projection range, is defined as follows:

[0016] in, , Indicates the average radial magnification; Under paraxial conditions, the modulation effect of the lens on the target annular region is equivalent to local thin lens imaging, and its equivalent focal length is denoted as . Let the axial distance from the vertex of the lens to the plane of the object being measured be... The axial distance from the lens to the equivalent image point in this region is The formula for the relationship between the three is:

[0017] The average radial magnification approximately satisfies:

[0018] Combining the two formulas above, the equivalent focal length expression is:

[0019] Under paraxial conditions, its equivalent focal length expression is:

[0020] in, Let be the radius of curvature of the lens surface; The refractive index of air, The refractive index of the lens material; The initial value of the radius of curvature for calculating the lens is given by the following formula:

[0021] Substituting the initial value of the radius of the lens curvature into the equivalent focal length expression, we obtain the radius of the lens curvature, which is given by the following formula:

[0022] S23. Based on the law of reflection, the angle of incidence of the light ray entering the lens is obtained by the following formula:

[0023] in, The angle of incidence of the light ray entering the lens; Based on Snell's law and lens thickness, a model is constructed to represent the lateral displacement of the incident light rays after passing through the lens, in order to correct the effect of the thick lens on the imaging position. The expression for the lateral displacement is as follows:

[0024] in, The refractive index of the lens material, This is a lateral displacement. Lens thickness; Satisfy the following formula:

[0025] The formula for converting the parameter range to thickness is:

[0026] in, This represents the margin coefficient.

[0027] Furthermore, a coordinate mapping from a ring to a plane is performed on the wide-angle imaging micro-element image, including: Obtain the geometric parameters of the effective annular region in the wide-angle imaging micro-element image, including the center coordinates. , inner diameter and outer diameter ; Reconstruct the annular region into a length of Width is The formula for a rectangular image is:

[0028]

[0029] in, For sampling operators; The formula for establishing the inverse mapping between the target rectangular pixel space and the original annular pixel space is:

[0030]

[0031]

[0032]

[0033] in, This represents the radius corresponding to any point in a wide-angle imaging micro-element image. Represents the ordinate of the rectangular image. Represents the x-coordinate of the rectangular image. Indicates the polar angle. This represents the coordinates of any point in a wide-angle imaging micro-element image after mapping, which are then mapped to the rectangular image.

[0034] Furthermore, image segmentation is performed on the rectangular image after mapping the wide-angle imaging micro-element image, including: Based on the preset number of wide-angle imaging micro-elements Wide-angle imaging micro-element image along Divided into equal parts Each sub-image block is mapped to a rectangular image based on a mapping relationship, and each sub-image block corresponds to the observation view of one of the wide-angle imaging micro-elements.

[0035] Furthermore, geometric distortion correction is performed on the segmented image to obtain the corrected segmented image, including: Capture images of the calibration object and obtain the homography matrix between the camera image plane and the world coordinate system. The formula is:

[0036]

[0037] in, Scale factor; These are the homogeneous coordinates of the image pixel coordinate system; It is a homography matrix. Represents the rotation matrix. Represents the translation vector. The homogeneous coordinates are used to define the plane coordinate system of the calibration plate. This is the intrinsic parameter matrix;

[0038] in, Images of the calibration objects and Focal length in direction, The coordinate axis tilt factor. The coordinates of the principal point on the calibration object image; Based on the intrinsic parameter constraint formula, the camera intrinsic parameter constraint equation is obtained through the homography matrix. The formula is as follows:

[0039]

[0040] in, The first column of the homography matrix H The second column of the homography matrix H is B; B is a symmetric matrix. Based on the principle of minimizing reprojection error, a nonlinear optimization objective function is constructed to perform high-precision correction on the original image with geometric distortion. The formula is as follows:

[0041] in, The number of images after segmentation. For the first The first image The actual pixel coordinates of the point; Indicates use The projected pixel coordinates obtained by solving for the actual pixels of the image; ;in, The radial distortion coefficient is... The tangential distortion coefficient; Indicates the first The translation vector of the image; Indicates the first Homogeneous coordinates of a point; The nonlinear least squares method is used to solve the nonlinear optimization objective function to obtain the optimal camera intrinsic parameter matrix and distortion parameters; Based on the optimal camera intrinsic matrix and distortion parameters, the pixel coordinates of the segmented image are established. pixel coordinates of the corrected image The mapping relationship between them is as follows: Pixel coordinates of the segmented image Convert to ideal normalized coordinates Its formula is:

[0042] Based on the optimal distortion parameters and the calculated distortion normalized coordinates Its formula is:

[0043]

[0044]

[0045] in, As an intermediate variable; The distortion-normalized coordinates are transformed back to the original image coordinate system using the optimal camera intrinsic parameter matrix to obtain the corresponding coordinates. Its formula is:

[0046]

[0047] The coordinates of all points in the segmented image are corrected using a mapping relationship to obtain the corrected segmented image.

[0048] Beneficial Effects: This invention provides a surround-view observation method based on wide-angle imaging micro-elements. By constructing multiple wide-angle imaging micro-element units around the target and combining them with a directional lens that selectively magnifies the effective imaging area of ​​the ring, along with digital image processing algorithms, a high-quality surround-view video is generated. In the imaging acquisition stage, this invention achieves parameterized matching between the optical system and observation requirements by designing the number, physical width, radius of curvature, and spatial tilt angle of the imaging micro-elements, ensuring the stability and continuity of the surround-view imaging acquisition. In the image processing stage, the image is mapped, segmented, and distortion is eliminated. Finally, the corrected sub-image sequence is sorted according to the segmentation angle, and a visually continuous high-fidelity three-dimensional surround-view dynamic video is output. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A flowchart of a panoramic observation method based on wide-angle imaging micro-elements provided by the present invention; Figure 2 This is a schematic diagram of a wide-angle imaging element. Figure 3 This is a schematic diagram of the lens structure; Figure 4 This is a schematic diagram of the annular effective imaging area and selective magnification in this embodiment; Figure 5 This is a schematic diagram of the image after processing the micro-element where 0° is located in this embodiment; Figure 6 This is a schematic diagram of the image after processing the micro-element at 90° in this embodiment; Figure 7 This is a schematic diagram of the image after processing the micro-element where 180° is located in this embodiment; Figure 8 This is a schematic diagram of the image after processing the micro-element at 270° in this embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] This embodiment provides a panoramic observation method based on wide-angle imaging micro-elements, such as... Figure 1 As shown, it includes: S1: Design the micro-element parameters for wide-angle imaging, which include the number of micro-elements, the width of micro-elements, the longitudinal curvature of micro-elements, the lateral curvature of micro-elements, and the tilt angle of micro-elements; S2: Based on the laws of rectilinear propagation and reflection, the reflected light path is obtained. Combining the parameters of the wide-angle imaging element and the distance between the observed object and the wide-angle imaging element, the structural parameters of the lens are designed to ensure that there are effective light rays in the reflected light path that meet the preset imaging quality requirements. The structural parameters include the radius, thickness and curvature of the lens; one side of the lens is a curved surface and the other side is a plane. S3: Select any set of structural parameters within the range of structural parameters, and design the camera observation position according to the camera's focal length so that the wide-angle imaging micro-element image in the lens is clearest under the structural parameters, and then acquire the wide-angle imaging micro-element image. S4: Process the wide-angle imaging micro-element image, including performing coordinate mapping from a ring to a plane, image segmentation, and geometric distortion correction on the wide-angle imaging micro-element image to obtain a corrected segmented image; S5: Sort the corrected images according to the segmentation angle, and continuously play the sorted images according to the preset panoramic viewing time requirements to realize panoramic observation of wide-angle imaging micro-elements.

[0053] Specifically, this method constructs multiple wide-angle imaging micro-elements with preset curvature and tilt angles around the observed object to perform decompositional acquisition of the target's local field of view. Lenses selectively magnify the effective annular imaging area and suppress redundancy in the central field of view, thereby improving imaging clarity and spatial utilization. In the image processing stage, the acquired images undergo annular-to-planar coordinate mapping, viewpoint segmentation, and geometric distortion correction, and are rearranged in angular order to generate a continuous annular image sequence. The wide-angle imaging micro-element array receives and reflects light from the large field of view of the observed object; the lens receives the reflected light and modulates and converges it, ultimately forming an image on the camera target surface. This method enables highly continuous and consistent annular observation of the target's three-dimensional spatial structure and its dynamic evolution process, and is suitable for applications such as complex physical field targets, spatial structure monitoring, and annular video imaging.

[0054] In a specific embodiment, the design scheme for wide-angle imaging micro-element parameters, including the number of micro-elements, micro-element width, micro-element longitudinal curvature, micro-element lateral curvature, and micro-element tilt angle, is as follows: S11. Determine the number and width of the micro-elements to meet the preset requirements for circumferential viewing speed and visual continuity. The formulas for the number and width are as follows:

[0055]

[0056] in, Indicates the number of micro-elements in wide-angle imaging. This represents the width of the wide-angle imaging element. This represents the average angular velocity, measured in degrees per second. This represents the minimum effective frame rate to ensure visual continuity. In this scheme, to achieve panoramic observation of the observed object at its central axis using a circumferential viewing speed, and to ensure visual continuity, 24fps for a basic sense of continuity. This represents the distance between the wide-angle imaging element and the axis of symmetry of the observed object during panoramic viewing. This represents the change in viewing angle per second during a panoramic view, measured in units of... ; S12. In order for each wide-angle imaging element to obtain the complete field of view of the measured object, the imaging element needs to use a wide-angle lens, based on the relationship between focal length and radius of curvature and the imaging formula expression.

[0057]

[0058] in, Indicates the radius of curvature. Indicates focal length. Indicates object distance, Indicates the virtual image distance; Obtain linear magnification The expression is

[0059] The object distance and virtual image distance are equivalent to the span of the wide-angle reflection micro-element and the height or width of the observed object. The longitudinal curvature and lateral curvature of each wide-angle imaging micro-element are determined to ensure that each wide-angle imaging micro-element can obtain the complete field of view of the measured object. The expressions for the longitudinal curvature and lateral curvature of the micro-element are as follows:

[0060]

[0061] in, Let the transverse curvature of the infinitesimal element be denoted as . The longitudinal curvature of the infinitesimal element; This is the margin coefficient; The longitudinal span of the wide-angle reflective micro-element; The height of the observed object; To observe the width of the object; S13. Preset the micro-element tilt angle to determine the initial position of the lens and camera; Wide-angle imaging micro-element, such as Figure 2 As shown, the wide-angle imaging micro-element is a tiny reflective surface (similar to a microprism or micromirror) distributed on a ring support. Each wide-angle imaging micro-element, as an independent optical acquisition unit, can clearly image and completely cover a specific local field of view of the observed object by designing various parameters.

[0062] In a specific embodiment, the reflected light path is obtained based on the laws of rectilinear propagation and reflection. Combining the parameters of the wide-angle imaging element and the distance between the observed object and the wide-angle imaging element, the structural parameter range of the lens is designed to ensure that the reflected light path contains effective light rays that meet the preset imaging quality requirements. The scheme includes the lens's radius, thickness, and curvature. To achieve directional magnification of the micro-element region in a wide-angle ring imaging system and effectively suppress field redundancy in the central region, thereby improving image clarity, a lens such as... Figure 3 As shown, this lens can achieve pincushion distortion (the magnification of the image increases with distance from the center (the edges are magnified)); S21. To ensure that all effective light rays from the target annular region can enter the lens and participate in imaging, the boundary light rays corresponding to the outer boundary of the target region must not exceed the effective aperture of the lens. Let the radius of the annular region be... The relationship between the radial position of the lens (i.e., the vertical distance between the lens and the observed object) and the annular region of the target is expressed as follows:

[0063] in, This indicates the vertical distance between the lens and the observed object. and Indicates the margin coefficient; The radius of a lens is represented by its physical radius, based on the law of reflection of light. The above expression should be satisfied to ensure that all effective light rays from the target annular region can enter the lens and participate in imaging. This represents the radial distance from the boundary of the target area to the system's optical axis. This represents the radial distance from the outer boundary of the target area to the system's optical axis; For the design of the micro-element tilt angle, The angle of incidence of the light ray onto the horizontal plane of the lens; This indicates an angle of 180 degrees; S22. Suppose that after the target area is modulated by the lens, the target projection range on the camera imaging plane is... , and These represent the radial positions of the inner and outer boundaries of the target region on the imaging plane, respectively. The average radial magnification required for this region, based on the target projection range, is defined as follows:

[0064] in, , Indicates the average radial magnification; Under paraxial conditions, the modulation effect of the lens on the target annular region is equivalent to local thin lens imaging, and its equivalent focal length is denoted as . Let the axial distance from the vertex of the lens to the plane of the object being measured be... The axial distance from the lens to the equivalent image point in this region is The formula for the relationship between the three is:

[0065] The average radial magnification approximately satisfies:

[0066] Combining the two formulas above, the equivalent focal length expression is:

[0067] A lens with one side curved and the other flat, under paraxial conditions, has the following equivalent focal length expression:

[0068] in, Let be the radius of curvature of the lens surface; The refractive index of air, The refractive index of the lens material; The initial value of the radius of curvature for calculating the lens is given by the following formula:

[0069] Substituting the initial value of the radius of the lens curvature into the equivalent focal length expression, we obtain the radius of the lens curvature, which is given by the following formula:

[0070] Therefore, when distance Refractive index of lens material and target magnification Once determined, the radius of curvature of the lens can be calculated. The initial design values; S23. To accurately describe the propagation path of light after entering the lens and to correct the influence of the thick lens structure on the imaging position, ensuring that all effective light rays from the target annular region can enter the lens and participate in imaging, the incident angle of the light rays entering the lens is obtained based on the law of reflection. The formula is as follows:

[0071] in, The angle of incidence of the light rays entering the lens (with the horizontal plane as a reference). Based on Snell's law and lens thickness, a model is constructed to represent the lateral displacement of the incident light rays after passing through the lens, in order to correct the effect of the thick lens on the imaging position. The expression for the lateral displacement is as follows:

[0072] in, The refractive index of the lens material, This is a lateral displacement. Lens thickness; Satisfy the following formula:

[0073] The formula for converting the parameter range to thickness is:

[0074] in, Indicates the margin coefficient; like Figure 4 As shown, in this scheme, the radius, thickness and curvature of the lens are calculated to determine the boundary of the acquisition range. After the incident beam is modulated by the lens, the curvature is designed to achieve directional magnification of the annular wide-angle imaging micro-element region and significantly suppress the imaging redundancy in the central region. At the same time, with the large angle deflection of the optical path, the image produces a preset geometric distortion in the edge region.

[0075] In a specific embodiment, a set of structural parameters is selected from the range of structural parameters. The camera observation position is designed according to the camera's focal length to make the wide-angle imaging micro-element image in the lens clearest under the given structural parameters. The scheme for acquiring the wide-angle imaging micro-element image is as follows: After determining the structural parameters of the lens that meet the requirements, the axial observation position of the camera relative to the lens is adjusted so that the intermediate image formed by the lens, which contains all the micro-element information of the wide-angle imaging, can be imaged most clearly on the photosensitive surface of the camera.

[0076] In a specific embodiment, the wide-angle imaging micro-element image is processed, including performing coordinate mapping from a ring to a plane, image segmentation, and geometric distortion correction on the wide-angle imaging micro-element image, to obtain the corrected segmented image. The coordinate mapping from a ring to a plane for a wide-angle imaging micro-element image includes: Obtain the geometric parameters of the effective annular region in the wide-angle imaging micro-element image, including the center coordinates. , inner diameter and outer diameter ; Reconstruct the annular region into a length of Width is The formula for a rectangular image is:

[0077]

[0078] in, For sampling operators; The formula for establishing the inverse mapping between the target rectangular pixel space and the original annular pixel space is:

[0079]

[0080]

[0081]

[0082] in, This represents the radius corresponding to any point in a wide-angle imaging micro-element image. Represents the ordinate of the rectangular image. Represents the x-coordinate of the rectangular image. Indicates the polar angle. This represents the coordinates of any point in a wide-angle imaging micro-element image, mapped to the rectangular image. Image segmentation is performed on the rectangular image after mapping the wide-angle imaging micro-element image, including: Because the wide-angle reflection elements are arranged in a cylindrical pattern, the number of wide-angle imaging elements is predetermined. Wide-angle imaging micro-element image along The degree is divided into equal parts Each sub-image block is mapped to a rectangular image based on a mapping relationship, and each sub-image block corresponds to the observation view of one of the wide-angle imaging micro-elements; Geometric distortion correction is performed on the segmented image to obtain the corrected segmented image, including: This method uses a chessboard as a calibration reference, captures images of the calibration object, and obtains the homography matrix between the camera image plane and the world coordinate system. The formula is:

[0083]

[0084] in, Scale factor; These are the homogeneous coordinates of the image pixel coordinate system; It is a homography matrix. Represents the rotation matrix. Represents the translation vector. The homogeneous coordinates are used to define the plane coordinate system of the calibration plate. This is the intrinsic parameter matrix;

[0085] in, Images of the calibration objects and Focal length in direction, The coordinate axis tilt factor. The coordinates of the principal point on the calibration object image are given; in this scheme, the principal point is the center point of the image. Based on the intrinsic parameter constraint formula, the camera intrinsic parameter constraint equation is obtained through the homography matrix. The formula is as follows:

[0086]

[0087] in, The first column of the homography matrix H The second column of the homography matrix H is B; B is a symmetric matrix. Based on the principle of minimizing reprojection error, a nonlinear optimization objective function is constructed to perform high-precision correction on the original image with geometric distortion. The formula is as follows:

[0088] in, The number of images after segmentation. For the first The first image The actual pixel coordinates of the point; Indicates use The projected pixel coordinates obtained by solving for the actual pixels of the image; ;in, The radial distortion coefficient is... The tangential distortion coefficient; Indicates the first The translation vector of the image; Indicates the first Homogeneous coordinates of a point; The nonlinear least squares method is used to solve the nonlinear optimization objective function to obtain the optimal camera intrinsic parameter matrix and distortion parameters; Based on the optimal camera intrinsic matrix and distortion parameters, the pixel coordinates of the segmented image are established. pixel coordinates of the corrected image The mapping relationship between them is as follows: Pixel coordinates of the segmented image Convert to ideal normalized coordinates Its formula is:

[0089] Based on the optimal distortion parameters and the calculated distortion normalized coordinates Its formula is:

[0090]

[0091]

[0092] in, As an intermediate variable; The distortion-normalized coordinates are transformed back to the original image coordinate system using the optimal camera intrinsic parameter matrix to obtain the corresponding coordinates. Its formula is:

[0093]

[0094] The coordinates of all points in the segmented image are corrected using a mapping relationship to obtain the corrected segmented image; After a series of processing steps including ring-plane coordinate mapping, sub-image segmentation, and distortion correction, the original image captured by the camera is converted into a standard perspective view, such as... Figures 5 to 8 As shown.

[0095] In a specific embodiment, the corrected images are sorted according to the segmentation angle, and the sorted images are played continuously according to a preset panoramic viewing time requirement. The scheme for panoramic observation of wide-angle imaging micro-elements is as follows: The n discrete sub-images obtained after correction and segmentation are assembled into a visually continuous and temporally controllable dynamic sequence according to the segmentation angle, forming a virtual "panoramic" perspective that rotates around the observed target.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A surround view observation method based on wide-angle imaging microcells, characterized in that, include: S1: Design the micro-element parameters for wide-angle imaging, which include the number of micro-elements, the width of micro-elements, the longitudinal curvature of micro-elements, the lateral curvature of micro-elements, and the tilt angle of micro-elements; S2: Based on the laws of rectilinear propagation and reflection, the reflected light path is obtained. Combining the parameters of the wide-angle imaging element and the distance between the observed object and the wide-angle imaging element, the structural parameters of the lens are designed to ensure that there are effective light rays in the reflected light path that meet the preset imaging quality requirements. The structural parameters include the radius, thickness and curvature of the lens; one side of the lens is a curved surface and the other side is a plane. S3: Select any set of structural parameters within the range of structural parameters, and design the camera observation position according to the camera's focal length so that the wide-angle imaging micro-element image in the lens is clearest under the structural parameters, and then acquire the wide-angle imaging micro-element image. S4: Process the wide-angle imaging micro-element image, including performing coordinate mapping from a ring to a plane, image segmentation, and geometric distortion correction on the wide-angle imaging micro-element image to obtain a corrected segmented image; S5: Sort the corrected images according to the segmentation angle, and continuously play the sorted images according to the preset panoramic viewing time requirements to realize panoramic observation of wide-angle imaging micro-elements.

2. The panoramic observation method based on wide-angle imaging micro-elements according to claim 1, characterized in that, The design includes the number of infinitesimal elements, their width, longitudinal curvature, and transverse curvature, among other things: S11. Determine the number and width of the micro-elements to meet the preset requirements for circumferential viewing speed and visual continuity. The formulas for the number and width are as follows: in, Indicates the number of micro-elements in wide-angle imaging. This represents the width of the wide-angle imaging element. This represents the average angular velocity, measured in degrees per second. This represents the minimum effective frame rate to ensure visual continuity, measured in frames per second. This represents the distance between the wide-angle imaging element and the axis of symmetry of the observed object during panoramic viewing. S12. Determine the longitudinal curvature and lateral curvature of each wide-angle imaging micro-element to ensure that each wide-angle imaging micro-element can obtain the complete field of view of the measured object. The expressions for the longitudinal curvature and lateral curvature of the micro-element are as follows: in, Let the transverse curvature of the infinitesimal element be denoted as . The longitudinal curvature of the infinitesimal element; This is the margin coefficient; The longitudinal span of the wide-angle reflective micro-element; The height of the observed object; To observe the width of the object.

3. The panoramic observation method based on wide-angle imaging micro-elements according to claim 1, characterized in that, The range of structural parameters for designing lenses includes: S21. Let the radius of the annular region be... The relationship between the radial position of the lens (i.e., the vertical distance between the lens and the observed object) and the annular region of the target is expressed as follows: in, This indicates the vertical distance between the lens and the observed object. and Indicates the margin coefficient; The radius of a lens is represented by its physical radius, based on the law of reflection of light. The above expression should be satisfied to ensure that all effective light rays from the target annular region can enter the lens and participate in imaging. This represents the radial distance from the boundary of the target area to the system's optical axis. This represents the radial distance from the outer boundary of the target area to the system's optical axis; For the design of the micro-element tilt angle, The angle of incidence of the light ray onto the horizontal plane of the lens; This indicates an angle of 180 degrees; S22. Suppose that the target area, after being modulated by the lens, projects onto the camera's imaging plane a range of... , and These represent the radial positions of the inner and outer boundaries of the target region on the imaging plane, respectively. The average radial magnification required for this region, based on the target projection range, is defined as follows: in, , Indicates the average radial magnification; Under paraxial conditions, the modulation effect of the lens on the target annular region is equivalent to local thin lens imaging, and its equivalent focal length is denoted as . Let the axial distance from the vertex of the lens to the plane of the object being measured be... The axial distance from the lens to the equivalent image point in this region is The formula for the relationship between the three is: The average radial magnification approximately satisfies: Combining the two formulas above, the equivalent focal length expression is: Under paraxial conditions, its equivalent focal length expression is: in, Let be the radius of curvature of the lens surface; The refractive index of air, The refractive index of the lens material; The initial value of the radius of curvature for calculating the lens is given by the following formula: Substituting the initial value of the radius of the lens curvature into the equivalent focal length expression, we obtain the radius of the lens curvature, which is given by the following formula: S23. Based on the law of reflection, the angle of incidence of the light ray entering the lens is obtained by the following formula: in, The angle of incidence of the light ray entering the lens; Based on Snell's law and lens thickness, a model is constructed to represent the lateral displacement of the incident light rays after passing through the lens, in order to correct the effect of the thick lens on the imaging position. The expression for the lateral displacement is as follows: in, The refractive index of the lens material, This is a lateral displacement. Lens thickness; Satisfy the following formula: The formula for converting the parameter range to thickness is: in, This represents the margin coefficient.

4. The panoramic observation method based on wide-angle imaging micro-elements according to claim 1, characterized in that, The coordinate mapping from a ring to a plane for a wide-angle imaging micro-element image includes: Obtain the geometric parameters of the effective annular region in the wide-angle imaging micro-element image, including the center coordinates. , inner diameter and outer diameter ; Reconstruct the annular region into a length of Width is The formula for a rectangular image is: in, For sampling operators; The formula for establishing the inverse mapping between the target rectangular pixel space and the original annular pixel space is: in, This represents the radius corresponding to any point in a wide-angle imaging micro-element image. Represents the ordinate of the rectangular image. Represents the x-coordinate of the rectangular image. Indicates the polar angle. This represents the coordinates of any point in a wide-angle imaging micro-element image after mapping, which are then mapped to the rectangular image.

5. The panoramic observation method based on wide-angle imaging micro-elements according to claim 1, characterized in that, Image segmentation is performed on the rectangular image after mapping the wide-angle imaging micro-element image, including: Based on the preset number of wide-angle imaging micro-elements Wide-angle imaging micro-element image along Divided into equal parts Each sub-image block is mapped to a rectangular image based on a mapping relationship, and each sub-image block corresponds to the observation view of one of the wide-angle imaging micro-elements.

6. The panoramic observation method based on wide-angle imaging micro-elements according to claim 1, characterized in that, Geometric distortion correction is performed on the segmented image to obtain the corrected segmented image, including: Capture images of the calibration object and obtain the homography matrix between the camera image plane and the world coordinate system. The formula is: in, Scale factor; These are the homogeneous coordinates of the image pixel coordinate system; It is a homography matrix. Represents the rotation matrix. Represents the translation vector. The homogeneous coordinates are used to define the plane coordinate system of the calibration plate. This is the intrinsic parameter matrix; in, Images of the calibration objects and Focal length in direction, The coordinate axis tilt factor. The coordinates of the principal point on the calibration object image; Based on the intrinsic parameter constraint formula, the camera intrinsic parameter constraint equation is obtained through the homography matrix. The formula is as follows: in, The first column of the homography matrix H The second column of the homography matrix H is B; B is a symmetric matrix. Based on the principle of minimizing reprojection error, a nonlinear optimization objective function is constructed to perform high-precision correction on the original image with geometric distortion. The formula is as follows: in, The number of images after segmentation. For the first The first image The actual pixel coordinates of the point; Indicates use The projected pixel coordinates obtained by solving for the actual pixels of the image; ;in, The radial distortion coefficient is... The tangential distortion coefficient; Indicates the first The translation vector of the image; Indicates the first Homogeneous coordinates of a point; The nonlinear least squares method is used to solve the nonlinear optimization objective function to obtain the optimal camera intrinsic parameter matrix and distortion parameters; Based on the optimal camera intrinsic matrix and distortion parameters, the pixel coordinates of the segmented image are established. pixel coordinates of the corrected image The mapping relationship between them is as follows: Pixel coordinates of the segmented image Convert to ideal normalized coordinates Its formula is: Based on the optimal distortion parameters and the calculated distortion normalized coordinates Its formula is: in, As an intermediate variable; The distortion-normalized coordinates are transformed back to the original image coordinate system using the optimal camera intrinsic parameter matrix to obtain the corresponding coordinates. Its formula is: The coordinates of all points in the segmented image are corrected using a mapping relationship to obtain the corrected segmented image.

Images