Suspension structure for a photosensitive image sensor and method for reproducibly controlling and compensating for thermal drift in the image of a photosensitive image sensor - Patents.com

Abstract

【Solution means】 The subject of the present invention is in particular the suspension structure of a photosensitive image sensor in a digital camera. This suspension structure has a PCB substrate with an image sensor, and the PCB substrate (P) with the image sensor (M) has at least two first mounting openings (Ot1) elastically incorporated, or the camera base plate (B) has at least two second mounting openings (Ot2) elastically incorporated. A method for reproducibly controlling and compensating the temperature drift of an image in a photosensitive image sensor of a digital camera in particular includes the step of changing the hardware of the camera, and the step of recording the temperature drift of the image of the camera in units of 1 °C for temperatures in the range of -35 °C to 100 °C, wherein the temperature is recorded using a temperature sensor in the camera, the step of recording the temperature drift, the step of calculating a polynomial compensation model using the recorded temperature drift of the image, and the step of compensating the temperature drift recorded by the camera using the calculated polynomial compensation model.

Description

[Technical Field]

[0001] The subject of the present invention is a suspension structure for a photosensitive image sensor and a method for reproducibly controlling and compensating for temperature drift in the images of a photosensitive image sensor, in particular a digital camera. [Background technology]

[0002] It is generally believed that after a camera is calibrated, its image parameters (intrinsic, extrinsic, and distortion correction parameters) will not change. However, in reality, these parameters may change as the camera is exposed to changing environmental conditions. Temperature is one of the parameters that affect the camera's calibration parameters. The effect of temperature on the camera's image processing manifests itself in two forms: image drift due to the camera's warm-up process, and temperature drift due to changes in the camera's ambient temperature.

[0003] Most current cameras (recording devices with photosensitive image sensors) do not compensate for the thermal drift of the image recorded by the image sensor when operating under conditions of varying ambient temperature. Such drift can be seen as a (temperature-dependent) shift or deformation of the image position of each photosensitive pixel, which can be up to 1.5-2 pixels away from its expected position on the photosensitive image sensor. On the other hand, experimentally observed thermal drift in large cameras can be up to several pixels, which can mean a physical shift of several centimeters of the object recorded by the camera (depending on the lens used).

[0004] Until now, the mechanical design of digital cameras using CMOS or CCD sensors has not used any connection between the sensor (or the PCB electronics board on which the sensor is mounted) and the housing (also known as the camera base, base plate, C-mount base, or lens adapter), which means that the sensor (more precisely, the temperature-sensitive aluminum plate or the temperature-sensitive mounting that attaches the PCB board on which the sensor is mounted to the camera base) deforms randomly, i.e., with non-repeatable thermodynamic degrees of freedom. As a result, the temperature drift of the recorded images (during warm-up and cooling of the camera element if the ambient temperature is not constant) is random and mathematical adjustments and the generation of compensation models are impossible, so the temperature drift of the images has not been given much importance until now.

[0005] Holder Handel describes methods for compensating for temperature effects in camera calibration in a series of three publications: "Compensation of thermal errors in vision-based measurement systems using a system identification approach," 9th International Conference on Signal Processing, pp. 1329–1333, 2008; "Analyzing the influence of camera temperature on the image acquisition process," SPIE, vol. 6805, pp. 1–8, 2008; and "Analyzing the effects of camera warm-up effects on image acquisition," Computer Vision-ACCV 2007, pp. 258–268, 2007. Handel presents a solution for compensating for drift due to both camera warm-up and external temperature changes. The proposed compensation method assumes the use of a pinhole camera model described by equations that do not consider distortion correction. This compensation is achieved by adjusting a linear model parameterized only by the camera's external parameters. The authors assume that the camera's internal parameters do not change with temperature. The authors also create a linear compensation model using only six of the ten parameters that describe the equation. In reality, such a model is a gross simplification and does not take into account all aspects of the influence of temperature on camera calibration. The internal coefficients of the camera also change as a result of thermal deformations of the camera housing, sensor, lens mount, and the lens itself.

[0006] International Publication No. 2021 / 164058 (patent document) discloses a method and system for calibrating the temperature drift of a ToF camera. The method includes the steps of: changing the environmental temperature of a sample of ToF cameras to be calibrated by adjusting the temperature with a temperature control jig; measuring the environmental temperature of the sample of ToF cameras with a temperature sensor; acquiring the temperature drift coefficients of each ToF camera to obtain a temperature drift coefficient set; acquiring a calibration temperature drift coefficient set from the temperature drift coefficient set, where the temperature drift coefficients in the calibration temperature drift coefficient set enable a predetermined percentage of the ToF cameras in the sample of ToF cameras to satisfy a predetermined measurement accuracy; acquiring the calibration temperature drift coefficient set; acquiring the measurement accuracy of the ToF camera to be calibrated based on each temperature drift coefficient in the calibration temperature drift coefficient set; selecting a temperature drift coefficient that enables the ToF camera to satisfy the predetermined measurement accuracy; and calibrating the ToF camera.

[0007] Chinese Patent Application Publication No. 110798593 (patent document) discloses a method for eliminating temperature drift in an industrial camera. This method includes the steps of: forming an image output by an industrial camera under the influence of temperature by superimposing P(n,m) on the pixel values ​​and temperature drift values ​​of corresponding positions in the image (Step 1). Two adjacent original images are considered to be two identical images, and the temperature drift values ​​of corresponding positions in the previous image can be approximated to the temperature drift values ​​of corresponding positions in the current image. This approximate temperature drift value is the average pixel temperature rise value of the original images, i.e., the average pixel temperature drift value (T), based on the fact that each photosensitive unit in the industrial camera is subject to the same temperature. The advantageous effects of this invention are that the industrial camera can be used normally without the need for temperature calibration by comparing two adjacent images, complex environmental factors can be controlled, the camera can operate stably, the time required for camera calibration can be shortened to a certain extent, and work efficiency can be improved.

[0008] Chinese Patent Application Publication No. 112270712 (patent document) discloses a method and system for calibrating temperature drift based on a depth camera module. The calibration method includes the steps of: adjusting the temperature of an environment in which the depth camera module is placed to a preset environmental temperature value; obtaining a measured depth value, a light source real-time temperature value, and a sensor real-time temperature value for a calibration plate of the depth camera module under the preset environmental temperature value; calculating a difference between the measured depth value and the actual depth value of the calibration plate to obtain a measurement error; constructing a fitness function; using the fitness function to fit the preset environmental temperature value, the light source real-time temperature value, the sensor real-time temperature value, and the measurement error; calculating an optimal solution of an undetermined coefficient in the fitness function; and using the optimal solution of the undetermined coefficient as the temperature drift coefficient of the depth camera module.

[0009] China Patent Application Publication No. 112393808 (Patent Document) discloses a temperature compensation method and system for a temperature-sensitive camera, which belongs to the technical field of temperature-sensitive cameras. To address the problem of low accuracy and measurement precision of temperature-sensitive cameras in the prior art, the invention provides a temperature compensation method and system for a temperature-sensitive camera. A temperature-sensitive sensor is connected in front of the temperature-sensitive camera, and a blackbody is also placed there. By calculating a function of the temperature measured by the temperature-sensitive camera, the actual temperature of the temperature-sensitive sensor, the temperature of the blackbody, the ambient temperature, and the measured distance of the object, the influences of factors such as camera drift, non-uniformity of the temperature-sensitive camera, the refractive index of the temperature sensor, the ambient temperature, and the measured distance of the object are calibrated. This invention achieves temperature compensation for a temperature-sensitive camera with low cost and high precision, and the corresponding calibration is performed in real time. After shipping, the temperature sensitivity can be continuously used after only one calibration. Furthermore, there is no need to place a blackbody during measurement.

[0010] Chinese Patent Application Publication No. 111182240 (Patent Document) discloses a method for automatically compensating for the temperature drift of an image sensor. By linearizing the captured image and performing real-time temperature pixel drift compensation, the problem of distortion of the image sensor's output pixel values ​​caused by relatively high environmental temperatures is solved, allowing the image sensor to operate normally for a long time in a narrow space with high temperatures and improving the environmental adaptability of the image sensor. The above method is versatile for any image sensor that is affected by temperature, thereby widening the application environment of the image sensor. Summary of the Invention [Means for solving the problem]

[0011] The object of the present invention is to develop a suspension structure for a photosensitive image sensor and a method for compensating for the temperature drift of an image (on a sensor, particularly a photosensitive image sensor such as a CCD or CMOS) in a reproducible and controlled manner. Technically eliminating or limiting the temperature drift of an image is very difficult and very expensive. Reproducible temperature drift cannot be achieved or controlled without modifying the camera hardware.

[0012] The present invention relates to a suspension structure for a light-sensitive image sensor, in particular in a digital camera, characterized in that the suspension structure has a PCB board with an image sensor, and the PCB board with the image sensor has at least two resiliently integrated first mounting openings, or the camera base plate (B) has at least two resiliently integrated second mounting openings.

[0013] The first mounting opening is preferably located on the first flexure.

[0014] The second mounting opening is preferably located on the second flexure.

[0015] The PCB board having the image sensor preferably has four first mounting openings.

[0016] The camera base plate preferably has four second mounting openings.

[0017] Preferably, the PCB board with the image sensor and the camera base plate are removably connected.

[0018] The method for reproducibly controlling and compensating for the thermal drift of images, in particular of a photosensitive image sensor of a digital camera, according to the present invention consists in modifying the camera hardware and recording the thermal drift of the images of said camera in increments of at least 1°C for temperatures ranging from -35°C to 100°C, wherein the temperatures are recorded using a temperature sensor in said camera, recording said thermal drift, using said recorded image thermal drift to calculate a polynomial compensation model, and using said calculated polynomial compensation model to compensate for the temperature drift recorded by said camera.

[0019] Preferably, the process is carried out periodically, either every second or for each image recorded by the camera.

[0020] Preferably, said steps are carried out before recording the image, during recording the image or after recording the image.

[0021] Advantages of the suspension structure for a photosensitive image sensor and the method for repeatably controlling and compensating for thermal drift of images in a photosensitive image sensor include eliminating or significantly reducing the irreproducibility of thermal drift observed in images recorded by digital cameras, low manufacturing costs, and easy and reliable implementation into existing products. [Brief explanation of the drawings]

[0022] The drawings show exemplary embodiments of the invention. [Figure 1] FIG. 1 shows a PCB board with an image sensor attached to a camera base with a lens attached. [Figure 2] FIG. 2 shows a suspension structure for a PCB substrate with an image sensor according to the present invention, which in this embodiment has an opening Ot1 and a flexure Sp1. [Figure 3] FIG. 3 shows a schematic diagram of the solution of FIG. [Figure 4] FIG. 4 shows a suspension structure for a PCB substrate with an image sensor according to the present invention, which in this embodiment has an opening Ot2 and a flexure Sp2. [Figure 5] FIG. 5 shows a schematic diagram of the solution of FIG. DETAILED DESCRIPTION OF THE INVENTION [Example]

[0023] 2 and 3 show a photosensitive image sensor suspension structure with a PCB board P carrying a camera image sensor M. The board is connected to the camera housing, i.e., the camera base B, which provides thermodynamic freedom for the photosensitive image sensor, allowing it to deform freely and repeatably under the influence of the camera's variable operating temperatures. The hardware is modified so that the PCB plate P carrying the image sensor M is mounted via four elastically integrated first mounting openings Ot1 instead of a standard fastening method. The first mounting openings Ot1 are located on first flexures Sp1, creating a frictionless and play-free flexible suspension structure for the PCB plate P carrying the image sensor M. The connection of the PCB plate P carrying the image sensor M to the camera base B is achieved by screws. [Example]

[0024] Figures 4 and 5 show the structure of a camera base B with a lens O mounted thereon. Base B has four second mounting openings Ot2 located on second flexures Sp2. A PCB board P carrying an image sensor M is connected to base B through the second mounting openings Ot2. This connection provides thermodynamic freedom for the photosensitive image sensor, allowing it to deform freely and reproducibly under the influence of the camera's variable operating temperatures. The hardware is modified so that the PCB plate P carrying the image sensor M is mounted using four elastically integrated second mounting openings Ot2 instead of the standard fixed mounting method. The second mounting openings Ot2 are located on second flexures Sp2, creating a frictionless and play-free flexible suspension for the PCB plate P carrying the image sensor M. The connection of the PCB plate P carrying the image sensor M to the camera base B is achieved by screws.

[0025] Example of the method A method for reproducibly controlling and compensating for image temperature drift, particularly in a photosensitive image sensor of a digital camera, involves hardware modification of the camera according to Example 1 or 2, and then, while mounted on a dedicated stand, recording the image temperature drift in 1°C increments for temperatures ranging from -35°C to 100°C. A temperature sensor within the camera is used to record the temperatures. The recorded image temperature drift is used to calculate a compensation model using mathematical adjustment. The calculated compensation model is used to correct images recorded by the camera, and a polynomial model is used as the compensation model. The above process is performed periodically, either every second during image recording or for each image recorded by the camera.

[0026] In the embodiments of the present invention, to achieve a desired effect, it is possible to use only Example 1, only Example 2, or both Example 1 and Example 2. In any case, the effect of reproducibly controlling and compensating for the temperature drift of the image of the photosensitive image sensor can be achieved.

[0027] The solution according to the invention is particularly suitable for use in 3D scanners and 2D cameras (with CCD and CMOS image sensors) where the fidelity of the captured image or the accurate representation of the camera's position and orientation in space is critical (such as medical scans, satellite photography, crime scene recording, cultural heritage recording, etc.), and the photograph or image needs to be based on a reliable image (with no spatial offset relative to the imaged object's actual position in space) for accurate capture and use.

Claims

1. A suspension structure for a photosensitive image sensor (M), in particular in a digital camera, comprising: a PCB substrate (P) having an image sensor (M); Camera base plate (B) and and The PCB board (P) having the image sensor (M) is detachably connected to the camera base plate (B); The PCB substrate (P) having the image sensor (M) has at least two first mounting openings (Ot1) resiliently mounted thereon, which are connected to the camera base plate (B); or The camera base plate (B) has at least two elastically mounted second mounting openings (Ot2) connected to the PCB board (P) having the image sensor (M). A suspension structure characterized by:

2. 2. A suspension structure according to claim 1, characterized in that the first mounting opening (Ot1) is located on a first flat flexure (Sp1).

3. 2. A suspension structure according to claim 1, characterized in that the second mounting opening (Ot2) is located on a second flat flexure (Sp2).

4. 3. A suspension structure according to claim 1 or 2, characterized in that the PCB substrate (P) carrying the image sensor (M) has four first mounting openings (Ot1).

5. 4. A suspension structure according to claim 1 or 3, characterized in that the camera base plate (B) has four second mounting openings (Ot2).

6. A method for reproducibly controlling and compensating for the thermal drift of an image, in particular in a photosensitive image sensor (M) of a digital camera, comprising: a) modifying the camera hardware in accordance with the suspension structure of claim 1; b) recording the temperature drift of the image of the camera to an increment of at least 1°C for temperatures ranging from -35°C to 100°C, wherein the temperatures are recorded using a temperature sensor within the camera; c) using the thermal drift of the recorded images to calculate a polynomial compensation model using mathematical adjustments; d) using the calculated polynomial compensation model to compensate for the temperature drift recorded by the camera; 10. A method comprising:

7. 7. The method of claim 6, wherein step d) is performed periodically, such as every second or every image recorded by the camera.

8. 7. The method of claim 6, wherein steps a) to c) are performed before recording an image, during recording an image, or after recording an image.

Images