Suspension structure of a photosensitive image sensor and method for reproducibly controlling and compensating temperature drift in an image of the photosensitive image sensor

Abstract

【Solution means】 The subject of the present invention is in particular the suspension structure of a photosensitive image sensor in a digital camera. This suspension structure has a PCB substrate with an image sensor, and the PCB substrate (P) with the image sensor (M) has at least two first mounting openings (Ot1) elastically incorporated, or the camera base plate (B) has at least two second mounting openings (Ot2) elastically incorporated. A method for reproducibly controlling and compensating the temperature drift of an image in a photosensitive image sensor of a digital camera in particular includes the step of changing the hardware of the camera, and the step of recording the temperature drift of the image of the camera in units of 1 °C for temperatures in the range of -35 °C to 100 °C, wherein the temperature is recorded using a temperature sensor in the camera, the step of recording the temperature drift, the step of calculating a polynomial compensation model using the recorded temperature drift of the image, and the step of compensating the temperature drift recorded by the camera using the calculated polynomial compensation model.

Description

Technical Field

[0001] The subject of the present invention is a suspension structure for a photosensitive image sensor and a method for reproducibly controlling and compensating for temperature drift in the images of a photosensitive image sensor, particularly in a digital camera.

Background Art

[0002] Generally, after a camera is calibrated, its image parameters (intrinsic parameters, extrinsic parameters, distortion correction parameters) are considered not to change. However, in reality, these parameters may change when the camera is exposed to changes in environmental conditions. Temperature is one of the parameters that affect the calibration parameters of the camera. The influence of temperature on the image processing of the camera appears in two forms. That is, image drift associated with the warm-up process of the camera and temperature drift associated with changes in the ambient temperature of the camera are observed.

[0003] Most current cameras (recording devices having a photosensitive image sensor) hardly compensate for the temperature drift recorded by the image sensor when operating under conditions where the ambient temperature changes. Such drift can be confirmed by the shift or deformation of the image position for each photosensitive pixel (which depends on temperature), and there is also a possibility of shifting up to 1.5 to 2 pixels from the expected position on the photosensitive image sensor. On the other hand, in a large camera, the experimentally observed temperature drift can reach up to several pixels, which may mean that the object recorded by the camera physically shifts by several centimeters (although it also depends on the lens used).

[0004] Up to now, in the mechanical design of digital cameras using CMOS or CCD sensors, since no connection part has been used between the sensor (or the PCB electronic board equipped with the sensor) and the housing (also referred to as the camera base, base plate, C-mount base or lens adapter), the sensor (more precisely, the temperature-sensitive aluminum plate or the temperature-sensitive attachment part for attaching the PCB board equipped with the sensor to the camera base) has been deformed disorderly, that is, with non-reproducible thermodynamic degrees of freedom. As a result, the temperature drift of the recorded images (when the ambient temperature is not constant and during the warm-up and cooling of the camera element) has been random, and since mathematical adjustment and the generation of a compensation model are impossible, the temperature drift of the images has not been regarded as important until now.

[0005] Holder Handel describes a method for compensating the influence of temperature in camera calibration in a series of three publications: "Compensation of thermal errors in vision based measurement Systems using a system identification approach, 9th International Conference on Signal Processing, pp. 1329 - 1333, 2008", "Analyzing the influence of camera temperature on the image acquisition process, SPIE, vol. 6805, pp. 1 - 8, 2008", and "Analyzing the effects of camera warm - up effects on image acquisition, Computer Vision - ACCV 2007, pp. 258 - 268, 2007". Handel presents a solution for a method to compensate for drifts associated with both camera warm - up and changes in external temperature. The proposed compensation method assumes the use of a pinhole camera model described by an equation that does not consider distortion correction. This compensation is achieved by adjusting a linear model that parameterizes only the external parameters of the camera. The author assumes that the internal parameters of the camera do not change even when affected by temperature. Also, the author creates a linear compensation model using six out of the ten parameters that describe the equation. In reality, such a model is highly simplified and does not consider all aspects of the influence of temperature in camera calibration. Due to thermal deformation of the camera housing, sensor, lens mount, and the lens itself, the internal coefficients of the camera also change.

[0006] International Publication No. WO 2021 / 164058 (Patent Document) discloses a method and system for calibrating the temperature drift of a ToF camera. This method includes the steps of changing the ambient temperature of a sample of the ToF camera to be calibrated by adjusting the temperature with a temperature control jig, measuring the ambient temperature of the sample of the ToF camera with a temperature sensor, obtaining a temperature drift coefficient set by obtaining the temperature drift coefficient of each ToF camera, obtaining a calibration temperature drift coefficient set from the temperature drift coefficient set, wherein the temperature drift coefficients in the calibration temperature drift coefficient set enable a predetermined percentage of the ToF cameras in the sample of the ToF camera to meet a preset measurement accuracy, the step of obtaining the calibration temperature drift coefficient set, obtaining the measurement accuracy of the ToF camera calibrated based on each temperature drift coefficient in the calibration temperature drift coefficient set, selecting a temperature drift coefficient that enables the ToF camera to meet a preset measurement accuracy, and calibrating the ToF camera.

[0007] The specification of Chinese Patent Application Publication No. CN 110798593 A (Patent Document) discloses a method for eliminating the temperature drift of an industrial camera. The method for eliminating the temperature drift of this industrial camera includes the step of forming, by superimposing P(n,m) on the pixel value and the temperature drift value at the corresponding position of the image, an image output by the industrial camera under the action of temperature (Step 1). Here, two adjacent original images are regarded as two identical images, and the temperature drift value at the corresponding position of the previous image can be approximated to the temperature drift value at the corresponding position of the current image. This approximate temperature drift value is the average pixel temperature rise value of the original image, that is, the average pixel temperature drift value (T), based on the fact that each photosensitive unit of the industrial camera is under the influence of the same temperature. The advantageous effects of the above invention are that the industrial camera can be used normally without the need for temperature calibration by comparing two adjacent images, complex environmental factors can be controlled, stable operation of the camera can be enabled, the time required for camera calibration can be shortened to a certain extent, and work efficiency can be improved.

[0008] Chinese Patent Application Publication No. 112270712 (Patent Document) discloses a method and system for calibrating temperature drift based on a depth camera module. This calibration method includes the steps of adjusting the temperature of the environment where the depth camera module is placed to a preset environmental temperature value; obtaining the measured depth value, the light source real-time temperature value, and the sensor real-time temperature value for the calibration plate of the depth camera module under the preset environmental temperature value; calculating the difference between the measured depth value and the actual depth value of the calibration plate to obtain the measurement error; constructing a fitting function; using the fitting function to fit the preset environmental temperature value, the light source real-time temperature value, the sensor real-time temperature value, and the measurement error; calculating the optimal solution of the undetermined coefficients in the fitting function; and setting the optimal solution of the undetermined coefficients as the temperature drift coefficient of the depth camera module.

[0009] Chinese Patent Application Publication No. 112393808 (Patent Document) discloses a temperature compensation method and system for a temperature-sensitive camera, belonging to the technical field of temperature-sensitive cameras. In order to solve the problems of low accuracy and measurement accuracy of temperature-sensitive cameras in the prior art, the above invention provides a temperature compensation method and system for a temperature-sensitive camera. A temperature-sensitive sensor is connected in front of the temperature-sensitive camera, and a blackbody is further arranged. By calculating the functions of the temperature measured by the temperature-sensitive camera, the actual temperature of the temperature-sensitive sensor, the temperature of the blackbody, the environmental temperature, and the measured distance of the entity, and by calculating the function of the environmental temperature and the distance of the measurement object, the influences caused by factors such as the drift of the camera, the non-uniformity of the temperature-sensitive camera, the refractive index of the temperature sensor, the environmental temperature, and the measured distance of the entity are calibrated. According to the above invention, the temperature compensation of the temperature-sensitive camera is realized with low cost and high accuracy, the corresponding calibration is performed in real time, and after factory shipment, the temperature sensitivity can be always used with only one calibration. Also, there is no need to arrange a blackbody during measurement.

[0010] Specification of Chinese Patent Application Publication No. 111182240 (Patent Document) discloses a method for automatically compensating the temperature drift of an image sensor. Through the linearization of the acquired image and real-time temperature pixel drift compensation, the problem of distortion of the output pixel value of the image sensor caused by a relatively high ambient temperature is solved. As a result, the image sensor can operate normally for a long time in a narrow space at high temperature, and the environmental applicability of the image sensor is improved. Since the above method is universal for any image sensor affected by temperature, the applicable environment of the image sensor becomes wide-ranging.

Summary of the Invention

Means for Solving the Problems

[0011] The object of the present invention is to develop a method for compensating by reproducibly controlling the suspension structure of a photosensitive image sensor and the temperature drift of an image (on a sensor, especially a photosensitive image sensor such as a CCD or CMOS). Technically removing or limiting the temperature drift of an image is very difficult and very expensive. Without changing the hardware of the camera, it is impossible to achieve and control reproducible temperature drift.

[0012] The present invention relates in particular to the suspension structure of a photosensitive image sensor in a digital camera. This suspension structure has a PCB board having an image sensor, and the PCB board having the image sensor is characterized by having at least two first mounting openings elastically incorporated, or the camera base plate (B) having at least two second mounting openings elastically incorporated.

[0013] Preferably, the first mounting opening is arranged on a first bending portion.

[0014] Preferably, the second mounting opening is arranged on a second bending portion.

[0015] The PCB substrate having the image sensor preferably has four first mounting openings.

[0016] The camera base plate preferably has four second mounting openings.

[0017] The PCB substrate having the image sensor and the camera base plate are preferably detachably connected.

[0018] The essence of the method according to the present invention for reproducibly controlling and compensating the temperature drift of an image, particularly in a photosensitive image sensor of a digital camera, is to change the hardware of the camera and record the temperature drift of the image of the camera at least in units of 1°C for temperatures in the range of -35°C to 100°C, wherein the temperature is recorded using a temperature sensor in the camera, recording the temperature drift, calculating a polynomial compensation model using the recorded temperature drift of the image, and compensating the temperature drift recorded by the camera using the calculated polynomial compensation model.

[0019] The step is preferably executed periodically every second or for each image recorded by the camera.

[0020] The step is preferably executed before, during, or after recording an image.

[0021] The advantages of the suspension structure of the photosensitive image sensor and the method for reproducibly controlling and compensating the temperature drift of an image in the photosensitive image sensor are that the non-reproducibility of the temperature drift observed in the images recorded by the digital camera is eliminated or significantly reduced, the manufacturing cost is low, and the implementation on existing products is easy and highly reliable.

Brief Description of the Drawings

[0022] This drawing shows exemplary embodiments of the present invention.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

DETAILED DESCRIPTION OF THE INVENTION

EXAMPLE

[0023] Figures 2 and 3 show a suspension structure of a photosensitive image sensor having a PCB substrate P with a camera image sensor M. The substrate is connected to a camera housing, i.e., a camera base B, and this connection provides a thermodynamic degree of freedom for the photosensitive image sensor. That is, the photosensitive image sensor can deform freely and repeatably under the influence of the variable operating temperature of the camera. Instead of attaching the PCB plate P having the image sensor M by a standard fixing method, the hardware is modified to attach it via four elastically incorporated first attachment openings Ot1. The first attachment openings Ot1 are arranged on the first flexure Spl and form a flexible suspension structure without friction and play for the PCB plate P having the image sensor M. The connection of the PCB plate P having the image sensor M to the camera base B is realized by screws.

EXAMPLE

[0024] Figures 4 and 5 show the structure of the camera base B with the lens O attached. The base B has four second mounting openings Ot2 arranged in the second flexure part Sp2. The PCB board P having the image sensor M is connected to the base B by the second mounting openings Ot2. This connection provides the thermodynamic degrees of freedom of the photosensitive image sensor. That is, the photosensitive image sensor can be deformed freely and reproducibly under the influence of the variable operating temperature of the camera. Instead of attaching the PCB plate P having the image sensor M by a standard fixing method, the hardware is changed to attach using four second mounting openings Ot2 that are elastically incorporated. The second mounting openings Ot2 are arranged on the second flexure part Sp2 and form a flexible suspension structure without friction and play for the PCB plate P having the image sensor M. The connection of the PCB plate P having the image sensor M to the camera base B is realized by screws.

[0025] Example of the method A method for reproducibly controlling and compensating the temperature drift of an image, particularly in a photosensitive image sensor of a digital camera, is to modify the hardware of the camera according to Example 1 or 2, and then, with the camera attached to a dedicated stand, the camera records the temperature drift of the image in 1°C increments for temperatures in the range of -35°C to 100°C. A temperature sensor in the camera is used to record the temperature. The recorded temperature drift of the image is used to calculate a compensation model using mathematical adjustment. The calculated compensation model is used to correct the images recorded by the camera, and a polynomial model is used as the compensation model. The above steps are executed periodically every 1 second during image recording or for each image recorded by the camera.

[0026] In the embodiments of the present invention, in order to achieve the desired effect, it is possible to use only Example 1, only Example 2, or Example 1 and Example 2 simultaneously. In any case, the effect of reproducibly controlling and compensating the temperature drift of the image of the photosensitive image sensor can be obtained.

[0027] The solution according to the present invention is particularly used for 3D scanners and 2D cameras having (CCD and CMOS image sensors) where the fidelity of the acquired image or the accurate representation of the position and orientation of the camera in space is emphasized, such as in (medical scans, satellite photographs, crime scene recordings, cultural heritage recordings, etc.). The photograph or image needs to be based on a reliable image that is (spatially unshifted with respect to the actual position of the imaged object in space) for accurate shooting and use.

Claims

1. A suspension structure, particularly for a photosensitive image sensor in a digital camera, having a PCB substrate with an image sensor, wherein the PCB substrate (P) having the image sensor (M) has at least two first mounting openings (Ot1) elastically incorporated therein, or the camera base plate (B) has at least two second mounting openings (Ot2) elastically incorporated therein characterizes the suspension structure.

2. The suspension structure according to Claim 1, wherein the first mounting opening (Ot1) is arranged on a first flexure part (Sp1).

3. The suspension structure according to Claim 1, wherein the second mounting opening (Ot2) is arranged on a second flexure part (Sp2).

4. The suspension structure according to Claim 1 or 2, wherein the PCB substrate (P) having the image sensor (M) has four first mounting openings (Ot1).

5. The suspension structure according to Claim 1 or 3, wherein the camera base plate (B) has four second mounting openings (Ot2).

6. The suspension structure according to any one of Claims 1 to 5, wherein the PCB substrate (P) having the image sensor (M) and the camera base plate (B) are detachably connected.

7. A method for reproducibly controlling and compensating the temperature drift of an image in a photosensitive image sensor of a digital camera, comprising the step of modifying the camera hardware according to the suspension structure of Claim 1, the step of recording the temperature drift of the camera's image in units of at least 1 °C for temperatures in the range of -35 °C to 100 °C, wherein the temperature is recorded using a temperature sensor in the camera, the step of calculating a polynomial compensation model using the recorded temperature drift of the image, and the step of compensating the temperature drift recorded by the camera using the calculated polynomial compensation model characterizes the method.

8. The method according to Claim 7, wherein the steps are performed periodically every 1 second or for each image recorded by the camera.

9. The method according to claim 7, wherein the step is performed before, during, or after recording of the image.

Images