Image acquisition device and brightness balancing method

Abstract

The invention provides an image acquisition device and a brightness balancing method. The device comprises a roller shutter type image sensor and a total reflection prism module. The total reflection prism module is located on the light sensing side of the rolling shutter type image sensor and comprises a total reflection prism and a rotating structure. The rotating structure is used for driving the total reflection prism to rotate from the first position to the second position. When the total reflection prism is located at the first position, a light beam entering the total reflection prism is transmitted and enters the light sensing side of the rolling shutter type image sensor. When the total reflection prism is located at the second position, the light beam entering the total reflection prism is totally reflected. The roller shutter type image sensor is used for receiving light beams transmitted by the total reflection prism module and converting optical signals into electric signals through photoelectric conversion. By controlling the rotation of the total reflection prism, the uncontrollable light source is turned on and turned off, the light is turned on and turned off quickly, and the problems of image distortion and the like are solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image acquisition device and a brightness balancing method. Background Art

[0002] In the field of video surveillance applications, cameras equipped with complementary metal-oxide-semiconductor (CMOS) image sensors are often used for video surveillance and high-definition snapshots. During operation, the photodiode of the CMOS sensor continuously captures incident photons and converts them into electrons stored in charge wells. The control unit can read or clear them, but cannot stop the exposure. A common solution is to control the exposure through a shutter. Commonly used shutters include rolling shutter (RS) and global shutter (GS). GS technology enables the entire picture to be exposed at the same time. All pixels in the image sensor sense light at the same time and are exposed at the same time.

[0003] Unlike GS, RS clears each pixel to zero at the start of exposure, then waits for the exposure time to complete before serially reading the signal value. RS technology can enable CMOS cameras to achieve faster speeds and frame rates, while also reducing manufacturing costs. However, in the case of improper exposure or rapid object movement, RS technology can result in partial exposure, ramped patterns (also known as the rolling shutter effect), and shaky images, resulting in image distortion. Therefore, how to use RS technology to eliminate image distortion and address the rolling shutter effect is a pressing technical issue in this field. Summary of the Invention

[0004] The present application provides an image acquisition device and a brightness balancing method. By controlling the rotation of a total reflection prism, an uncontrollable light source is turned on and off, thereby achieving rapid switching of light, solving problems such as image distortion, meeting the image acquisition requirements in scenes where objects are moving rapidly, achieving good image capture effects, and enhancing the user's visual experience.

[0005] In a first aspect, an image acquisition device is provided. The image acquisition device includes a rolling shutter image sensor and a total reflection prism module. The total reflection prism module is located on the photosensitive side of the rolling shutter image sensor, and the total reflection prism module includes a total reflection prism and a rotating structure. The rotating structure is used to drive the total reflection prism to rotate from a first position to a second position. When the total reflection prism is in the first position, the light beam incident on the total reflection prism is transmitted and incident on the photosensitive side of the rolling shutter image sensor. When the total reflection prism is in the second position, the light beam incident on the total reflection prism is totally reflected. The rolling shutter image sensor is used to receive the light beam transmitted by the total reflection prism module and convert the light signal into an electrical signal through photoelectric conversion.

[0006] Based on the above solution, by controlling the rotation of the total reflection prism, the uncontrollable light source can be turned on and off, the rapid switching of light is achieved, and problems such as image distortion are solved. The image acquisition requirements in scenes where objects are moving rapidly are met, good image shooting effects are achieved, and the user's visual experience is enhanced.

[0007] In conjunction with the first aspect, in certain implementations of the first aspect, a sandwich layer is provided within the total internal reflection prism. The sandwich layer includes an optically less dense medium, and the area of ​​the total internal reflection prism other than the sandwich layer includes an optically dense medium. The refractive index of the optically less dense medium is less than the refractive index of the optically dense medium.

[0008] Based on the above scheme, a total internal reflection prism consists of two media with different refractive indices. By controlling the angle of incidence of a light beam at the interface between the two media, the direction of the light beam's exit is controlled. By utilizing the principle of total internal reflection to turn the light beam on and off, this method achieves rapid light switching. The overall solution offers a simple and stable mechanical structure, low cost, and excellent light beam switching performance.

[0009] In some optional implementations, the total reflection prism includes a cube structure or a rectangular parallelepiped structure, which is not particularly limited in this application.

[0010] In combination with the first aspect, in some implementations of the first aspect, the optically sparse medium includes vacuum or air.

[0011] Based on the above solution, the total reflection prism includes setting a vacuum or air layer in the crystal, which simplifies the structure of the total reflection prism and reduces the production cost.

[0012] In conjunction with the first aspect, in certain implementations of the first aspect, when the total internal reflection prism is in the first position, the incident angle of the light beam incident on the critical surface between the optically less dense medium and the optically denser medium is a first angle, and the first angle is less than the critical angle of total internal reflection between the optically less dense medium and the optically denser medium.

[0013] Based on the above solution, when the total reflection prism is in the first position, the incident angle to the critical surface is smaller than the total reflection critical angle, and the light beam can be emitted normally and reach the photosensitive area of ​​the rolling shutter image sensor.

[0014] In conjunction with the first aspect, in certain implementations of the first aspect, the total reflection prism rotates from the first position around the rotating structure by a second angle to reach the second position, wherein the sum of the first angle and the second angle is greater than a critical angle for total reflection.

[0015] Based on this scheme, the total reflection prism rotates from its first position to a second angle around the rotating structure, thereby changing the incident angle of the light beam on the critical surface. The incident angle of the light beam on the critical surface is greater than the critical angle for total reflection at the interface, causing total reflection of the incident light beam. This prevents the light beam from reaching the photosensitive side of the image sensor, achieving rapid light switching. The overall device is simple and highly stable.

[0016] In conjunction with the first aspect, in certain implementations of the first aspect, the image acquisition device further includes a controller configured to control the rotating structure to drive the total reflection prism to switch between the first position and the second position.

[0017] Based on the above solution, the image acquisition device controls the rotation of the total reflection prism through the controller, thereby controlling the rapid switching of light.

[0018] It should be understood that the controller can send control information to the total reflection prism module to cause the total reflection prism module to rotate through manual triggering. The controller can also send control information to the total reflection prism module according to a preset timing to cause the total reflection prism module to rotate. The controller can also receive other indication information to send control information to the total reflection prism module. This application does not make any special restrictions on this.

[0019] In a second aspect, a brightness balancing method is provided. The method is applied to an image acquisition device. The image acquisition device includes a rolling shutter image sensor and a total reflection prism module. The total reflection prism module is located on the photosensitive side of the rolling shutter image sensor and includes a total reflection prism and a rotating structure. At a first moment, the rotating structure is controlled to drive the total reflection prism to rotate to a first position, causing the light beam incident on the total reflection prism to be transmitted and incident on the photosensitive side of the rolling shutter image sensor. The rolling shutter image sensor is used to receive the light beam transmitted by the total reflection prism and convert the light signal into an electrical signal through photoelectric conversion. At a second moment, the rotating structure is controlled to drive the total reflection prism to rotate to a second position, causing the light beam incident on the total reflection prism module to be totally reflected. The rolling shutter image sensor includes N rows of photosensitive cells. The period between the first moment and the second moment is the time during which the photosensitive cells of the rolling shutter image sensor from the first row to the Nth row are simultaneously exposed. N is a positive integer greater than or equal to 2.

[0020] Based on the above scheme, the light beam is turned on and off by controlling the rotation of the total reflection prism. By causing the incident light beam to be totally reflected by the total reflection prism, the light beam cannot reach the photosensitive area of ​​the image sensor, thereby turning off the light source. By controlling the total reflection prism to switch between the first position and the second position, N rows of photosensitive cells of the rolling shutter image sensor are exposed simultaneously, achieving global exposure and eliminating image distortion and jello effect caused by object movement. However, the overall image acquisition device has lower cost and faster response speed than the global shutter image acquisition device.

[0021] It should be understood that the present application can control the rotating structure to drive the total reflection prism through indication information, and can also control the rotating structure to drive the total reflection prism through mechanical connection, and the present application does not make any special limitation on this.

[0022] It should be understood that the beneficial technical effects of the second aspect include all the beneficial technical effects of the first aspect mentioned above, and this application will not go into details here.

[0023] In conjunction with the second aspect, in certain implementations of the second aspect, the first moment includes the moment when the Nth row of photosensitive units starts to be exposed, and the second moment includes the moment when the first row of photosensitive units stops being exposed.

[0024] It should be understood that the second moment is a moment after the first moment.

[0025] In conjunction with the second aspect, in certain implementations of the second aspect, the method further includes obtaining light source illumination information, setting a global exposure duration of the rolling shutter image sensor to a first duration, the first duration being between the first moment and the second moment.

[0026] Based on the above solution, the exposure duration is determined by obtaining the light source illumination information. The exposure duration includes the duration of simultaneous exposure of the first to Nth rows of photosensitive cells. The first duration is less than or equal to the duration between the first moment and the second moment.

[0027] In conjunction with the second aspect, in certain implementations of the second aspect, a layer is provided in the total reflection prism, the layer comprising a less optically dense medium, and the area of ​​the total reflection prism other than the layer comprises a denser medium, the refractive index of the less optically dense medium being less than the refractive index of the denser medium.

[0028] In conjunction with the second aspect, in certain implementations of the second aspect, the optically sparse medium includes vacuum or air.

[0029] In combination with the second aspect, in certain implementations of the second aspect, when the total reflection prism is in the first position, the incident angle of the light beam incident on the critical surface of the optically sparse medium and the optically dense medium is a first angle, and the first angle is less than the total reflection critical angle between the optically sparse medium and the optically dense medium.

[0030] In combination with the second aspect, in some implementations of the second aspect, the total reflection prism rotates from the first position around the rotating structure by a second angle to reach the second position, and the sum of the first angle and the second angle is greater than the critical angle of total reflection.

[0031] In a third aspect, embodiments of the present application provide a control device, which may be a controller or any control device including a controller. The device includes at least one processor configured to execute a computer program or instructions stored in at least one memory to implement the method described in the second aspect or any possible implementation of the second aspect.

[0032] Optionally, the processor is coupled to the memory, or the memory and the processor may be deployed separately or centrally.

[0033] Optionally, the device further comprises an optical communication interface, and the processor is coupled to the optical communication interface. In one implementation, the optical communication interface may be an optical transceiver, or an optical input / output interface.

[0034] In a fourth aspect, embodiments of the present application provide a computer-readable storage medium. The computer-readable storage medium stores a computer program (also referred to as code or instructions) that, when executed on a computer, causes the computer to perform the method of the second aspect or any possible implementation of the second aspect.

[0035] In a fifth aspect, embodiments of the present application provide a computer program product. The computer program product includes a computer program (also referred to as code or instructions), which, when executed, causes a computer to perform the method of the second aspect or any possible implementation of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The exposure timing diagrams corresponding to a rolling shutter and a global shutter are shown.

[0037] Figure 2 A schematic diagram showing a "pseudo-global" shutter mode.

[0038] Figure 3 This is an image acquisition device provided in an embodiment of the present application.

[0039] Figure 4 3 is a rotation diagram of a total reflection prism module 320 provided in an embodiment of the present application.

[0040] Figure 5 This is a flow chart of a brightness balancing method provided in an embodiment of the present application.

[0041] Figure 6This is a schematic diagram of a sensor exposure time setting interval provided in an embodiment of the present application.

[0042] Figure 7 This is a controller structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solution in this application will be described below with reference to the accompanying drawings.

[0044] It should be noted that, in the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in this article is merely a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two, and “at least one” and “one or more” refer to one, two or more. The singular expressions “a”, “an”, “said”, “the” and “this” are intended to also include expressions such as “one or more”, unless there is a clear indication to the contrary in the context.

[0045] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0046] In the description of the embodiments of the present application, the terms "up", "down", "left", "right", "vertical", "horizontal", etc. indicate orientations or positional relationships that are defined relative to the orientations or positions of the components schematically placed in the drawings. It should be understood that these directional terms are relative concepts. They are used for relative descriptions and clarifications, rather than indicating or implying that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation. They may change accordingly according to changes in the orientation of the components placed in the drawings, and therefore cannot be understood as limitations on the present application.

[0047] In the embodiments of the present application, the same reference numerals are used to represent the same component or the same element. In addition, the components in the drawings are not drawn to scale, and the sizes and dimensions of the components shown in the drawings are only exemplary and should not be understood as limiting the present application.

[0048] In certain specialized video surveillance applications, cameras using commonly used CMOS image sensors cannot meet the demands of high-definition snapshots. The operating mode of CMOS sensors does not allow the start and end of exposure to be controlled by a single signal line. The photodiodes in CMOS sensors continuously capture incoming photons and convert them into electrons, storing them in charge wells. The control unit can read out or reset these electrons, but it cannot stop exposure. The shutter is the device in the camera that controls exposure time. Currently, two types of shutters are commonly used: the global shutter (GS) and the rolling shutter (RS).

[0049] The following is an illustrative introduction to some terms related to the solutions in the embodiments of this application:

[0050] (1) Exposure.

[0051] Exposure in the embodiment of the present application refers to the entire process in which light is irradiated onto the image sensor through the lens and a photoelectric reaction is generated on the image sensor. The image sensor includes a plurality of photosensitive units, and each of the plurality of photosensitive units can receive the light irradiated onto the photosensitive unit and convert it into an electrical signal when in a working state. In other words, the conditions under which each photosensitive unit can perform exposure include: the photosensitive unit is in a working state, and light is irradiated onto the photosensitive unit. As long as one of these two conditions is not met (for example, although there is light irradiating the photosensitive unit, the photosensitive unit is in a dormant state, or although the photosensitive unit is in a working state, no light is irradiated onto the photosensitive unit, or the photosensitive unit is in a dormant state and no light is irradiated onto the photosensitive unit), the photosensitive unit cannot perform exposure.

[0052] (2) Preset exposure time and actual exposure time.

[0053] The preset exposure time and actual exposure time in the embodiments of the present application are both for the photosensitive unit. The actual exposure time refers to the time interval during which the photosensitive unit actually performs exposure. The preset exposure time refers to the time interval from when the photosensitive unit is switched to the working state to when it is switched to the sleep state. This time interval can be set in the image sensor by a person skilled in the art in software. In this case, if the image sensor receives an exposure instruction, the image sensor will first switch the photosensitive unit to the working state, and then switch the photosensitive unit to the sleep state after the preset exposure time has been set. During this time interval, the photosensitive unit may have received light (i.e., actually performed exposure), or may not have received light (i.e., not actually performed exposure).

[0054] (3) Rolling shutter.

[0055] RS clears any pixel at the start of exposure, then waits for the exposure time to elapse before reading out the signal value. Data readout is serial, so clearing / exposure / readout can only be performed row by row, usually from top to bottom. Rolling shutter CMOS sensors equip each column of pixels with an analog-to-digital converter (ADC), which can significantly shorten conversion time and enable CMOS cameras to have faster conversion speeds and frame rates. Each row of pixels on the sensor begins exposing the next frame after completing the readout of the previous frame, resulting in faster speeds, higher frame rates, and lower overall costs for CMOS cameras.

[0056] The rolling shutter in the embodiment of the present application refers to a shutter that uses a row-by-row exposure method for imaging. In the rolling shutter, the photosensitive units in the image sensor can be divided into N (N is a positive integer) rows of photosensitive units, and each row of photosensitive units in the N rows of photosensitive units can correspond to a row of areas on the actual imaged picture. Each row of photosensitive units is used to receive light from a row of areas corresponding to the actual scene and expose and image the row of areas on the actual imaged picture (that is, generate an image signal corresponding to the row of areas). In this case, the row-by-row exposure in the rolling shutter refers to sequentially selecting N rows of photosensitive units from top to bottom, that is, switching N rows of photosensitive units to the working state in turn to control the N rows of photosensitive units to be exposed in turn. The preset exposure time lengths of these N rows of photosensitive units can be the same, different, or partially the same and partially different, and there is no specific limitation.

[0057] (4) Global shutter.

[0058] The global shutter in the embodiments of this application refers to a shutter that simultaneously exposes all photosensitive cells for imaging. When the shutter closes, all photosensitive cells on a global shutter image sensor end exposure simultaneously. Global shutter image sensors have a built-in buffer that can store the signals collected by the photosensitive cells after exposure, facilitating continuous signal readout. Therefore, a global shutter image sensor supports simultaneous exposure start and end for all photosensitive cells.

[0059] It should be understood that the photosensitive cells in the rolling shutter and global shutter are used to convert light signals into electrical signals during the exposure process, thereby recording the light signal information. For example, the photosensitive cells in the image sensor may be, for example, photosensors. Exposure of the photosensitive cells refers to light irradiating the photosensitive cells, causing them to convert the light signals into electrical signals.

[0060] Figure 1 The following diagram shows the exposure timings for a rolling shutter and a global shutter. For example, N rows of photosites all have the same preset exposure duration. In this example, it is assumed that the N rows of photosites are named the first row (bottom row), the second row, ..., and the Nth row (top row), and that each row has the same preset exposure duration. Figure 1 (a) shows a timing diagram of a rolling shutter image sensor using row-by-row exposure. Figure 1 (b) in FIG. 1 shows a timing diagram of a global shutter image sensor in which all photosensitive units start exposure and end exposure at the same time. Figure 1 As shown in (a), the exposure time period of the first row of photosensitive units is [t0, t1], and the starting and ending points of the exposure time of each subsequent row of photosensitive units are different. Among them, the time difference between the exposure end time points of two adjacent rows of photosensitive units is just the time required to read out the signal of one row of photosensitive units, that is, when the signal collected by the previous row of photosensitive units is read out, the exposure of the next row of photosensitive units is just finished, so as to continue to read out the signal collected by the next row of photosensitive units. Since the exposure start time and end time of each row of photosensitive units are different, when the photographed object is in a high-speed motion state, the image will be deformed. Since the deformation of the image is similar to the deformation caused by jelly being squeezed, the phenomenon that occurs when the rolling shutter image sensor shoots a moving object is called the "jelly" effect. As shown in (a) of FIG. Figure 1 As shown in (b) of the figure, the exposure time period for each row of photosensors in a global shutter image sensor is [t0, t1]. Since the exposure time period for all photosensors is the same, there is no image distortion when using a global shutter image sensor to capture high-speed moving objects.

[0061] (5)Jello effect.

[0062] The jello effect in the embodiment of the present application occurs in a scene where a rolling shutter is used to shoot a moving object. The movement mentioned here refers to relative movement. For example, when a camera in dynamic motion shoots a static object, or when a static camera shoots a dynamic moving object, the object being shot is moving relative to the camera. Figure 1 From the timing relationship of the exposure times of the N rows of photosensitive cells of the RS-type image sensor shown in (a), it can be seen that when light is always irradiated on the N rows of photosensitive cells, these N rows of photosensitive cells actually sense light and form images in different time periods. In this case, if the object being photographed moves relative to the camera device, the position reference of the picture captured by these N rows of photosensitive cells in their respective light-sensing time periods also changes, causing the N areas corresponding to these N rows of photosensitive cells to tilt when they are combined into a complete image.

[0063] In summary, the use of a global shutter image sensor can eliminate distortion in the captured image to a certain extent, but the image acquisition device of a global shutter image sensor is relatively expensive. Furthermore, GS requires all pixels on the entire screen to be exposed simultaneously in order to complete the imaging of the entire screen at the same time. RS exposes each row of pixels in sequence until all pixels are exposed. Because signal interference occurs between the photosensitive elements corresponding to different pixels during exposure, when GS exposes all pixels simultaneously, the photosensitive elements corresponding to each pixel interfere with each other at the same time, resulting in severe noise. RS, on the other hand, uses a row-by-row exposure method, so there is a time difference between the interference between the photosensitive elements corresponding to pixels in different rows, resulting in relatively less noise. For this reason, RS is currently preferred in the field of imaging. Compared with GS, RS is not only less expensive, but also has higher acquisition speed and sensitivity.

[0064] However, existing RS structures still have some difficult-to-overcome drawbacks. For example, when the exposure is inappropriate or the object moves quickly, the RS mode can exhibit partial exposure, skew (also known as the rolling shutter effect), and wobble. Therefore, eliminating the distortion associated with RS is a pressing technical issue in the field.

[0065] Figure 2 A schematic diagram of a "pseudo-global" shutter mode is shown. In order to retain the advantages of the RS mode and eliminate the image distortion caused by the RS mode, a "pseudo-global" shutter mode is often used for imaging.

[0066] It should be understood that in some specific implementations, the "pseudo-global" shutter mode is also called the "full row" mode, and this application does not make any special limitation on this.

[0067] The "pseudo-global" shutter mode combines the high frame rate of RS with the global exposure of GS, turning on the light source only when all rows in the image frame can be exposed simultaneously. Compared to GS sensors, the "pseudo-global" shutter mode eliminates the image distortion that can be caused by the RS mode's row-by-row exposure while still allowing for faster frame rates and lower read noise and dark current.

[0068] However, the "pseudo-global" shutter mode has certain limitations and can only be used in environments with controllable lighting. If the light source is uncontrollable, turning it on and off is impossible. For example, during the day, the ambient sunlight is uncontrollable, and the light source cannot be manually turned on and off, so the "pseudo-global" shutter mode cannot be used.

[0069] Based on this, the present application provides an image acquisition device that can eliminate image distortion caused by the RS effect when using RS, and can also work under external uncontrollable light sources.

[0070] To facilitate understanding of the image acquisition device provided in the embodiments of this application, we first explain some of its possible application scenarios. In one optional implementation, the image acquisition device provided in the embodiments of this application can be used in scenarios where high-speed moving objects are captured. This includes, but is not limited to, intelligent transportation scenarios, where vehicles on the road are captured. For example, in a checkpoint scenario, the vehicles to be captured on the road are moving at high speeds, reaching speeds of over 100 kilometers per hour for extended periods of time.

[0071] Figure 3 The image acquisition device provided in the embodiment of the present application includes a lens module 310 , a total reflection prism module 320 , a sensor 330 and a controller 340 .

[0072] The light beam emitted by the light source illuminates the object being photographed, and the reflected light from the object enters the lens module 310. The light then passes through the lens module 310 and exits into the total reflection prism module 320. The light beam emitted by the total reflection prism module 320 is received by the sensor 330. The controller 340 is coupled to the sensor 330 and the total reflection prism module 320. Based on parameters such as the exposure time and delay time, the controller 340 sends a drive signal to the total reflection prism module 320, causing the total reflection prism module 320 to rapidly rotate around the rotating structure 321 at a specific angle to achieve the desired on-off switching of the light beam.

[0073] It should be understood that the light sources described in this application include light sources that cannot be controlled by humans, such as sunlight, as well as other lighting devices, fill light devices, etc. This application does not impose any special limitations on this.

[0074] It should be understood that the photographed objects described in this application include portraits, objects, vehicles, scenery, etc. The photographed objects can be still objects or fast-moving objects, and this application does not impose any special restrictions on this.

[0075] It should be understood that the lens module 310 described herein includes the lenses required for capturing an object for imaging. Those skilled in the art may configure the lens module 310 based on specific usage scenarios, and this application does not impose any specific limitations thereon. The lens module 310 described herein includes, but is not limited to, lenses, lens holders, motors, and other structures.

[0076] In an optional implementation, in the embodiment of the present application, the lens in the lens module 310 can be a zoom lens or a fixed focus lens, without specific limitation. When the lens in the lens module 310 is a zoom lens, those skilled in the art can select the zoom range of the lens, the aperture range of the lens, and the resolution of the image sensor according to the scene in which the image acquisition device will be used after leaving the factory during the development stage of the image acquisition device. In this way, the user can subsequently select the required focal length and aperture according to the shooting scene they want during use, and use the image sensor of this resolution to shoot images or videos. For example, in the field of surveillance cameras used for security functions, the zoom range of the zoom lens can be set to 10mm to 30mm, the aperture can be set to a fixed aperture of F / 1.7 (several optional values ​​can also be set, and the user can select one of the values ​​to perform the current shooting task when using it), and the resolution of the image sensor can be set to 1920x1000.

[0077] The sensor 330 serves as a receiving device for the imaging information of the entire image acquisition device. The controller 340 drives the total reflection prism module 320 to rotate around the rotating structure 321 through the exposure time and the related delay time set by the sensor 330, thereby changing the transmission direction of the light beam emitted by the total reflection prism module 320, thereby realizing fast switching of light. Figure 4 The structure and rotation of the total reflection prism module 320 are described.

[0078] Figure 4 3 is a rotation diagram of a total reflection prism module 320 provided in an embodiment of the present application.

[0079] The total reflection prism module 320 includes a rotating structure 321, a rarefaction medium 322, and a denser medium 323. The refractive index of the rarefaction medium 322 is n1, and the refractive index of the denser medium 323 is n2, where n2 is greater than n1. Different media have different refractive indices. A medium with a smaller refractive index is called a rarefaction medium, and a medium with a larger refractive index is called a denser medium. From the relationship It can be seen that the propagation speed of light in a denser medium is smaller than that in a less dense medium. Where c represents the speed of light in a vacuum, v represents the speed of light when light enters the medium, and n represents the refractive index of the medium. According to the law of refraction, n1 sinθ1 = n2 sinθ2, it can be seen that when moving from a denser medium to a less dense medium, the angle of refraction is greater than the angle of incidence. When the angle of refraction is just close to the boundary surface of the medium (evanescent wave), the angle of incidence (critical angle of total reflection) is recorded as θ c ,

[0080] When the total reflection prism module 320 is in its initial position, the incident light is totally reflected by the total reflection prism module 320. The incident light beam passes through the total reflection prism module 320 and reaches the sensor 330. When the total reflection prism module 320 receives a driving signal from the controller 340, it is driven to rotate around the rotating structure 321 at an angle of θ, causing the incident light to be totally reflected by the total reflection prism module 320. The sensor 330 cannot receive the light beam and is in a beam-interrupted state.

[0081] It should be understood that the prism composed of the optically sparse medium 322 and the optically dense medium 323 in the total reflection prism module 320 is a specific implementation of the total reflection prism in the above implementation, and does not constitute any limitation to the scope of protection of this application.

[0082] It should be understood that the initial position of the total reflection prism module 320 is a specific implementation of the first position in the above implementation, and does not constitute any limitation on the protection scope of the present application.

[0083] It should be understood that the position of the total reflection prism module 320 after rotating by θ degrees based on the initial position is a specific implementation of the second position in the above implementation, and does not constitute any limitation to the protection scope of the present application.

[0084] In view of this, the present application also provides a brightness balancing method to adaptively balance the brightness of different photosensitive units, thereby avoiding brightness differences caused by inconsistent exposure times.

[0085] The embodiment of the present application provides a brightness balancing method, which is applied to an image acquisition device. Specifically, the structure of the image acquisition device can refer to the above Figures 3 and 4 The corresponding embodiments will not be described again in detail.

[0086] Figure 5 This is a flow chart of a brightness balancing method provided in an embodiment of the present application.

[0087] Step 501 : Obtain light source illumination information, and set the aperture number of the lens module 310 and the exposure time T of the sensor 330 .

[0088] Obtaining light source illumination information should be understood as obtaining illumination information of all light sources irradiating the photographed object, including obtaining illumination information of natural light sources that cannot be artificially controlled, such as sunlight, and also including obtaining illumination information of artificially controllable light sources, such as fill lights.

[0089] The controller 340 sets the relevant parameters of the lens module 310 according to the light source illumination information. Specifically, it includes parameters such as the aperture number. It also sets the exposure time T of the sensor 330 according to the light source illumination information. The setting of the exposure time T of the sensor 330 is as follows: Figure 6 shown.

[0090] Figure 6 This is a schematic diagram of a sensor exposure time setting interval provided in an embodiment of the present application.

[0091] As shown in the figure, there are N rows of photosensitive units, where N is a natural number greater than or equal to 2. The preset exposure duration corresponding to each row of photosensitive units is the same. The first row of photosensitive units starts exposure at time T0 and ends exposure at time T2. The exposure duration is [T0, T2], that is, the exposure duration is (T2-T0). The exposure start time intervals of two adjacent rows of photosensitive units are the same. The second to Nth rows of photosensitive units start exposure after time T0 in sequence. The Nth row of photosensitive units starts exposure at time T1, which is a moment between [T0, T2]. Time T1 is greater than time T0 and less than time T2. That is, the start exposure moment of the Nth row of photosensitive units is within the exposure state of the first row of photosensitive units.

[0092] The exposure time T is set to the time period between [T1, T2], that is, T = T2 - T1. In the time period between [T1, T2], the first row to the Nth row of photosensitive cells are exposed simultaneously.

[0093] It should be understood that the exposure time T is a specific implementation of the first duration in the above implementation and does not constitute any limitation on the protection scope of this application.

[0094] It should be understood that the T1 moment is a specific implementation of the first moment in the above implementation, and does not constitute any limitation to the protection scope of this application.

[0095] It should be understood that the T2 moment is a specific implementation of the second moment in the above implementation and does not constitute any limitation on the protection scope of this application.

[0096] Step 502: Send instruction information 1.

[0097] Instruction message 1 is used to instruct the image acquisition device to perform a global reset to achieve simultaneous exposure of each row of photosensitive cells. Controller 340 sends instruction message 1, which instructs total reflection prism module 320 to reset, thereby causing sensor 330 to initiate a global reset exposure command at time T1. Sensor 330's exposure time is T, resulting in simultaneous exposure of photosensitive cells in rows 1 through N of sensor 330.

[0098] At this time, the light beam incident on the total reflection prism module 320 is in a passing state and is received by the sensor 330. The light beam reflected from the photographed object passes through the lens module 310 and is incident on the total reflection prism module 320. In the total reflection prism module 320, the light beam is incident from the optically dense medium 323 to the optically less dense medium 322. The incident angle of the light beam on the interface between the optically dense medium 323 and the optically less dense medium 322 is θ0, θ0 < θ c . Among them, θ c is the critical angle of total internal reflection of the optically less dense medium 322 and the optically dense medium 323. n1 is the refractive index of the optically less dense medium 322 , and n2 is the refractive index of the optically dense medium 323 .

[0099] Step 503: Send instruction information 2.

[0100] Instruction information 2 is used to instruct the total reflection prism module 320 to rotate Δθ degrees around the rotating structure 321 at time T2, so that the incident light beam incident on the total reflection prism module 320 is totally reflected at the interface between the optically rarefying medium 322 and the optically dense medium 323, thereby preventing the light beam from reaching the receiving surface of the sensor 330. The controller 340 sends instruction information 2, which is used to instruct the total reflection prism module 320 to rotate Δθ degrees around the rotating structure 321 at time T2. The rotation angle Δθ satisfies Δθ+θ0>θ c As a result, the incident light beam entering the total reflection prism module 320 is totally reflected at the interface between the two media, changing the emission direction of the light beam, and no light beam is received on the receiving surface of the sensor 330 .

[0101] It should be understood that in an optional implementation, at time T2, sensor 330 sends instruction information to controller 340, instructing controller 340 to send instruction information 2 to total reflection prism module 320, thereby instructing total reflection prism module 320 to deflect and change the propagation direction of the light beam. The above implementation is merely a specific optional implementation and should not constitute any limitation on the scope of protection of this application.

[0102] It should be understood that in another optional implementation, after sending instruction information 1, sensor 330 automatically sends instruction information 2 to total reflection prism module 320, instructing total reflection prism module 320 to deflect at time T2 to change the propagation direction of the light beam. The above implementation is merely a specific optional implementation and should not constitute any limitation on the scope of protection of this application.

[0103] Step 504: Stop exposure.

[0104] Since the total reflection prism module 320 is deflected at time T2, the light beam cannot reach the receiving surface of the sensor 330. Therefore, the sensor 330 cannot receive image information at time T2 and the exposure ends.

[0105] It should be understood that in an optional implementation, after receiving instruction information 1, sensor 330 initiates a global reset exposure command at time T1. Instruction information 1 is also used to instruct automatic termination of exposure after exposure time T, that is, automatic termination of exposure at time T2. The above implementation is merely a specific optional implementation and does not limit the scope of protection of this application.

[0106] It should be understood that in another optional implementation, the receiving surface of the sensor 330 does not receive image information at time T2, and the sensor 330 automatically stops exposure. The above implementation is only a specific optional implementation and should not constitute any limitation on the scope of protection of this application.

[0107] It should be understood that in another optional implementation, the controller 340 sends instruction information 3 to the sensor 330 to instruct the sensor 330 to stop exposure. The above implementation is only a specific optional implementation and should not constitute any limitation on the scope of protection of this application.

[0108] It should be understood that the controller 340 may be a chip. For example, the controller may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0109] During implementation, each step of the brightness balancing method in the present application can be completed by an integrated logic circuit of the hardware in the controller or by instructions in the form of software. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the controller. The software module can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the controller reads the information in the memory and, in combination with its hardware, completes the steps of the brightness balancing method in the present application.

[0110] It should be noted that the controller in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the brightness balancing method in the present application can be completed by hardware integrated logic circuits in the controller or by software instructions. The above-mentioned controller can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in combination with its hardware, completes the steps of the above-mentioned method.

[0111] It is understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0112] Figure 7 This is a controller structure provided in an embodiment of the present application.

[0113] like Figure 7 As shown, the controller 340 may include a processor 1101 and a memory 1102. In the embodiments of the present application, the processor 1101 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application may be directly implemented and executed by a hardware processor, or by a combination of hardware and software components within the processor. The program code executed by the processor 1101 to implement the above methods may be stored in the memory 1102. The memory 1102 and the processor 1101 are coupled. Coupling in the embodiments of the present application refers to an indirect coupling or communication connection between devices, units, or modules, and may be electrical, mechanical, or other forms, used for information exchange between the devices, units, or modules. The processor 1101 may operate in conjunction with the memory 1102. The memory 1102 may be a non-volatile memory, such as a hard disk drive (HDD), or a volatile memory, such as a random-access memory (RAM). The memory 1102 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0114] Based on the above embodiments, embodiments of the present application further provide a computer storage medium. This storage medium stores a software program that, when read and executed by one or more processors, can implement the methods provided by any one or more of the above embodiments. The computer storage medium may include various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0115] Based on the above embodiments, embodiments of the present application further provide a chip. The chip includes a processor configured to implement the functions described in any one or more of the above embodiments, such as acquiring or processing the data frames described in the above methods. Optionally, the chip also includes a memory. The memory is used to store the necessary program instructions and data for execution by the processor. The chip may be comprised of a single chip or may include a chip and other discrete components.

[0116] The embodiment of the present application further provides a computer program product comprising instructions, which, when executed by a computer, enables the computer to implement the method executed by the controller 340 in the above method embodiment.

[0117] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0119] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0120] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0121] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An image acquisition device, characterized in that: include: A rolling shutter image sensor and a total reflection prism module, wherein the total reflection prism module is located on the photosensitive side of the rolling shutter image sensor, and the total reflection prism module includes a total reflection prism and a rotating structure; The rotating structure is used to drive the total reflection prism to rotate from the first position to the second position; When the total reflection prism is located at the first position, the light beam incident on the total reflection prism is transmitted and incident on the photosensitive side of the rolling shutter image sensor; When the total reflection prism is located at the second position, the light beam incident on the total reflection prism is totally reflected; The rolling shutter image sensor is used to receive the light beam transmitted by the total reflection prism module and convert the light signal into an electrical signal through photoelectric conversion.

2. The device according to claim 1, characterized in that An interlayer is provided in the total reflection prism, the interlayer includes an optically sparse medium, and the area of ​​the total reflection prism other than the interlayer includes an optically dense medium, and the refractive index of the optically sparse medium is smaller than the refractive index of the optically dense medium.

3. The device according to claim 2, characterized in that The optically sparse medium includes vacuum or air.

4. The device according to claim 2 or 3, characterized in that When the total reflection prism is located at the first position, the incident angle of the light beam incident on the critical surface of the optically sparse medium and the optically dense medium is a first angle, and the first angle is smaller than the total reflection critical angle between the optically sparse medium and the optically dense medium.

5. The device according to claim 4, characterized in that The total reflection prism rotates from the first position around the rotating structure by a second angle to reach the second position, and the sum of the first angle and the second angle is greater than the total reflection critical angle.

6. The device according to any one of claims 1 to 5, characterized in that Also includes: A controller is used to control the rotating structure to drive the total reflection prism to switch between the first position and the second position.

7. A brightness balancing method, characterized in that: The method is applied to an image acquisition device, the image acquisition device including a rolling shutter image sensor and a total reflection prism module, the total reflection prism module being located on the photosensitive side of the rolling shutter image sensor, the total reflection prism module including a total reflection prism and a rotating structure; the method comprising: At a first moment, the rotating structure is controlled to drive the total reflection prism to rotate to a first position, so that a light beam incident on the total reflection prism is transmitted and incident on a photosensitive side of a rolling shutter image sensor, wherein the rolling shutter image sensor is configured to receive the light beam transmitted by the total reflection prism and convert the light signal into an electrical signal through photoelectric conversion; At a second moment, the rotating structure is controlled to drive the total reflection prism to rotate to a second position, so that the light beam incident on the total reflection prism module is totally reflected; The rolling shutter image sensor includes N rows of photosensitive units, and the time between the first moment and the second moment is the time when the first to Nth rows of photosensitive units of the rolling shutter image sensor are simultaneously exposed, where N is a positive integer greater than or equal to 2.

8. The method according to claim 7, characterized in that The first moment includes the moment when the Nth row of photosensitive units begins to be exposed; The second moment includes the moment when the first row of photosensitive units stops exposing.

9. The method according to claim 7 or 8, characterized in that Also includes: Light source illumination information is obtained to set a global exposure duration of the rolling shutter image sensor to a first duration, where the first duration is between the first moment and the second moment.

10. The method according to any one of claims 7 to 9, characterized in that An interlayer is provided in the total reflection prism, the interlayer includes an optically sparse medium, and the area of ​​the total reflection prism other than the interlayer includes an optically dense medium, and the refractive index of the optically sparse medium is smaller than the refractive index of the optically dense medium.

11. The method according to claim 10, characterized in that The optically sparse medium includes vacuum or air.

12. The method according to claim 10 or 11, characterized in that When the total reflection prism is located at the first position, the incident angle of the light beam incident on the critical surface of the optically sparse medium and the optically dense medium is a first angle, and the first angle is smaller than the total reflection critical angle between the optically sparse medium and the optically dense medium.

13. The method according to claim 12, characterized in that The total reflection prism rotates from the first position around the rotating structure by a second angle to reach the second position, and the sum of the first angle and the second angle is greater than the total reflection critical angle.

14. A control device, characterized in that: The control device comprises at least one processor configured to execute a computer program or instruction stored in at least one memory, so that the control device performs the method according to any one of claims 7 to 13.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions. When the computer instructions are executed on a computer, the method according to any one of claims 7 to 13 is executed.

16. A computer program product, characterized in that The present invention comprises computer executable codes or computer executable instructions, which enable the method according to any one of claims 7 to 13 to be implemented when the computer executable codes or computer executable instructions are executed by a computer.

Images