Camera module and vehicle

Abstract

The invention provides a camera module and a vehicle. The camera module comprises a lens group used for generating and transmitting first light; the beam splitter is arranged on the first side of the lens group and used for receiving the first light and splitting the first light into second light and third light; the camera sensor is used for receiving the second light and generating a first image; and the event camera sensor is used for receiving the third light and generating a second image. According to the camera module provided by the invention, the camera sensor and the event camera sensor are combined, so that the detection capability of the camera can be improved, the environmental adaptability of the camera module is improved, the influence of severe weather on the sensors is eliminated, and effective sensing of the environment is realized.

Description

Technical Field

[0001] This invention relates to the field of camera technology, and in particular to a camera module and a vehicle. Background Technology

[0002] Currently, camera modules on vehicles typically acquire images of the external environment solely through their camera sensors.

[0003] Camera modules are greatly affected by the environment. For example, on snowy or rainy days, snowflakes or raindrops can obstruct the view, and the images generated by the camera sensor will be filled with snowflakes and raindrops, making it difficult for the camera module to effectively perceive the external environment. Summary of the Invention

[0004] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. The summary section of this invention is not intended to limit the key features and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.

[0005] The present invention is proposed to address the aforementioned problems. According to one aspect of the present invention, a camera module is provided, comprising: a lens assembly for generating and propagating a first light ray; a beam splitter disposed on a first side of the lens assembly for receiving the first light ray and splitting the first light ray into a second light ray and a third light ray; a camera sensor for receiving the second light ray and generating a first image; and an event camera sensor for receiving the third light ray and generating a second image.

[0006] In one embodiment of the present invention, the camera module includes a first scanner disposed on the second side of the lens group. The first scanner is used to propagate ambient light to the lens group and change the propagation direction of the ambient light.

[0007] In one embodiment of the present invention, the camera module includes a second scanner, which is connected to the beam splitter, the camera sensor and the event camera sensor. The second scanner is used to drive the beam splitter, the camera sensor and the event camera sensor to move, thereby changing the propagation direction of the first light.

[0008] In one embodiment of the present invention, the camera module includes a third scanner connected to the lens group, the third scanner being used to drive the lens group to move, thereby changing the propagation direction of the first light.

[0009] In one embodiment of the present invention, the camera sensor and the event camera sensor are one of a point image sensor, a line image sensor, and a block image sensor.

[0010] In one embodiment of the present invention, the camera sensor and the event camera sensor are area array image sensors.

[0011] In one embodiment of the present invention, the camera sensor has a first light receiving area and a second light receiving area, wherein the pixel density of the second light receiving area is greater than the pixel density of the first light receiving area.

[0012] In one embodiment of the present invention, the camera sensor is one of a charge-coupled element sensor and a complementary metal-oxide-semiconductor sensor;

[0013] The event camera sensor is one of the following: a dynamic vision sensor, an asynchronous time-based image sensor, and a dynamic active pixel vision sensor.

[0014] In one embodiment of the present invention, the propagation direction of the second light ray is different from that of the third light ray.

[0015] According to another aspect of the present invention, a vehicle is provided, the vehicle including the aforementioned camera module.

[0016] According to the camera module and vehicle of this application, by setting up a beam splitter, a camera sensor and an event camera sensor, it is possible to effectively acquire regular images and dynamic images of moving objects. Thus, on snowy and rainy days, the dynamic images of snowflakes and raindrops can be used to remove snowflakes and raindrops from regular images. As a result, the camera module can perceive the external environment more effectively and has better environmental adaptability. Attached Figure Description

[0017] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0018] Figure 1 This is a schematic diagram of a camera module according to a first embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of a camera sensor according to the first embodiment of the present invention;

[0020] Figure 3This is a schematic diagram of a camera module according to a second embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of a camera module according to a third embodiment of the present invention;

[0022] Figure 5 This is a schematic diagram of a camera module according to a fourth embodiment of the present invention. Detailed Implementation

[0023] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0024] It should be understood that the invention can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein. Rather, providing these embodiments will make the disclosure thorough and complete, and will fully convey the scope of the invention to those skilled in the art. In the drawings, for clarity, the dimensions and relative dimensions of layers and regions may be exaggerated. The same reference numerals denote the same elements throughout.

[0025] It should be understood that when an element or layer is referred to as "on," "adjacent to," "connected to," or "coupled to" other elements or layers, it may be directly on, adjacent to, connected to, or coupled to other elements or layers, or there may be intervening elements or layers. Conversely, when an element is referred to as "directly on," "directly adjacent to," "directly connected to," or "directly coupled to" other elements or layers, there are no intervening elements or layers. It should be understood that although the terms first, second, third, etc., may be used to describe various elements, components, areas, layers, and / or portions, these elements, components, areas, layers, and / or portions should not be limited by these terms. These terms are only used to distinguish one element, component, area, layer, or portion from another element, component, area, layer, or portion. Therefore, without departing from the teachings of this invention, the first element, component, area, layer, or portion discussed below may be referred to as the second element, component, area, layer, or portion.

[0026] Spatial relation terms such as “below,” “under,” “below,” “under,” “above,” “above,” etc., are used herein for convenience of description to describe the relationship between one element or feature shown in the figure and other elements or features. It should be understood that, in addition to the orientation shown in the figure, spatial relation terms are intended to also include different orientations of the device in use and operation. For example, if the device in the figure is flipped, then the element or feature described as “below” or “under” the other element or feature will be oriented “above” the other element or feature. Therefore, the exemplary terms “below” and “under” can include both upper and lower orientations. The device may be otherwise oriented (rotated 90 degrees or otherwise) and the spatial descriptive terms used herein will be interpreted accordingly.

[0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. When used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprise” and / or “comprising,” when used in this specification, identify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups. When used herein, the term “and / or” includes any and all combinations of the associated listed items.

[0028] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms as defined in commonly used dictionaries shall be construed as having the meaning consistent with their meaning in the relevant field and / or the context of this specification, and shall not be interpreted in an ideal or overly formal sense, unless expressly defined herein.

[0029] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0030] Reference Figure 1The following is an illustrative description of a camera module according to a first embodiment of the present invention. This camera module can be a camera module for use in a vehicle. The camera module includes a lens assembly 101, a beam splitter 102, a camera sensor 103, and an event camera sensor 104.

[0031] Lens assembly 101 is used to generate and propagate the first ray 21. Specifically, lens assembly 101 collects ambient light 20. After the ambient light 20 enters lens assembly 101, lens assembly 101 generates the first ray 21 and propagates it outward. Lens assembly 101 may include multiple lenses. Lens assembly 101 can converge the light entering it and can also perform certain processing on the light, such as distortion correction and chromatic aberration correction.

[0032] Beam splitter 102 is disposed on the first side of lens group 101, that is, the side of lens group 101 facing beam splitter 102, for receiving first light 21 and splitting the first light 21 from lens group 101 into second light 22 and third light 23. Beam splitter 102 can be an optical device that splits a beam of light into two beams, typically composed of a metal film or a dielectric film. Exemplarily, beam splitter 102 can be a semi-transparent and semi-reflective lens, which can transmit part of the first light 21 incident on it and reflect part of it, thereby forming second light 22 and third light 23 that propagate in different directions. That is, the propagation direction of second light 22 and second light 23 are different; for example, the propagation directions of second light 22 and second light 23 can be perpendicular to each other. In some embodiments, beam splitter 102 can be a planar beam splitter or a cubic beam splitter. In some embodiments, beam splitter 102 can be a thin-film beam splitter, a crystal beam splitter, a Brewster window, and a wedge beam splitter. In some embodiments, the beam splitter 102 can be used to split the first light 21 into a second light 22 and a third light 23 that propagate in the same direction. For example, the second light 22 and the third light 23 can be two parallel beams of light spaced a certain distance apart.

[0033] A camera sensor 103 is positioned along the propagation direction of the second light ray 22 to receive the second light ray 22 and generate a first image. The camera sensor 103 has an image acquisition area, configured to acquire the received second light ray 22 through the image acquisition area (also called the photosensitive area or photosensitive surface). The camera sensor 103 is a device with photoelectric conversion function, capable of converting the optical signal of the second light ray 22 acquired on the image acquisition area into an electrical signal proportional to the optical signal. The camera sensor 103 can be a charge-coupled device (CCD) sensor or a complementary metal-oxide-semiconductor (CMOS) sensor; that is, the camera sensor 103 can be one of a CCD sensor or a CMOS sensor. Exemplarily, in this embodiment, the camera sensor 103 can be an area array image sensor, whose image acquisition area has a pixel array, used to acquire an image of the entire field of view of the camera module.

[0034] The event camera sensor 104 is positioned along the propagation direction of the third ray 23 to receive the third ray 23 and generate a second image. Unlike the camera sensor 103, the event camera sensor 104 only observes "motion" in the scene, specifically "changes in brightness." It only outputs the brightness change (1 or 0) of the corresponding pixel when there is a brightness change, offering advantages such as fast response, wide dynamic range, and no motion blur. Traditional cameras, to some extent, capture a static / still space, while the purpose of an event camera is to capture moving objects. For a single pixel, the event camera sensor 104 only outputs when the received light intensity changes. For example, if the brightness increases and exceeds a threshold, the corresponding pixel will output a brightness increase event. The event camera sensor 104 does not have the concept of frames; when the scene changes, it generates a series of pixel-level outputs. Each pixel of the event camera sensor 104 operates independently and asynchronously, resulting in a large dynamic range. In summary, the camera sensor 103 captures the scene at a fixed frame rate, with all pixels working synchronously. The event camera operates independently and asynchronously for each pixel, with a high sampling rate and output only changes in brightness. An event (brightness change) includes the time of occurrence, the pixel coordinates of the occurrence, and the polarity of the event. The polarity of the event indicates whether the brightness has increased or decreased compared to the previous sample. The event camera sensor 104 can be a Dynamic Vision Sensor (DVS), an Asynchronous Time-based Image Sensor (ATIS), or a Dynamic and Active Pixel Vision Sensor (DAVIS). That is, the event camera sensor 104 can be one of these three types. For example, in this embodiment, the event camera sensor 104 can be an area array image sensor, whose image acquisition area has a pixel array, used to acquire dynamic images of the entire field of view of the camera module.

[0035] According to the camera module of this application, on snowy or rainy days, a first image of the camera's field of view, i.e., a conventional frame image (containing raindrops or snowflakes), can be generated by the camera sensor 103. A second image of moving objects in the camera's field of view, i.e., a dynamic image of raindrops or snowflakes, can be generated by the event camera sensor 104. Therefore, the dynamic image of snowflakes and raindrops can be used to remove raindrops or snowflakes from the first image. Specifically, the first and second images can be interpolated to remove raindrops or snowflakes from the first image. Then, a filtering algorithm is used to filter the first image after removing raindrops or snowflakes, resulting in a relatively clear first image without raindrops or snowflakes. For example, the filtering method can be smoothing filtering, sharpening filtering, linear filtering, or nonlinear filtering. Thus, the camera module of this embodiment can effectively eliminate the obstruction of raindrops or snowflakes, more effectively perceive the external environment, and has better environmental adaptability. It should be noted that the camera module of this embodiment can not only eliminate the obstruction of raindrops or snowflakes but also eliminate the obstruction of any other type of moving object.

[0036] For example, the camera module also includes an image processor, which performs difference processing on the first image and the second image and uses a filtering algorithm to filter the image to output a first image that does not contain raindrops or snowflakes.

[0037] According to the camera module of this application, the lateral velocity can also be detected by calculating the changes of the target object in the camera sensor 103 and the event camera 104, combined with information such as timestamps. Specifically, the target object can be identified first based on the first image and the second image. Then, based on the pixel changes of the target object in the first and second images, the lateral pixel distance of the target object in the first and second images can be determined. Then, based on the correspondence between the pixel distance and the real distance, the real distance of the target object can be obtained. Then, based on the timestamp corresponding to the pixel changes, the movement time of the target object can be obtained. Finally, based on the real distance and the movement time, the lateral movement velocity of the target object can be obtained. The lateral movement velocity obtained based on the first image and the lateral movement velocity obtained based on the second image can be cross-checked to achieve more accurate velocity measurement.

[0038] For example, the camera module also includes an image processor, which is used to identify the target object based on the first image and the second image, and to calculate the lateral movement speed of the target object.

[0039] See appendix Figure 2In this embodiment, the camera sensor 103 has a first light-receiving area 1031 and a second light-receiving area 1032, which are used to receive second light 22. The pixel density of the second light-receiving area 1032 is greater than that of the first light-receiving area 1031. The second light-receiving area 1032 corresponds to the area in the field of view of the camera module that needs to be finely detected. For example, the second light-receiving area 1032 can be configured to capture an image in the center of the field of view of the camera module, and the first light-receiving area 1031 is arranged around the second light-receiving area 1032. In some embodiments, the second light-receiving area 1032 may include a plurality of sub-regions arranged at intervals, which can be configured as needed by those skilled in the art.

[0040] Corresponding to the camera sensor 103, the event camera sensor 104 also has a first light receiving area and a second light receiving area. The first and second light receiving areas are used to receive the third light 23, and the pixel density of the second light receiving area is greater than that of the first light receiving area. The configuration of the first and second light receiving areas on the event camera sensor 104 can be the same as the configuration of the first light receiving area 1031 and the second light receiving area 1032 on the camera sensor 103, and will not be described again here.

[0041] According to the camera module of this application embodiment, the first light receiving area 1031 and the second light receiving area 1032 on the camera sensor 103 can be controlled to acquire a first frame image with the same pixel density (e.g., the pixel density of the first light receiving area 1031, i.e., a relatively low pixel density). Then, it is determined whether there is a target object that needs to be finely identified in the first frame image and whether the target object is located in the field of view corresponding to the second light receiving area 1032. If it is determined to be so, the second light receiving area 1032 is controlled to acquire a fine image of the field of view with its maximum pixel density (i.e., a relatively high pixel density), thereby achieving fine identification of the target object. The event camera sensor 104 can also be controlled in a similar way to achieve fine identification, which will not be repeated here.

[0042] In some embodiments, the pixel density of the entire light-receiving area on the camera sensor 103 may be the same, and the pixel density of the entire light-receiving area on the event camera sensor 104 may be the same.

[0043] Reference Figure 3The following is an illustrative description of a camera module according to a second embodiment of the present invention. This camera module can be a camera module for use in a vehicle. The camera module includes a lens assembly 101, a beam splitter 102, a camera sensor 103, an event camera sensor 104, and a first scanner 105.

[0044] The specific structures and configurations of the lens assembly 101, beam splitter 102, camera sensor 103, and event camera sensor 104 have been described in the above embodiments and will not be repeated here. It should be noted that in this embodiment, the camera sensor 103 and event camera sensor 104 can be point image sensors (whose image acquisition area has only one pixel), line image sensors (whose image acquisition area has only one row or column of pixels), or block image sensors (whose image acquisition area has only a small pixel array; a block image sensor is not an area pixel sensor, and it can only acquire a portion of the camera module's field of view). That is, the camera sensor 103 and event camera sensor 104 are both one type of point image sensor, line image sensor, or block image sensor.

[0045] The first scanner 105 is disposed on the second side of the lens assembly 101, that is, the first scanner 105 and the beam splitter 102 are respectively disposed on both sides of the lens assembly 101. The first scanner 105 is used to propagate ambient light 20 to the lens assembly 101 and change the propagation direction of the ambient light 20, that is, it can allow ambient light 20 from different directions to propagate to the lens assembly 101 through the first scanner 105. Thus, scanning can be performed by the first scanner 105 so that the camera sensor 103 and the event camera sensor 104 can acquire images of the entire field of view of the camera module. It should be noted that the first scanner 105 can propagate ambient light 20 to the lens assembly 101 through methods such as reflection, refraction, and transmission.

[0046] For example, the first scanner 105 can be a microelectromechanical system (MEMS) scanning mirror. A MEMS scanning mirror is an optical MEMS device manufactured using optical MEMS technology, integrating a micro-optical mirror with a MEMS driver. The movement of a MEMS scanning mirror includes both translational and torsional mechanical motion. When its optical deflection angle is large (above 10°), its main functions are to achieve light direction deflection, pattern scanning, and image scanning.

[0047] In some embodiments, the first scanner 105 may also be a mirror and an actuator connected together. The mirror is used to reflect ambient light 20 incident on the mirror to the lens group 101, and the actuator is used to drive the mirror to move (e.g., translate and / or rotate) to change the propagation direction of the ambient light 20, thereby achieving scanning.

[0048] By setting the first scanner 105, a scan can be performed at a preset scanning frequency to achieve a global search of the field of view of the camera module and obtain the first frame image. Then, based on the first frame image, the local area that needs to be searched in detail is identified, and the local area is scanned at a higher scanning frequency (that is, the light of the local area is transmitted to the lens group 101 by the first scanner 105) to obtain a detailed image of the local area.

[0049] Conventional camera modules typically use only one camera sensor, often an area array image sensor. In backlit scenes, one or more ghosting images frequently appear, aligned with the light source—a phenomenon known in photography as ghosting. This is generally caused by multiple reflections of incident light after entering the camera module. Because area array image sensors have high reflectivity, this often results in even more reflections, exacerbating the ghosting problem. Furthermore, the exposure of area array image sensors cannot be adaptively adjusted, failing to completely eliminate ghosting. When ghosting is detected, it often obscures the real object, creating blind spots and compromising the safety of autonomous driving.

[0050] According to the camera module of the second embodiment of this application, firstly, since a point image sensor, line image sensor, or block image sensor is used, its reflectivity is lower than that of an area array image sensor, which can significantly reduce ghosting caused by reflection. Secondly, since the image of the entire field of view is not formed all at once, but gradually through scanning, the exposure can be adaptively adjusted during the scanning process. That is, different areas of the image of the entire field of view can be exposed with different exposure rates during the scanning process, thereby effectively solving the ghosting problem.

[0051] Furthermore, distortion correction in conventional camera modules is achieved by optimizing lens structural parameters and using specific algorithms. For example, optimizing lens structural parameters (such as curvature and thickness) is used to correct lens distortion. This requires high-performance lenses and necessitates the use of high-performance, complex lens modules, which indirectly increases the R&D cost of camera modules.

[0052] According to the camera module of the second embodiment of this application, distortion correction can be achieved through scanning, that is, distortion correction can be achieved using mathematical functions, without the need to optimize the lens group itself to achieve distortion correction, which can effectively reduce the distortion correction cost of the lens group 101 and the complexity of the lens group 101 structure.

[0053] Reference Figure 4The following is an illustrative description of a camera module according to a third embodiment of the present invention. This camera module can be a camera module for use in a vehicle. The camera module includes a lens assembly 101, a beam splitter 102, a camera sensor 103, an event camera sensor 104, and a second scanner 106.

[0054] The specific structure and arrangement of the lens group 101, beam splitter 102, camera sensor 103, and event camera sensor 104 have been described in the first embodiment and will not be repeated here. It should be noted that in this embodiment, the camera sensor 103 and event camera sensor 104 can be a point image sensor (whose image acquisition area has only one pixel), a line image sensor (whose image acquisition area has only one row or column of pixels), or a block image sensor (whose image acquisition area has only a small pixel array; a block image sensor is not an area pixel sensor, and it can only acquire a portion of the camera module's field of view). That is, the camera sensor 103 and event camera sensor 104 are both one type of point image sensor, line image sensor, or block image sensor.

[0055] The second scanner 106 is simultaneously connected to the beam splitter 102, the camera sensor 103, and the event camera sensor 104. It is used to move (e.g., translate and / or rotate) these components to change the propagation direction of the first light ray 21. Specifically, it changes the direction of the first light ray 21 entering the beam splitter 102, and consequently, it changes the propagation direction of the ambient light 20 corresponding to the second and third light rays 22 and 23 received by the camera sensor 103 and event camera sensor 104. This allows the camera sensor 103 and event camera sensor 104 to acquire images of different areas within the camera module's field of view. Furthermore, scanning can be performed by the second scanner 106 to enable the camera sensor 103 and event camera sensor 104 to acquire images of the entire field of view of the camera module.

[0056] For example, the second scanner 106 can be a motor or microelectromechanical system (MEMS) actuator. The motor or MEMS actuator can be connected to the beam splitter 102, camera sensor 103, and event camera sensor 104 via connectors, driving them to translate and / or rotate as a whole to perform scanning, enabling the camera sensor 103 and event camera sensor 104 to acquire images of the entire field of view of the camera module. The second scanner 106 can also be any other actuator capable of moving the beam splitter 102, camera sensor 103, and event camera sensor 104 as a whole to achieve scanning.

[0057] By setting the second scanner 106, it can first scan at a preset scanning frequency to achieve a global search of the camera module's field of view and obtain the first frame image. Then, based on the first frame image, it can identify the local area that needs to be searched in detail and scan the local area at a higher scanning frequency to obtain a detailed image of the local area.

[0058] According to the camera module of the third embodiment of this application, firstly, since a point image sensor, line image sensor, or block image sensor is used, its reflectivity is lower than that of an area array image sensor, which can significantly reduce ghosting caused by reflection. Secondly, since the image of the entire field of view is not formed all at once, but gradually through scanning, the exposure can be adaptively adjusted during the scanning process. That is, different areas of the image of the entire field of view can be exposed with different exposure rates during the scanning process, thereby effectively solving the ghosting problem.

[0059] According to the camera module of the third embodiment of this application, distortion correction can be achieved through scanning, that is, distortion correction can be achieved using mathematical functions, without the need to optimize the lens group itself to achieve distortion correction, which can effectively reduce the distortion correction cost of the lens group 101 and the complexity of the lens group 101 structure.

[0060] Reference Figure 5 The following is an illustrative description of a camera module according to a fourth embodiment of the present invention. This camera module can be a camera module for use in a vehicle. The camera module includes a lens assembly 101, a beam splitter 102, a camera sensor 103, an event camera sensor 104, and a third scanner 107.

[0061] The specific structure and arrangement of the lens group 101, beam splitter 102, camera sensor 103, and event camera sensor 104 have been described in the first embodiment and will not be repeated here. It should be noted that in this embodiment, the camera sensor 103 and event camera sensor 104 can be a point image sensor (whose image acquisition area has only one pixel), a line image sensor (whose image acquisition area has only one row or column of pixels), or a block image sensor (whose image acquisition area has only a small pixel array; a block image sensor is not an area pixel sensor, and it can only acquire a portion of the camera module's field of view). That is, the camera sensor 103 and event camera sensor 104 are both one type of point image sensor, line image sensor, or block image sensor.

[0062] The third scanner 107 is connected to the lens assembly 101 and is used to move the lens assembly 101 (e.g., translate and / or rotate) to change the propagation direction of the first light ray 21. That is, it changes the propagation direction of the ambient light 20 corresponding to the second light ray 22 and the third light ray 23 received by the camera sensor 103 and the event camera sensor 104. As a result, the camera sensor 103 and the event camera sensor 104 can acquire images of different areas in the field of view of the camera module. Furthermore, scanning can be performed by the third scanner 107 so that the camera sensor 103 and the event camera sensor 104 can acquire images of the entire field of view of the camera module.

[0063] For example, the third scanner 107 can be a motor or microelectromechanical system (MEMS) actuator. The motor or MEMS actuator can be connected to the lens assembly 101 via a connector, causing it to translate and / or rotate as a whole to perform scanning, enabling the camera sensor 103 and the event camera sensor 104 to acquire images of the entire field of view of the camera module. The third scanner 107 can also be any other actuator capable of moving the lens assembly 101 to achieve scanning.

[0064] By setting the third scanner 107, a scan can be performed at a preset scanning frequency to achieve a global search of the camera module's field of view and obtain the first frame image. Then, based on the first frame image, a local area that needs to be searched in detail can be identified, and the local area can be scanned at a higher scanning frequency to obtain a detailed image of the local area.

[0065] According to the camera module of the fourth embodiment of this application, firstly, since a point image sensor, line image sensor, or block image sensor is used, its reflectivity is lower than that of an area array image sensor, which can significantly reduce ghosting caused by reflection. Secondly, since the image of the entire field of view is not formed all at once, but gradually through scanning, the exposure can be adaptively adjusted during the scanning process. That is, different areas of the image of the entire field of view can be exposed with different exposure rates during the scanning process, thereby effectively solving the ghosting problem.

[0066] According to the camera module of the fourth embodiment of this application, distortion correction can be achieved through scanning, that is, distortion correction can be achieved using mathematical functions, without the need to optimize the lens group itself to achieve distortion correction, which can effectively reduce the distortion correction cost of the lens group 101 and the complexity of the lens group 101 structure.

[0067] This application also provides a vehicle that includes the camera module described in the above embodiments.

[0068] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0069] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0070] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0071] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0072] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0073] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0074] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0075] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing some or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0076] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0077] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A camera module, characterized in that, include: The lens assembly is used to generate and propagate the first ray of light; A beam splitter is disposed on the first side of the lens assembly to receive the first light beam and split the first light beam into a second light beam and a third light beam. A camera sensor is used to receive the second light and generate a first image; In addition, an event camera sensor is used to receive the third light rays and generate a second image.

2. The camera module according to claim 1, characterized in that, Includes a first scanner, which is disposed on the second side of the lens assembly. The first scanner is used to transmit ambient light to the lens assembly and change the propagation direction of the ambient light.

3. The camera module according to claim 1, characterized in that, The device includes a second scanner, which is connected to the beam splitter, the camera sensor, and the event camera sensor. The second scanner is used to move the beam splitter, the camera sensor, and the event camera sensor to change the propagation direction of the first light.

4. The camera module according to claim 1, characterized in that, It includes a third scanner, which is connected to the lens group and is used to move the lens group to change the propagation direction of the first light.

5. The camera module according to any one of claims 2-4, characterized in that, The camera sensor and the event camera sensor are one of the following: point image sensor, line image sensor, and block image sensor.

6. The camera module according to claim 1, characterized in that, The camera sensor and the event camera sensor are area array image sensors.

7. The camera module according to claim 6, characterized in that, Both the camera sensor and the event camera sensor have a first light receiving area and a second light receiving area, and the pixel density of the second light receiving area is greater than the pixel density of the first light receiving area.

8. The camera module according to claim 1, characterized in that, The camera sensor is one of a charge-coupled device (CCD) sensor and a complementary metal-oxide-semiconductor (CMOS) sensor. The event camera sensor is one of the following: a dynamic vision sensor, an asynchronous time-based image sensor, and a dynamic active pixel vision sensor.

9. The camera module according to claim 1, characterized in that, The propagation direction of the second ray is different from that of the third ray.

10. A vehicle, characterized in that, Includes the camera module as described in any one of claims 1-9.

Images