A multifunctional imaging module with integrated multi-camera modules for smart door locks

Abstract

The embodiment of the present invention discloses a multifunctional imaging module for an intelligent door lock that integrates multiple camera modules, including an upper imaging submodule and a lower imaging submodule; the upper imaging submodule is connected to the lower imaging submodule, and only the upper imaging submodule is connected to a host computer or an external device; both submodules include a light source module and a camera module; only the upper imaging submodule includes a data processing and main control module, which can realize the control of the upper imaging submodule and the lower imaging submodule and image and data processing. The embodiment of the present invention has at least one of a cat's eye function and a face or palm swiping function, wherein the cat's eye function can enlarge the imaging field of view under the cat's eye scene by combining or splicing the images of the upper imaging submodule and the lower imaging submodule; in addition, the embodiment of the present invention can reduce the number of client host computer interface designs, save the volume of the imaging module layout, and only requires one main control chip, which greatly saves costs.

Description

Technical Field

[0001] The present invention relates to the field of smart door lock technology, and more particularly to a multifunctional imaging module integrating multiple camera modules for a smart door lock. Background Art

[0002] In the field of smart door locks, imaging modules integrating peephole, face recognition, and palm recognition functions have gradually entered the public eye. The peephole function of traditional smart door locks is typically used for monitoring images in the upper to mid-field of view. As application requirements evolve, peephole functions that simultaneously monitor the upper to mid-field of view, ground-level, and low-lying areas are becoming a mainstream solution for smart door locks. Peephole monitoring in ground-level and low-lying areas is often used to monitor express parcels left at doorsteps to prevent loss.

[0003] In traditional solutions, in order to meet the above requirements, two imaging modules are usually required, namely an upper imaging module and a lower imaging module. The two imaging modules work independently and are independently connected to external devices, resulting in a more complex structure and higher cost for such imaging modules. Summary of the Invention

[0004] The present invention provides a multifunctional imaging module integrating multiple camera modules for a smart door lock, which simultaneously has a cat's eye function and at least one of a face or palm scanning function. The imaging field of view area in the cat's eye scene is enlarged, while the number of customer host computer interface designs is reduced, the volume of the imaging module layout is saved, and a single main control chip is used to save costs.

[0005] An embodiment of the present invention provides a multifunctional imaging module integrated with a multi-camera module for a smart door lock, comprising an upper imaging sub-module and a lower imaging sub-module;

[0006] The upper camera imaging submodule includes an upper camera light source module, an upper camera module, a data processing and main control module, a first upper camera interface module and a second upper camera interface module; the data processing and main control module is a single-chip structure; the upper camera light source module, the upper camera module, the first upper camera interface module and the second upper camera interface module are respectively electrically connected to the data processing and main control module;

[0007] The lower camera imaging submodule includes a lower camera light source module, a lower camera module and a lower camera interface module; the lower camera light source module and the lower camera module are electrically connected to the lower camera interface module respectively;

[0008] The second upper camera interface module is used to electrically connect to a host computer or an external device, and the lower camera interface module is electrically connected to the first upper camera interface module through an FPC adapter cable, so that the lower camera light source module and the lower camera module in the lower camera imaging sub-module are respectively electrically connected to the data processing and main control module in the upper camera imaging sub-module, and the lower camera imaging sub-module is electrically connected to the host computer or the external device through the upper camera imaging sub-module;

[0009] The upper camera module includes a first infrared camera unit and a first visible light camera unit, or includes a first infrared and visible light two-in-one camera unit; the lower camera module includes a second infrared camera unit and a second visible light camera unit, or includes a second infrared and visible light two-in-one camera unit;

[0010] The upper camera module is used to capture images of a first field of view area below the horizontal plane and at least a portion of the field of view area above the horizontal plane, and the lower camera module is used to capture images of a second field of view area below the horizontal plane, and obtain image data respectively;

[0011] The upper light source module is used to project at least floodlight to a first viewing area below the horizontal plane and at least a portion of the viewing area above the horizontal plane, and the lower light source module is used to project at least floodlight to a second viewing area below the horizontal plane;

[0012] The imaging module includes a cat's eye function mode and a face / palm scanning function mode;

[0013] In the cat's eye function mode, the data processing and main control module is used to control the upper light source module and the lower light source module to at least project flood light when the ambient light brightness is lower than a preset threshold, and is also used to control the upper camera module and the lower camera module to capture images. The data processing and main control module is also used to process the image data captured by the upper camera module and the lower camera module to obtain cat's eye images of the upper camera module and / or the lower camera module, respectively, or to combine them to obtain a combined cat's eye image or to splice them to obtain a spliced ​​cat's eye image; wherein the field of view areas of the combined cat's eye image and the spliced ​​cat's eye image are both larger than the field of view areas of the upper camera module and the lower camera module, respectively;

[0014] In the face / palm swiping function mode, the data processing and main control module is used to control the upper light source module to at least project floodlight, and is also used to control the upper camera module to capture images; the data processing and main control module is also used to process the image data captured by the upper camera module to obtain a face swiping image or a palm swiping image, and perform liveness detection and identity authentication based on the face swiping image or the palm swiping image.

[0015] According to the technical solution of the embodiment of the present invention, the imaging module is connected to the host computer or external device only by the upper imaging sub-module through at least one interface and a connecting line, and then connected to the lower imaging sub-module through at least one interface, so as to perform communication transmission and power supply, etc. As a result, the number of interface designs of the client's host computer is reduced, and the volume of the imaging module layout is saved. Moreover, the imaging module can realize the control and image and data processing of the upper imaging sub-module and the lower imaging sub-module only through the data processing and main control module of the upper imaging sub-module, that is, only one main control chip is needed, which greatly saves the chip cost. In addition, the imaging module has multiple functions, including at least one of a cat's eye function and a face or palm swiping function, wherein the cat's eye function can enlarge the imaging field of view area under the cat's eye scene by combining or splicing the images of the upper imaging sub-module and the lower imaging sub-module, effectively matching the middle and upper field of view area, as well as the imaging field of view area of ​​the ground and low areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a system architecture diagram of a multifunctional imaging module integrated with multiple camera modules for a smart door lock provided by an embodiment of the present invention;

[0017] Figure 2 yes Figure 1 A schematic structural diagram of the camera module and light source module in the multifunctional imaging module shown;

[0018] Figure 3 This is a schematic diagram of the physical structure of a camera module and a light source module in a multifunctional imaging module provided by an embodiment of the present invention;

[0019] Figure 4 and Figure 5 1 is a schematic structural diagram of two camera units provided by an embodiment of the present invention;

[0020] Figure 6 This is a schematic diagram of the physical structure of a camera module and a light source module in another multifunctional imaging module provided by an embodiment of the present invention;

[0021] Figure 7 This is a schematic diagram of the physical structure of a camera module and a light source module in another multifunctional imaging module provided by an embodiment of the present invention;

[0022] Figure 8 This is a schematic diagram of the physical structure of a camera module and a light source module in another multifunctional imaging module provided by an embodiment of the present invention;

[0023] Figure 9 This is a schematic diagram of the physical structure of a camera module and a light source module in another multifunctional imaging module provided by an embodiment of the present invention;

[0024] Figure 10 and Figure 111 is a schematic structural diagram of two structured light projection units provided by an embodiment of the present invention;

[0025] Figure 12 This is a schematic diagram of the physical structure of a camera module and a light source module in another multifunctional imaging module provided by an embodiment of the present invention;

[0026] Figure 13 and Figure 14 Schematic diagram of the structure of two upper light source modules provided by an embodiment of the present invention;

[0027] Figure 15 This is a schematic diagram of the connection structure between a data processing and main control module and a camera module provided by an embodiment of the present invention;

[0028] Figure 16 yes Figure 15 A specific connection diagram of the data processing and main control module and the camera module shown;

[0029] Figure 17 yes Figure 15 Another specific connection diagram of the data processing and main control module and the camera module shown;

[0030] Figure 18 This is a schematic diagram of the workflow of a data processing and main control module provided by an embodiment of the present invention;

[0031] Figure 19 This is a schematic diagram of the workflow of another data processing and main control module provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0033] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. It should be noted that the directional words such as "upper", "lower", "left", and "right" described in the embodiments of the present invention are described based on the angles shown in the accompanying drawings and should not be understood as limiting the embodiments of the present invention. In addition, in the context, it is also necessary to understand that when it is mentioned that an element is formed "on" or "under" another element, it can not only be formed directly "on" or "under" another element, but can also be formed indirectly "on" or "under" another element through an intermediate element. The terms "first", "second", etc. are only used for descriptive purposes and do not indicate any order, quantity or importance, but are only used to distinguish different components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0034] The term "including" and its variations used in the present invention are open inclusions, that is, "including but not limited to." The term "based on" means "based at least in part on." The term "one embodiment" means "at least one embodiment."

[0035] It should be noted that the concepts such as "first" and "second" mentioned in the present invention are only used to distinguish the corresponding contents, and are not used to limit the order or mutual dependence.

[0036] It should be noted that the modifications of "one" and "multiple" mentioned in the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0037] Figure 1 This is a system architecture diagram of a multifunctional imaging module integrated with multiple camera modules for smart door locks provided by an embodiment of the present invention. Figure 1 The multifunctional imaging module includes an upper imaging submodule 1 and a lower imaging submodule 2. The upper imaging submodule 1 includes an upper light source module 10, an upper camera module 11, a data processing and main control module 12, a first upper camera interface module 13, and a second upper camera interface module 14. The data processing and main control module 12 is a single-chip structure. The upper light source module 10, the upper camera module 11, the first upper camera interface module 13, and the second upper camera interface module 14 are electrically connected to the data processing and main control module 12. The lower imaging submodule 2 includes a lower light source module 20, a lower camera module 21, and a lower camera interface module 22. The lower light source module 20 and the lower camera module 21 are electrically connected to the lower camera interface module 22.

[0038] The second upper camera interface module 14 is used to electrically connect to a host computer or external device 3. The lower camera interface module 22 is electrically connected to the first upper camera interface module 13 via a flexible printed circuit (FPC) adapter cable. This allows the lower light source module 20 and lower camera module 21 in the lower camera imaging sub-module 2 to be electrically connected to the data processing and main control module 12 in the upper camera imaging sub-module 1, respectively. Furthermore, the lower camera imaging sub-module 2 is electrically connected to the host computer or external device 3 via the upper camera imaging sub-module 1. Specifically, the FPC adapter cable 4 can be a Mobile Industry Processor Interface (MIPI) cable, for example.

[0039] The upper camera module 11 includes a first infrared camera unit and a first visible light camera unit, or a first infrared and visible light two-in-one camera unit; the lower camera module 21 includes a second infrared camera unit and a second visible light camera unit, or a second infrared and visible light two-in-one camera unit. Figure 2 yes Figure 1 The structural diagram of the camera module and light source module in the multifunctional imaging module is shown in FIG. Figure 1 and Figure 2 In a specific embodiment, the optional upper camera module 11 includes a first infrared camera unit 103 and a first visible light camera unit 102 , and the lower camera module 21 includes a second infrared and visible light two-in-one camera unit 105 . Figure 2 The specific location distribution of each component is only given as an example. In specific applications, it can be set according to actual needs and is not limited here.

[0040] Among them, the first infrared camera unit 103 and the second infrared camera unit represent cameras that are sensitive to infrared light and can collect infrared light for infrared imaging. The first visible light camera unit 102 and the second visible light camera unit represent cameras that are sensitive to visible light and can collect visible light for visible light imaging. The first infrared and visible light two-in-one camera unit and the second infrared and visible light two-in-one camera unit 105 represent cameras that are sensitive to both infrared light and visible light and can collect infrared light and visible light for imaging respectively. The specific structures and functions of the upper camera module 11 and the lower camera module 21 are not limited here, and those skilled in the art can select and set them according to actual requirements for the bands.

[0041] The upper camera module 11 is used to capture images of the first field of view area below the horizontal plane and at least part of the field of view area above the horizontal plane, and the lower camera module 21 is used to capture images of the second field of view area below the horizontal plane, and obtain image data respectively; the upper light source module 10 is used to project at least floodlight to the first field of view area below the horizontal plane and at least part of the field of view area above the horizontal plane, and the lower light source module 20 is used to project at least floodlight to the second field of view area below the horizontal plane.

[0042] It can be understood that the multifunctional imaging module in the embodiment of the present invention is applied to a smart door lock, wherein the basic function of the upper camera module 11 and the lower camera module 21 is to capture images of different areas in the vertical direction, thereby realizing a cat's eye imaging function. Specifically, the upper camera module 11 captures images of the first field of view area below the horizontal plane and at least part of the field of view area above the horizontal plane, with the purpose of performing cat's eye monitoring on the middle and upper field of view areas; the lower camera module 21 captures images of the second field of view area below the horizontal plane, with the purpose of performing cat's eye monitoring on the ground and low areas. Correspondingly, the upper light source module 10 and the lower light source module 20 respectively project at least floodlight to the image capture area of ​​the corresponding camera module to fill in the light for the camera and ensure the brightness of the picture.

[0043] The imaging module includes a cat's eye function mode and a face / palm scan function mode; in the cat's eye function mode, the data processing and main control module 12 is used to control the upper light source module 10 and the lower light source module 20 to at least project floodlight when the ambient light brightness is lower than a preset threshold, and is also used to control the upper camera module 11 and the lower camera module 21 to collect images. The data processing and main control module 12 is also used to process the image data collected by the upper camera module 11 and the lower camera module 21, respectively, to obtain the cat's eye image of the upper camera module 11 and / or the lower camera module 21, or to combine them to obtain a combined cat's eye image. eye image or stitching to obtain a stitched cat's eye image; wherein, the field of view areas of the combined cat's eye image and the stitched cat's eye image are respectively larger than the field of view areas of the upper camera module 11 and the lower camera module 21; in the face / palm swiping function mode, the data processing and main control module 12 is used to control the upper light source module 10 to at least project floodlight, and is also used to control the upper camera module 11 to capture images; the data processing and main control module 12 is also used to process the image data captured by the upper camera module 11 to obtain a face swiping image or a palm swiping image, and perform liveness detection and identity authentication based on the face swiping image or the palm swiping image.

[0044] Specifically, the imaging module is multifunctional, including at least one of a cat's eye function and a face or palm scan function. The cat's eye function is implemented by multiple camera modules, namely, the upper imaging sub-module 1 and the lower imaging sub-module 2. For example, cat's eye images captured separately by the upper imaging sub-module 1 and the lower imaging sub-module 2 can be spliced ​​together to form a spliced ​​cat's eye image, which is then displayed on a display screen. It is understood that this spliced ​​cat's eye image is essentially a single, integrated surveillance image. Furthermore, the cat's eye images captured separately by the upper imaging sub-module 1 and the lower imaging sub-module 2 can be combined, that is, displayed separately in different areas on the same display screen, and then displayed together on the same display screen without the need for splicing based on the image content. It is understood that this spliced ​​cat's eye image is essentially two separate surveillance images, but displayed simultaneously on the same display screen. Through this imaging module, the imaging field of view in the cat's eye scene can be enlarged to effectively match the upper and middle field of view areas, as well as the imaging field of view areas of the ground and low areas. Of course, the upper camera imaging sub-module 1 and the lower camera imaging sub-module 2 may not be combined or spliced ​​with each other, but may output images separately to meet the user's need to view the upper camera image area or the lower camera image area separately.

[0045] First, it should be noted that for the cat's eye function, the working process of the upper and lower camera modules also needs to consider the brightness of the ambient light to determine whether fill light is needed from the upper and lower camera light source modules 10 and 20. Specifically, an ambient light sensor can be set to collect ambient light, and the brightness of the ambient light is compared with a preset threshold to determine the current brightness of the smart door lock. When the ambient light brightness exceeds the preset threshold, it means that the current ambient light is bright and a clear image can be obtained, and no fill light is needed; when the ambient light brightness is lower than the preset threshold, it means that the current ambient light is dim and a clear image cannot be obtained, and the camera needs to be filled with light.

[0046] Furthermore, when the environment is bright and the cat's eye function mode is on, the upper camera module 11 uses RGB mode, that is, outputs RGB images, and the lower camera module 21 uses RGB mode, that is, outputs RGB images. When the environment is dark and the cat's eye function mode is on, the upper camera module 11 uses IR mode, that is, outputs black and white images, and the lower camera module 21 uses IR mode, that is, outputs black and white images. Optionally, as needed, the images of the upper imaging sub-module 1 and the lower imaging sub-module 2 can be combined or spliced ​​and then output to the cat's eye, or the images of the upper imaging sub-module 1 or the lower imaging sub-module 2 can be output to the cat's eye separately. Optionally, when combining or splicing images, the resolution of the images output by the upper imaging sub-module 1 and the lower imaging sub-module 2 and the area of ​​the displayed field of view can be flexibly adjusted according to the customer's requirements for the cat's eye display field of view and image resolution. For example, if the resolution of the final combined or spliced ​​image is 1280*720, if the field of view of the upper camera image is to account for a larger proportion, the cat's eye image output of the upper camera module 11 can be made 720*720, and the cat's eye image output of the lower camera module 21 can be made 560*720, thereby combining or splicing the two cat's eye images; if the field of view of the lower camera image is to account for a larger proportion, the cat's eye image output of the upper camera module 11 can be appropriately adjusted to 560*720, and the cat's eye image output of the lower camera module 21 can be made 720*720, thereby combining or splicing the two cat's eye images.

[0047] Specifically, the fill light function is implemented by the upper light source module 10 and the lower light source module 20, wherein the upper light source module 10 and the lower light source module 20 are used to project at least flood light; Figure 2 , optionally, the upper light source module 10 includes at least a first floodlighting unit 101, and the lower light source module 20 includes at least a second floodlighting unit 104. After the cat's eye function mode is turned on, it is turned on when the environment is dark to fill in the light for the camera to take pictures, and it is turned on synchronously after the face or palm swiping function mode is turned on to fill in the light for the camera to take pictures; the fill-in light wavelengths of the two light source modules are not restricted to be the same. The upper camera module 11 and the lower camera module 21 are used to capture and image the reflected light after the floodlight is projected onto the target object and / or the reflected light of the target object under ambient light. Among them, the upper camera module 11 is used for at least one of the cat's eye function mode and the face or palm swiping function mode, wherein the cat's eye function mode is mainly used for target objects in the conventional middle and upper field of view area, such as people. The lower camera module 21 is only used for the cat's eye function mode, and is mainly used for target objects on the ground and low areas, such as pets and express deliveries.

[0048] Specifically, the data processing and main control module 12 is used to control the switching between the cat's eye function mode and the face or palm swiping function mode, as well as image and data processing. The data processing and main control module 12 is a single-chip structure, meaning that it is only provided with a single chip in the upper camera imaging sub-module 1. This chip is fixed to the main board of the upper camera imaging sub-module 1 using surface mounting technology. It is electrically connected to the main board with the upper camera light source module 10 and the upper camera module 11 on the same side or opposite side of the main board, and is electrically connected to the main board through the main board. In other words, the control and data processing functions of each module in the upper camera imaging sub-module 1 and the lower camera imaging sub-module 2 are all implemented by this single chip. There is no need to set up another chip in the lower camera imaging sub-module 2, nor is there a need to set up control and data processing chips for the upper camera imaging sub-module 1 and the lower camera imaging sub-module 2 respectively. Instead, a single chip set up in the upper camera imaging sub-module 1 is responsible for both the upper and lower camera functions, which can save the number of chips, simplify the module structure, and reduce costs.

[0049] Specifically, the first upper camera interface module 13, the second upper camera interface module 14 and the lower camera interface module 22 include all interfaces for communication between modules, and can optionally include MIPI interface, DVP interface, IIC interface, USB interface, UART serial communication interface, MIPI-DSI interface, SPI interface and other interfaces. A DC socket, RJ45 network port and the like can also be provided. The first upper camera interface module 13, the second upper camera interface module 14 and the lower camera interface module 22 are fixed on their respective mainboards by welding, are electrically conductive with the mainboard, and are electrically connected to other modules through the mainboard. The first upper camera interface module 13 is connected to the lower camera interface module 22 through an adapter cable, and the second upper camera interface module 14 is connected to the host computer or external device 3, which can perform communication transmission and power supply, etc.

[0050] In summary, the technical solutions of the embodiments of the present invention have the following technical effects:

[0051] 1. Traditional solutions typically require two imaging modules: an upper imaging module and a lower imaging module. Each requires at least one interface and cable to connect to a host computer or external device for communication, transmission, and power supply. However, the solution provided by the present invention utilizes only one imaging module, with the upper imaging submodule connected to the host computer or external device via at least one interface and cable, and then connected to the lower imaging submodule via at least one interface for communication, transmission, and power supply. This reduces the number of host computer interfaces required and saves space in the imaging module layout.

[0052] 2. In traditional solutions, each of the two imaging modules, the upper and lower imaging modules, requires at least one main control module for control and image and data processing, meaning at least two main control chips are required. However, in the solution provided by the present invention, the imaging module can control and process both the upper and lower imaging sub-modules, using only the upper imaging sub-module's data processing and main control module, requiring only one main control chip, significantly saving costs.

[0053] 3. In the solution provided by the present invention, the imaging module has multiple functions, including at least one of a cat's eye function and a face or palm scanning function. The cat's eye function combines or splices the images of the upper imaging sub-module and the lower imaging sub-module, so as to enlarge the imaging field of view area under the cat's eye scene, effectively matching the upper and middle field of view areas, as well as the imaging field of view areas of the ground and low areas.

[0054] In an optional embodiment of the present invention, the length L of the FPC adapter cable 4 satisfies the following: 20 cm ≤ L ≤ 60 cm. Specifically, the FPC adapter cable 4 can be any size from 20 to 60 cm, and its relative position to other modules is not restricted. Thus, the FPC adapter cable 4 can meet and adapt to the relatively remote location of the upper and lower imaging submodules in the smart door lock, while ensuring that the lower imaging submodule can be controlled, data processed, communicated, and powered by the upper imaging submodule.

[0055] Further optionally, the lower camera module 21 includes an imaging chip, the imaging chip includes a power receiving port, and the power receiving port is connected to a plurality of filter capacitors in parallel; and the capacitance of at least one filter capacitor is ≥5 μf.

[0056] In the traditional solution, the maximum length of the FPC adapter cable 4 is usually 20cm. When the length of the FPC adapter cable 4 is greater than 20cm, it may cause problems such as damage to the integrity of the transmission signal, susceptibility to noise interference, unstable image output, excessive noise, and unstable power supply to the lower camera imaging module. In order to solve the problems that may exist in the above-mentioned traditional solution due to the long adapter cable and ensure the stability of power supply, signal transmission, and image output, multiple filter capacitors can be added near the DOVDD and / or DVDD power supply ports of the imaging chip in the lower camera module 21, wherein at least one filter capacitor has a capacitance of ≥5uf, to reduce the power supply ripple of the camera module, avoid damage to the integrity of the transmission signal, and avoid problems such as susceptibility to noise interference of the signal, ensure stable image output and stable power supply of the lower camera imaging submodule, and reduce the camera module signal and image output noise.

[0057] Alternatively, in some embodiments, the upper imaging sub-module 1 further includes an upper camera mainboard, on which the upper light source module 10, the upper camera module 11, the data processing and main control module 12, and the first upper camera interface module 13 are all disposed, and the upper light source module 10, the upper camera module 11, and the first upper camera interface module 13 are respectively electrically connected to the data processing and main control module 12 via the upper camera mainboard. The lower imaging sub-module 2 further includes a lower camera mainboard, on which the lower light source module 20, the lower camera module 21, and the lower camera interface module 22 are all disposed, and the lower light source module 20 and the lower camera module 21 are respectively electrically connected to the lower camera interface module 22 via the lower camera mainboard.

[0058] In these embodiments, the imaging modules essentially employ a common substrate design. That is, the upper imaging sub-module 1 allows the upper light source module 10 and upper camera module 11 to no longer be connected to the mainboard via connectors as independent components. Instead, they are directly attached to the mainboard, sharing a common substrate with all other modules in the upper imaging sub-module 1, such as the data processing and main control module 12. The lower light source module 20 and lower camera module 21 in the lower imaging sub-module 2 are no longer connected to the mainboard via connectors as independent components. Instead, they are directly attached to the mainboard, sharing a common substrate with all other modules in the lower imaging sub-module 2.

[0059] Optionally, in some embodiments, the lower imaging sub-module 2 is typically installed below the lock body of the smart door lock. The lower imaging sub-module 2 can be perpendicular to the lock body (i.e., the lower panel of the lock body is horizontal and parallel to the ground) or at an angle to the ground (typically, the angle between the lower imaging sub-module 2 and the ground is preferably between 0° and 45°). This installation position offers the advantages of being highly discreet and difficult to detect, and the use of a lens with a narrow field of view allows for precise coverage of the ground and low-lying areas.

[0060] Optionally, in some embodiments, the upper camera imaging sub-module 1 may further include a wireless module, which includes WIFI, Bluetooth, 4G, 5G, etc., for network connection and communication.

[0061] Optionally, in some embodiments, the upper imaging sub-module 1 may further include a power supply module for supplying power to all modules.

[0062] Optionally, in some embodiments, the upper imaging sub-module 1 may further include a storage module, such as a DDR or Flash memory, for storing various types of data.

[0063] Optionally, in some embodiments, the upper imaging sub-module 1 may further include a security module for monitoring system security.

[0064] Optionally, in an embodiment of the present invention, the upper light source module may include a first floodlighting unit; the first floodlighting unit includes a first light source configured to emit infrared light; or the first floodlighting unit includes a first light source and a second light source, the first light source configured to emit infrared light, the second light source configured to emit at least one visible light and / or at least one infrared light, and the infrared light emitted by the second light source is different from the infrared light emitted by the first light source. The lower light source module may include a second floodlighting unit, the second floodlighting unit including a third light source configured to emit infrared light or visible light.

[0065] The specific structures of the camera module and the light source module in the multifunctional imaging module according to the embodiment of the present invention are exemplified and introduced below with reference to the accompanying drawings. Figure 3 is a schematic diagram of the physical structure of a camera module and a light source module in a multifunctional imaging module provided by an embodiment of the present invention. Figure 4 and Figure 5 This is a schematic diagram of the structure of two camera units provided by an embodiment of the present invention. First, refer to Figure 1-Figure 5 The upper imaging submodule of the imaging module includes an upper light source module 10 and an upper camera module 11 , and the lower imaging submodule includes a lower light source module 20 and a lower camera module 21 .

[0066] Specifically, the upper light source module 10 of this embodiment comprises a first floodlighting unit 101, while the lower light source module 20 comprises a second floodlighting unit 104. The first floodlighting unit 101 includes at least a first light source (not shown), which emits infrared light and can be, for example, a light-emitting diode (LED) or a laser. The first light source can also be a light source that integrates multiple desired light spectra, such as simultaneously or independently emitting white light or infrared light. The first floodlighting unit 101 can also include a first light source and a second light source (not shown), which emit infrared light and can be a light source that includes one or more colors, such as white, red, green, blue, or yellow, and emits at least one visible light or infrared light. The second floodlighting unit 104 includes at least a third light source (not shown), which emits infrared light or visible light and can be, for example, a light-emitting diode (LED) or a laser. The third light source can also be a light source that integrates multiple desired light spectra, such as simultaneously or independently emitting white light or infrared light.

[0067] Continue to refer Figure 1-Figure 5Optionally, the upper camera module of this embodiment may include a first visible light camera unit 102 and a first infrared camera unit 103; the first visible light camera unit 102 may be composed of an imaging chip 10111, a lens 10112, a filter 10113, and a bracket or base; the imaging chip 10111 is an RGB imaging chip, and the filter 10113 is an infrared cutoff filter that passes visible light; the first infrared camera unit 103 may be composed of an imaging chip 10111, a lens 10112, a filter 10113, and a bracket or base; the imaging chip 10111 is an IR imaging chip, and the filter 10113 is an infrared bandpass filter, which is used to pass light with a wavelength corresponding to the upper light source module 10; the lens 10112 and the filter 10113 are respectively located on the optical path of the imaging chip 10111, and the bracket or base is used to carry and accommodate the imaging chip 10111, the lens 10112 and the filter 10113.

[0068] Optionally, the lower camera module 21 of this embodiment includes a second infrared and visible light two-in-one camera unit 105, which can also be composed of an imaging chip 10111, a lens 10112, a filter 10113, a bracket or a base; the imaging chip 10111 can be an RGB imaging chip or an RGB-IR imaging chip; the filter 10113 can be a dual-pass filter that can pass visible light and light with a wavelength corresponding to the lower light source module 20; the filter 10113 can also be used with a filter. The optical switcher (IR-CUT) uses two filters 10113 with different passbands, one is an infrared cutoff filter that passes visible light, and the other is a full-pass glass plate or an infrared bandpass filter; when the ambient brightness is higher than a certain threshold, the IR-CUT switches the infrared cutoff filter to the optical path to cut off light outside the visible light band; when the ambient brightness is lower than a certain threshold, the IR-CUT switches the full-pass glass plate or the infrared bandpass filter to the optical path to cut off light outside the corresponding wavelength of the lower camera light source module 20.

[0069] Specifically, if Figure 4 As shown, the lens 10112 can be a threaded lens, the lens barrel of the lens 10112 is fixed in the base by threads, and the base is fixed to the main board by glue; it can also be an integrated lens structure, the lens barrel of the lens 10112 is fixed in the bracket by AA glue, and the bracket is fixed to the main board by glue. In addition, optionally, as Figure 5 As shown, the lens 10112 can also adopt a metasurface lens, a diffraction lens, a refractive lens or any hybrid form of the above lenses, fixed in the lens barrel of the lens 10112. Figure 4As shown, the above filter 10113 can be integrated into the lens barrel of the lens 10112, or can be directly fixed on a bracket or base. In addition, optionally, as Figure 5 As shown, when the lens 10112 adopts a metasurface lens, the filter 10113 can also adopt a filter film, which is plated on the upper surface or lower surface of the metalens microstructure, so that the lens 10112 integrates both imaging function and filter function.

[0070] The advantages of this solution are: 1. The upper camera module can simultaneously realize the cat's eye function and at least one of the face or palm scanning functions; 2. The upper camera module is divided into an independent first visible light camera unit and a first infrared camera unit, each with its own filter, which can avoid the difficulty in controlling the switching of filters and reduce costs; 3. The first visible light camera unit in the upper camera module adopts an infrared cutoff filter that passes visible light, and the first infrared camera unit adopts an infrared bandpass filter, which is used to pass light with a wavelength corresponding to the lower light source module. Therefore, the image output by the upper camera module has no color cast problem; 4. The lower camera module adopts a second infrared and visible light two-in-one camera unit, which can reduce the number of camera units, thereby saving materials and assembly costs of the imaging module.

[0071] Optionally, Figure 6 FIG. 1 is a schematic diagram of the physical structure of a camera module and a light source module in another multifunctional imaging module provided by an embodiment of the present invention. Figure 6 As shown, compared to Figure 3 In the embodiment shown, the lower camera module 21 of this embodiment may also include a second visible light camera unit 106 and a second infrared camera unit 107; the second visible light camera unit 106 may be composed of an imaging chip 10111, a lens 10112, a filter 10113, a bracket or a base; the imaging chip 10111 is an RGB imaging chip, and the filter 10113 is an infrared cutoff filter that passes visible light; the second infrared camera unit 107 may be composed of an imaging chip 10111, a lens 10112, a filter 10113, a bracket or a base, the imaging chip 10111 is an IR imaging chip, and the filter 10113 is an infrared bandpass filter, which is used to pass light with a wavelength corresponding to the lower light source module 20; the lens 10112 and the filter 10113 are respectively located on the optical path of the imaging chip 10111, and the bracket or base is used to carry and accommodate the imaging chip 10111, the lens 10112 and the filter 10113.

[0072] The advantages of this solution are: 1. The lower camera module is divided into an independent second visible light camera unit and a second infrared camera unit, which also have their own filters, thereby avoiding the difficulty in controlling the switching of filters; 2. The second visible light camera unit in the lower camera module adopts an infrared cutoff filter that passes visible light, and the second infrared camera unit adopts an infrared bandpass filter that is used to pass light with a wavelength corresponding to the lower camera light source module. Therefore, the output image of the lower camera module does not have a color cast problem.

[0073] Optionally, Figure 7 This is a schematic diagram of the physical structure of a camera module and a light source module in another multifunctional imaging module provided by an embodiment of the present invention. Figure 7 As shown, compared to Figure 3 In the embodiment shown, the upper camera module 11 of this embodiment may also include a first infrared and visible light two-in-one camera unit 108, which may be composed of an imaging chip 10111, a lens 10112, a filter 10113, a bracket or a base; the imaging chip 10111 may be an RGB imaging chip or an RGB-IR imaging chip; the filter 10113 may be a dual-pass filter that can pass visible light and light of a wavelength corresponding to the upper light source module 10; the filter 10113 may also be used in conjunction with an IR-CUT, using two filters 10113 with different passbands, one being an infrared cutoff filter that passes visible light, and the other being an infrared bandpass filter; when the ambient brightness is higher than a certain threshold, the IR-CUT switches the infrared cutoff filter to the optical path to cut off light outside the visible light band; when the ambient brightness is lower than a certain threshold, the IR-CUT switches the full-pass glass or infrared bandpass filter to the optical path to cut off light other than the wavelength corresponding to the upper light source module 10.

[0074] The advantage of this solution is that both the upper camera module and the lower camera module use a two-in-one infrared and visible light camera unit, which can further reduce the number of camera units, thereby saving the material and assembly costs of the imaging module.

[0075] Optionally, Figure 8 This is a schematic diagram of the physical structure of a camera module and a light source module in another multifunctional imaging module provided by an embodiment of the present invention. Figure 8 As shown, compared to Figure 3 In the illustrated embodiment, the upper camera module of this embodiment may be composed of a first infrared and visible light two-in-one camera unit 108 , and the lower camera module 21 may be composed of a second visible light camera unit 106 and a second infrared camera unit 107 .

[0076] The advantages of this solution are: 1. The upper camera module adopts a first infrared and visible light two-in-one camera unit, which can also reduce the number of camera units, thereby saving the material and assembly costs of the imaging module; 2. The lower camera module is divided into an independent second visible light camera unit and a second infrared camera unit, which also have their own filters, thereby avoiding the difficulty in controlling the switching of filters; 3. The second visible light camera unit in the lower camera module adopts an infrared cut-off filter that passes visible light, and the second infrared camera unit adopts an infrared bandpass filter for passing light of a wavelength corresponding to the lower light source module. Therefore, the output image of the lower camera module has no color cast problem.

[0077] Figure 3-Figure 8 The specific location distribution of each component is only given as an example. In specific applications, it can be set according to actual needs and is not limited here.

[0078] Optionally, in some embodiments, the upper imaging sub-module 1 may also be provided with an ambient light sensor or a proximity light sensor. The ambient light sensor is used to detect the ambient light intensity and sense whether the ambient light intensity is below a certain threshold value. The threshold value is generally set in the range of 5-100 Lux. When the ambient light intensity increases or decreases, other modules can be selectively controlled to respond. The proximity sensor is used to sense whether there is a target object within the working distance range of the imaging module and generate a trigger signal when the target object approaches. Other modules can selectively respond based on the trigger signal.

[0079] Optionally, in embodiments of the present invention, the upper light source module may further include a structured light projection unit configured to project structured light; the structured light projection unit may include a fourth light source configured to emit infrared light. The specific structure of the light source module with the structured light projection unit in embodiments of the present invention will be further illustrated and described below with reference to the accompanying drawings.

[0080] Figure 9 This is a schematic diagram of the physical structure of a camera module and a light source module in another multifunctional imaging module provided by an embodiment of the present invention. Figure 10 and Figure 11 is a schematic diagram of the structure of two structured light projection units provided by an embodiment of the present invention, refer to Figure 9 , compared to Figure 3 In the embodiment shown, the upper light source module 10 of this embodiment can be further added with a structured light projection unit 109 for projecting structured light. In the upper light source module 10, the first floodlighting unit 101 and the structured light projection unit 109 can be divided into two independent units. Specifically, Figure 10 and Figure 11As shown, the structured light projection unit 109 includes at least a fourth light source 10101 and a dimming element. The fourth light source 10101 is used to emit infrared light, and can specifically be a laser light source such as a VCSEL (vertical cavity surface emitting laser), an EEL (edge ​​emitting laser), or an HCSEL (horizontal cavity surface emitting laser). The dimming element is located on the light-emitting side of the fourth light source 10101, and can specifically be a collimator 10102 and a diffraction optical element 10103, or a collimating and diffraction integrated optical element 10104, or a metasurface lens 10105, or a reasonable combination of the above three possible optical elements. After passing through the dimming element, the fourth light source 10101 projects structured light with certain characteristic information toward the imaging target, such as common speckle structured light.

[0081] This embodiment can project structured light in the face or palm swiping function mode. After the reflected light is formed on the face or palm, the speckle pattern of the face or palm can be directly collected, or a three-dimensional structured light image carrying the depth information of the face or palm can be collected, and the depth image can be output using the structured light image. The above-mentioned speckle pattern or depth image is used to perform liveness judgment, enriching the information input for liveness judgment, thereby increasing the recognition accuracy of the face and palm swiping functions, effectively preventing or avoiding deception or attacks on the recognition system, increasing security, and further improving the usage scenarios of the imaging module.

[0082] Figure 12 This is a schematic diagram of the physical structure of a camera module and a light source module in another multifunctional imaging module provided by an embodiment of the present invention. Figure 13 and Figure 14 is a schematic diagram of the structure of two upper light source modules provided by the embodiment of the present invention. Figure 9 In the embodiment shown, the upper light source module of this embodiment can further adopt a two-in-one solution of a structured light projection unit and a floodlighting unit, such as Figure 12 As shown, the first floodlighting unit and the structured light projection unit of the upper light source module can be integrated into the same projector 110 .

[0083] Specifically, if Figure 13 and Figure 14 As shown, the projector 110 includes a first light source 10100, a fourth light source 10101, and a spacer block 10106. The first light source 10100 and the fourth light source 10101 emit light alternately and share a dimming element. The first light source 10100 is used to emit flood light, and the fourth light source 10101 is used to emit structured light. The spacer block 10106 can generally be a printed circuit board, a ceramic substrate, or a metal conductive block, and can be placed below the first light source 10100 or below the fourth light source 10101 (the spacer block 10106 is placed below the fourth light source 10101 in the figure). The dimming element is located on the light-emitting side of the light source, and can specifically be Figure 13The collimating lens 10102 and the diffractive optical element 10103 shown can be Figure 14 The collimating diffraction integrated optical element 10104 shown can be a metasurface lens 10105 or a reasonable combination of the three possible optical elements mentioned above. The first light source 10100 projects uniform flood light toward the imaging target after passing through the dimming element; the fourth light source 10101 projects structured light toward the imaging target after passing through the dimming element.

[0084] Compared to Figure 9 In the illustrated embodiment, the present embodiment further integrates the structured light projection unit and the first floodlighting unit into one projector 110 , which can improve the use scenario of the imaging module while simplifying the complexity of the module and facilitating manufacturing and production.

[0085] Figure 15 This is a schematic diagram of the connection structure between a data processing and main control module and a camera module provided by an embodiment of the present invention, with reference to Figure 15 In an optional embodiment of the present invention, an upper camera imaging sub-module may be provided including a mipi switching switch 5; the mipi switching switch 5 includes a first input terminal 51, a second input terminal 52 and an output terminal 53; the first input terminal 51 is electrically connected to the upper camera module 11, the second input terminal 52 is electrically connected to the lower camera module 21, and the output terminal 53 is electrically connected to the data processing and main control module 12; the mipi switching switch 5 is configured to switch between a first conduction state and a second conduction state; wherein the first conduction state is that the first input terminal 51 is conductively connected to the output terminal 53, and the second conduction state is that the second input terminal 52 is conductively connected to the output terminal 53.

[0086] Among them, the first conduction state and the second conduction state refer to the two mipi channel conduction states of the mipi switching switch 5. By switching the two mipi channel conduction states, the data processing and main control module 12 can be connected with different camera units to receive and process the image data collected by the connected camera units.

[0087] Specifically, in this embodiment, in order to use only one data processing and main control module 12 to achieve control and image and data processing for multiple camera modules, the data processing and main control module 12 needs to have multiple MIPI channels. In the upper camera module 11 and the lower camera module 21, each camera unit corresponds to at least one MIPI channel. Since the more MIPI channels required by the data processing and main control module 12, the higher the chip cost, in this embodiment, a MIPI switch 5 is used to allow the lower camera module 21 and the upper camera module 11 to share a MIPI channel. For example, a signal can be initiated at the end of each frame to start the MIPI switch to achieve MIPI channel switching. The data processing and main control module 12 can time-share the upper camera module 11 and the lower camera module 21 based on the switching of the MIPI channels, and correspondingly switch the control logic, image processing method, data processing method, etc. If the camera of the lower camera module 21 and the infrared camera of the upper camera module 11 share a MIPI channel, the frame rate of the visible light camera in the upper camera module 11 can be guaranteed, and the frame rate of the cat's eye function mode implemented by the visible light camera can be guaranteed; if the camera of the lower camera module 21 and the visible light camera of the upper camera module 11 share a MIPI channel, the face or palm scanning function mode implemented by the infrared camera in the upper camera module 11 can be made faster. Therefore, using the MIPI switch 5 can reduce the number of MIPI channels required for data processing and main control module 12, select lower-cost chips, and save costs.

[0088] Figure 16 yes Figure 15 A specific connection diagram of the data processing and main control module and the camera module is shown in FIG. Figure 16 In a specific embodiment, the optional upper camera module 11 includes a first infrared camera unit 103 and a first visible light camera unit 102; the lower camera module 21 includes a second infrared and visible light two-in-one camera unit 105; the first infrared camera unit 103 or the first visible light camera unit 102 is electrically connected to the first input end 51; the second infrared and visible light two-in-one camera unit 105 is electrically connected to the second input end 52; the first conduction state is that the first input end 51 is conductively connected to the output end 53, and the second conduction state is that the second input end 52 is conductively connected to the output end 53.

[0089] This embodiment means that based on Figure 3The camera modules in the embodiment are set up in a manner that the first visible light camera unit 102 or the first infrared camera unit 103 of the upper camera module 11 can share a MIPI channel with the second infrared and visible light two-in-one camera unit 105 of the lower camera module 21, that is, the data processing and main control module 12 is connected through the MIPI switching switch 5, and the visible light image in the upper camera module 11 and the image of the lower camera module 21 are time-shared multiplexed, or the infrared image in the upper camera module 11 and the image of the lower camera module 21 are time-shared multiplexed.

[0090] Figure 17 yes Figure 15 Another specific connection diagram of the data processing and main control module and the camera module is shown in FIG. Figure 17 In another specific embodiment, the optional upper camera module 11 includes a first infrared camera unit 103 and a first visible light camera unit 102, and the lower camera module 21 includes a second infrared camera unit 107 and a second visible light camera unit 106; the first input terminal 51 includes a first sub-input terminal 511 and a second sub-input terminal 512; the first infrared camera unit 103 is electrically connected to the first sub-input terminal 511, and the first visible light camera unit 102 is electrically connected to the second sub-input terminal 512; the second input terminal 52 includes a third sub-input terminal 523 and a fourth sub-input terminal 524; the second infrared camera unit 107 is electrically connected to the third sub-input terminal 523, and the second visible light camera unit 106 is electrically connected to the fourth sub-input terminal 524.

[0091] The first conduction state includes a first sub-conduction state and a second sub-conduction state, the first sub-conduction state is that the first sub-input terminal 511 is conductive with the output terminal 53, and the second sub-conduction state is that the second sub-input terminal 512 is conductive with the output terminal 53; the second conduction state includes a third sub-conduction state and a fourth sub-conduction state, the third sub-conduction state is that the third sub-input terminal 523 is conductive with the output terminal 53, and the fourth sub-conduction state is that the fourth sub-input terminal 524 is conductive with the output terminal 53; the mipi switching switch 5 is configured to switch between the first sub-conduction state, the second sub-conduction state, the third sub-conduction state and the fourth sub-conduction state.

[0092] This embodiment means that based on Figure 6In the embodiment, in the camera module setting method in which the upper camera module 11 and the lower camera module 21 are respectively provided with an infrared camera unit and a visible light camera unit, the first visible light camera unit 102 and the first infrared camera unit 103 of the upper camera module 11 can share a MIPI channel with the second visible light camera unit 106 and the second visible light camera unit 106 of the lower camera module 21, that is, the data processing and main control module 12 is connected through the MIPI switching switch 5, and the visible light image and infrared image in the upper camera module 11 and the visible light image and infrared image of the lower camera module 21 are time-shared multiplexed.

[0093] It should be noted here that since the upper camera module 11 and the lower camera module 21 can be provided with more than one camera unit, in addition to connecting the camera unit and the data processing and main control module 12 through the mipi switching switch 5 in the above embodiment, some camera units can also be provided to be directly connected to the data processing and main control module 12, and independently occupy a mipi channel of the data processing and main control module 12. Therefore, when the camera unit that independently occupies the mipi channel is working, the collected image can be directly provided to the data processing and main control module 12 for processing and output.

[0094] Figure 18 This is a workflow diagram of a data processing and main control module provided by an embodiment of the present invention, with reference to Figure 18 ,refer to Figure 15 and Figure 18 In an embodiment of the present invention, the working process of the multifunctional imaging module may include a first time-division multiplexing stage T1, the first time-division multiplexing stage T1 includes multiple first processing cycles t1, the first processing cycle t1 includes multiple image output frames, and the number of image output frames in different first processing cycles t1 is the same; the first processing cycle t1 includes M1 first image output frames ta and N1 second image output frames tb, and the solid line waveform in the figure represents the first image output frame ta, and the solid line waveform represents the second image output frame tb; wherein M1 and N1 are both positive integers; the mipi switching switch 5 is configured to be in a first conductive state in the first image output frame ta and in a second conductive state in the second image output frame tb.

[0095] It can be understood that since the MIPI switch 5 has different conduction states, that is, in the two conduction states of the MIPI switch 5, namely the first conduction state and the second conduction state, the data processing and main control module 12 can process and obtain different images respectively, and output images captured by different camera modules in different image output frames, namely the first image output frame ta and the second image output frame tb. Specifically, in the first conduction state, the data processing and main control module 12 is connected to the upper camera module 11, and in the second conduction state, the data processing and main control module 12 is connected to the lower camera module 21. Therefore, it can be seen that the M1 first image frames and N1 second image frames included in the first processing cycle t1 indicate that the multiple image output frames in the processing cycle can be allocated to output a certain number of images captured by the upper camera module 11 and a certain number of images captured by the lower camera module 21. The specific numerical values ​​of M1 and N1 can control the output ratio of the images captured by the upper camera module 11 and the lower camera module 21 during the processing cycle. For example, when M1 = N1, the data processing and main control module 12 can, through the conduction control of the MIPI switch 5, output half the image output frames of the images captured by the upper camera module 11, and half the image output frames of the images captured by the lower camera module 21. Furthermore, the MIPI switch 5 can optionally be alternately switched between a first conduction state and a second conduction state, thereby causing the data processing and main control module 12 to alternately output the images captured by the upper camera module 11 and the lower camera module 21. Furthermore, by maintaining the cyclic output of the images in the first processing cycle t1 during the first time-division multiplexing stage T1, the images of the upper camera module 11 and the lower camera module 21 can be stably output at a fixed ratio, forming a continuous video output that meets the requirements of cat-eye monitoring.

[0096] Figure 19 This is another workflow diagram of a data processing and main control module provided by an embodiment of the present invention, with reference to Figure 19 In another embodiment of the present invention, the working process of the multifunctional imaging module may further include a second time-division multiplexing stage T2, the second time-division multiplexing stage T2 includes multiple second processing cycles t2, the second processing cycle t2 includes multiple image output frames, and the number of image output frames in different second processing cycles t2 is the same; the second processing cycle t2 includes M2 first image output frames ta and N2 second image output frames tb. In the figure, the solid line waveform represents the first image output frame ta, and the solid line waveform represents the second image output frame tb; wherein M2 and N2 are both positive integers; M2 / N2≠M1 / N1.

[0097] In this embodiment, the working logic of the second time-sharing multiplexing stage T2 is the same as that of the first time-sharing multiplexing stage T1, and both control the ratio of the two image output frames, namely the first image output frame ta and the second image output frame tb, in each processing cycle by switching the two conduction states of the mipi switching switch 5. The only difference is that the ratio of the two image output frames in the second processing cycle t2 is different, that is, M2 / N2≠M1 / N1, thereby realizing the time-sharing multiplexing stage of two output ratios. It can be seen from this that in the embodiment of the present invention, the output ratio of the images captured by the upper camera module 11 and the lower camera module 21 can be adjusted by controlling the ratio of the conduction states of the mipi switching switch 5, so as to adapt to different application scenarios of the smart door lock, flexibly increase the image output ratio of the upper camera module 11 or the lower camera module 21, and meet the needs of cat-eye monitoring.

[0098] It should be noted that if Figure 18 and Figure 19 In the embodiment, the time of the first time-division multiplexing stage T1 and the second time-division multiplexing stage T2 can be the same or different, and the number of image output frames in the first processing cycle t1 and the second processing cycle t2 can be the same or different. Those skilled in the art can make selections and adjustments to meet the requirements of specific scenarios.

[0099] The advantages of this embodiment are: First, by adopting Figure 18 and / or Figure 19 The solution of the embodiment shown can adapt to different application scenarios of smart door locks, flexibly increase the image output ratio of the upper camera module 11 or the lower camera module 21, and meet the needs of cat-eye monitoring. Figure 18 and / or Figure 19 The solution of the embodiment shown can increase the number of image output frames of the face or palm swiping function mode according to the scene requirements, increase the recognition speed of face or palm swiping, and thus increase the response speed of the face or palm swiping function mode. Figure 18 and / or Figure 19 The solution of the illustrated embodiment can flexibly perform different image processing on different frame images according to the needs of the scene, simplify the image processing process of some frame images, and thus further improve the response speed of the cat's eye or face scanning and palm scanning function modes.

[0100] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, combinations, and substitutions are possible for those skilled in the art without departing from the scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A multifunctional imaging module integrated with multiple camera modules for smart door locks, characterized in that: It includes an upper imaging sub-module and a lower imaging sub-module; The upper camera imaging submodule includes an upper camera light source module, an upper camera module, a data processing and main control module, a first upper camera interface module and a second upper camera interface module; the data processing and main control module is a single-chip structure; the upper camera light source module, the upper camera module, the first upper camera interface module and the second upper camera interface module are respectively electrically connected to the data processing and main control module; The lower camera imaging submodule includes a lower camera light source module, a lower camera module and a lower camera interface module; the lower camera light source module and the lower camera module are electrically connected to the lower camera interface module respectively; The second upper camera interface module is used to electrically connect to a host computer or an external device, and the lower camera interface module is electrically connected to the first upper camera interface module through an FPC adapter cable, so that the lower camera light source module and the lower camera module in the lower camera imaging sub-module are respectively electrically connected to the data processing and main control module in the upper camera imaging sub-module, and the lower camera imaging sub-module is electrically connected to the host computer or the external device through the upper camera imaging sub-module; The upper camera module includes a first infrared camera unit and a first visible light camera unit, or includes a first infrared and visible light two-in-one camera unit; the lower camera module includes a second infrared camera unit and a second visible light camera unit, or includes a second infrared and visible light two-in-one camera unit; The upper camera module is used to capture images of a first field of view area below the horizontal plane and at least a portion of the field of view area above the horizontal plane, and the lower camera module is used to capture images of a second field of view area below the horizontal plane, and obtain image data respectively; The upper light source module is used to project at least floodlight to a first viewing area below the horizontal plane and at least a portion of the viewing area above the horizontal plane, and the lower light source module is used to project at least floodlight to a second viewing area below the horizontal plane; The imaging module includes a cat's eye function mode and a face / palm scanning function mode; In the cat's eye function mode, the data processing and main control module is used to control the upper light source module and the lower light source module to at least project flood light when the ambient light brightness is lower than a preset threshold, and is also used to control the upper camera module and the lower camera module to capture images, and the data processing and main control module is further used to process the image data captured by the upper camera module and the lower camera module to obtain the cat's eye images of the upper camera module and / or the lower camera module, respectively, or to combine them to obtain a combined cat's eye image or to splice them to obtain a spliced ​​cat's eye image; wherein the field of view areas of the combined cat's eye image and the spliced ​​cat's eye image are respectively larger than the field of view areas of the upper camera module and the lower camera module; In the face / palm swiping function mode, the data processing and main control module is used to control the upper light source module to at least project floodlight, and is also used to control the upper camera module to capture images; the data processing and main control module is also used to process the image data captured by the upper camera module to obtain a face swiping image or a palm swiping image, and perform liveness detection and identity authentication based on the face swiping image or the palm swiping image.

2. The multifunctional imaging module according to claim 1, characterized in that: The upper camera imaging sub-module also includes a mipi switch; The mipi switch includes a first input end, a second input end and an output end; the first input end is electrically connected to the upper camera module, the second input end is electrically connected to the lower camera module, and the output end is electrically connected to the data processing and main control module; The mipi switch is configured to switch between a first conduction state and a second conduction state; wherein the first conduction state is that the first input end is connected to the output end, and the second conduction state is that the second input end is connected to the output end.

3. The multifunctional imaging module according to claim 2, characterized in that: The upper camera module includes a first infrared camera unit and a first visible light camera unit; the lower camera module includes a second infrared and visible light two-in-one camera unit; The first infrared camera unit or the first visible light camera unit is electrically connected to the first input end; the second infrared and visible light two-in-one camera unit is electrically connected to the second input end; The first conductive state is that the first input terminal is conductively connected to the output terminal, and the second conductive state is that the second input terminal is conductively connected to the output terminal.

4. The multifunctional imaging module according to claim 2, characterized in that: The upper camera module includes a first infrared camera unit and a first visible light camera unit, and the lower camera module includes a second infrared camera unit and a second visible light camera unit; The first input terminal includes a first sub-input terminal and a second sub-input terminal; the first infrared camera unit is electrically connected to the first sub-input terminal, and the first visible light camera unit is electrically connected to the second sub-input terminal; the second input terminal includes a third sub-input terminal and a fourth sub-input terminal; the second infrared camera unit is electrically connected to the third sub-input terminal, and the second visible light camera unit is electrically connected to the fourth sub-input terminal; The first conduction state includes a first sub-conduction state and a second sub-conduction state, the first sub-conduction state is that the first sub-input terminal is conductively connected to the output terminal, and the second sub-conduction state is that the second sub-input terminal is conductively connected to the output terminal; The second conduction state includes a third sub-conduction state and a fourth sub-conduction state, the third sub-conduction state is that the third sub-input terminal is conductively connected to the output terminal, and the fourth sub-conduction state is that the fourth sub-input terminal is conductively connected to the output terminal; The mipi switch is configured to switch between the first sub-conducting state, the second sub-conducting state, the third sub-conducting state and the fourth sub-conducting state.

5. The multifunctional imaging module according to claim 2, characterized in that: The working process of the multifunctional imaging module includes a first time-division multiplexing stage, the first time-division multiplexing stage includes a plurality of first processing cycles, the first processing cycles include a plurality of image output frames, and the number of the image output frames in different first processing cycles is the same; The first processing cycle includes M1 first image output frames and N1 second image output frames; wherein M1 and N1 are both positive integers; The mipi switch is configured to be in the first conductive state during the first image output frame and in the second conductive state during the second image output frame.

6. The multifunctional imaging module according to claim 5, characterized in that: The working process of the multifunctional imaging module further includes a second time-division multiplexing stage, the second time-division multiplexing stage includes a plurality of second processing cycles, the second processing cycles include a plurality of image output frames, and the number of the image output frames in different second processing cycles is the same; The second processing cycle includes M2 first image output frames and N2 second image output frames; wherein M2 and N2 are both positive integers; M2 / N2≠M1 / N1.

7. The multifunctional imaging module according to claim 1, characterized in that: The length L of the FPC adapter cable satisfies: 20 cm ≤ L ≤ 60 cm.

8. The multifunctional imaging module according to claim 1, characterized in that: The lower camera module includes an imaging chip, the imaging chip includes a power receiving port, and the power receiving port is connected in parallel with a plurality of filter capacitors; and the capacitance of at least one of the filter capacitors is ≥5 μf.

9. The multifunctional imaging module according to claim 1, characterized in that: The upper camera imaging sub-module also includes an upper camera mainboard, the upper camera light source module, the upper camera module, the data processing and main control module and the first upper camera interface module are all arranged on the upper camera mainboard, and the upper camera light source module, the upper camera module and the first upper camera interface module are respectively electrically connected to the data processing and main control module through the upper camera mainboard; The lower camera imaging sub-module also includes a lower camera mainboard, and the lower camera light source module, the lower camera module and the lower camera interface module are all arranged on the lower camera mainboard, and the lower camera light source module and the lower camera module are respectively electrically connected to the lower camera interface module through the lower camera mainboard.

10. The multifunctional imaging module according to claim 1, characterized in that: The upper light source module includes a first floodlighting unit; the first floodlighting unit includes a first light source, and the first light source is used to emit infrared light; or the first floodlighting unit includes a first light source and a second light source, the first light source is used to emit infrared light, and the second light source is used to emit at least one visible light and / or at least one infrared light, and the infrared light emitted by the second light source is different from the infrared light emitted by the first light source; The downlight module includes a second floodlight unit, and the second floodlight unit includes a third light source, and the third light source is used to emit infrared light or visible light. And / or, the upper light source module further includes a structured light projection unit, and the structured light projection unit is used to project structured light; The structured light projection unit includes a fourth light source, and the fourth light source is used to emit infrared band light.

Images