Multifunctional imaging module integrated with multi-camera module and used for intelligent door lock

Abstract

The embodiment of the invention discloses a multifunctional imaging module integrated with a multi-camera module and used for an intelligent door lock. The multifunctional imaging module comprises an upper camera imaging sub-module and a lower camera imaging sub-module, the upper shooting imaging sub-module is connected with the lower shooting imaging sub-module, and only the upper shooting imaging sub-module is connected with an upper computer or external equipment; each of the two sub-modules comprises a light source module and a camera module; only the upper shooting imaging sub-module comprises a data processing and main control module which can realize control and image and data processing of the upper shooting imaging sub-module and the lower shooting imaging sub-module. The embodiment of the invention simultaneously has at least one of a cat eye function and a face brushing or palm brushing function, and the cat eye function can enlarge an imaging view field area in a cat eye scene by combining or splicing images of the upper shooting imaging sub-module and the lower shooting imaging sub-module; in addition, according to the embodiment of the invention, the number design of upper computer interfaces of clients can be reduced, the layout volume of the imaging module is saved, only one main control chip is needed, and the cost is greatly saved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of intelligent door locks, and particularly to a multifunctional imaging module integrating multiple camera modules for an intelligent door lock. Background Art

[0002] In the field of intelligent door locks, imaging modules integrating functions such as cat's-eye, face brushing, and palm brushing have gradually come into the public view. The cat's-eye function of traditional intelligent door locks is usually used for functions such as monitoring pictures in the conventional upper middle field of view area. With the development of application requirements, the cat's-eye function for simultaneously monitoring the upper middle field of view area, the ground, and low areas is gradually becoming the mainstream solution for intelligent door locks. For cat's-eye monitoring of the ground and low areas, it is commonly used to monitor express parcels placed in front of the door to prevent loss, etc.

[0003] In the traditional solution, in order to meet the above requirements, usually two imaging modules are required, namely an upper camera imaging module and a lower camera imaging module. The two imaging modules work independently and are respectively connected to external devices independently, resulting in a relatively complex structure and high cost of such imaging modules. Summary of the Invention

[0004] The present invention provides a multifunctional imaging module integrating multiple camera modules for an intelligent door lock, which simultaneously has at least one of the cat's-eye function and the face brushing or palm brushing function. In the cat's-eye scenario, the imaging field of view area becomes larger, while the number of customer host computer interface designs is reduced, the volume of the imaging module layout is saved, and a single main control chip is used to save costs.

[0005] The embodiments of the present invention provide a multifunctional imaging module integrating multiple camera modules for an intelligent door lock, including an upper camera imaging sub-module and a lower camera imaging sub-module;

[0006] The upper camera imaging sub-module includes an upper camera light source module, an upper camera module, a data processing and main control module, a first upper camera interface module, and a second upper camera interface module; the data processing and main control module is of a single-chip structure; the upper camera light source module, the upper camera module, the first upper camera interface module, and the second upper camera interface module are respectively electrically connected to the data processing and main control module;

[0007] The lower camera imaging sub-module includes a lower camera light source module, a lower camera module, and a lower camera interface module; the lower camera light source module and the lower camera module are respectively electrically connected to the lower camera interface module;

[0008] The second upper camera interface module is used for electrically connecting to a host computer or an external device. The lower camera interface module is electrically connected to the first upper camera interface module through an FPC transfer cable, so that the lower camera light source module and the lower camera module in the lower camera imaging sub-module are respectively electrically connected to the data processing and main control module in the upper camera imaging sub-module, and the lower camera imaging sub-module is electrically connected to the host computer or the external device through the upper camera imaging sub-module;

[0009] The upper camera module includes a first infrared camera unit and a first visible light camera unit, or includes a first infrared and visible light two-in-one camera unit; the lower camera module includes a second infrared camera unit and a second visible light camera unit, or includes a second infrared and visible light two-in-one camera unit;

[0010] The upper camera module is used for collecting images of a first field of view area below the horizontal plane and at least part of the field of view area above the horizontal plane, and the lower camera module is used for collecting images of a second field of view area below the horizontal plane, and respectively obtaining image data;

[0011] The upper camera light source module is used for projecting at least floodlight on a first field of view area below the horizontal plane and at least part of the field of view area above the horizontal plane, and the lower camera light source module is used for projecting at least floodlight on a second field of view area below the horizontal plane;

[0012] The imaging module includes a cat's eye function mode and a face / hand brushing function mode;

[0013] In the cat's eye function mode, the data processing and main control module is used for respectively controlling the upper camera light source module and the lower camera light source module to project at least floodlight when the ambient light brightness is lower than a preset threshold, and is also used for respectively controlling the upper camera module and the lower camera module to collect images. The data processing and main control module is also used for processing the image data collected by the upper camera module and the lower camera module, respectively obtaining cat's eye images of the upper camera module and / or the lower camera module, or combining them to obtain a combined cat's eye image or stitching them to obtain a stitched cat's eye image; wherein, the field of view areas of the combined cat's eye image and the stitched cat's eye image are respectively larger than the field of view areas of the upper camera module and the lower camera module;

[0014] In the face / hand brushing function mode, the data processing and main control module is used for controlling the upper camera light source module to project at least floodlight, and is also used for controlling the upper camera module to collect images; the data processing and main control module is also used for processing the image data collected by the upper camera module, obtaining a face brushing image or a hand brushing image, and performing live body detection and identity authentication based on the face brushing image or the hand brushing image.

[0015] In the technical solution of the embodiment of the present invention, the imaging module can perform communication transmission, power supply, etc. only by connecting the upper camera imaging sub-module to the host computer or external device through at least one interface and connection line, and then connecting to the lower camera imaging sub-module through at least one interface. Thus, the design of the number of interfaces of the customer's host computer is reduced, and the volume of the imaging module layout is saved. Moreover, the imaging module can realize the control of the upper camera imaging sub-module and the lower camera imaging sub-module, as well as image and data processing only through the data processing and main control module of the upper camera imaging sub-module, that is, only one main control chip is needed, which greatly saves the chip cost. In addition, the imaging module has multiple functions, including at least one of the cat's eye function and the face or palm brushing function. Among them, the cat's eye function can enlarge the imaging field of view area in the cat's eye scenario by combining or splicing the images of the upper camera imaging sub-module and the lower camera imaging sub-module, effectively matching the upper field of view area, as well as the imaging field of view areas of the ground and low areas. Description of the Drawings

[0016] Figure 1 is the system architecture diagram of a multi-functional imaging module for an intelligent door lock provided by an embodiment of the present invention;

[0017] Figure 2 is Figure 1 the structural schematic diagram of the camera module and the light source module in the multi-functional imaging module shown;

[0018] Figure 3 is the physical structure schematic diagram of a camera module and a light source module in a multi-functional imaging module provided by an embodiment of the present invention;

[0019] Figure 4 and Figure 5 are the structural schematic diagrams of two camera units provided by an embodiment of the present invention;

[0020] Figure 6 is another physical structure schematic diagram of a camera module and a light source module in a multi-functional imaging module provided by an embodiment of the present invention;

[0021] Figure 7 is yet another physical structure schematic diagram of a camera module and a light source module in a multi-functional imaging module provided by an embodiment of the present invention;

[0022] Figure 8 is yet another physical structure schematic diagram of a camera module and a light source module in a multi-functional imaging module provided by an embodiment of the present invention;

[0023] Figure 9 is yet another physical structure schematic diagram of a camera module and a light source module in a multi-functional imaging module provided by an embodiment of the present invention;

[0024] Figure 10 and Figure 11It is a schematic structural diagram of two structured light projection units provided by an embodiment of the present invention;

[0025] Figure 12 It is a schematic physical structure diagram of a camera module and a light source module in another multifunctional imaging module provided by an embodiment of the present invention;

[0026] Figure 13 and Figure 14 It is a schematic structural diagram of two upward shooting light source modules provided by an embodiment of the present invention;

[0027] Figure 15 It is a schematic connection structure diagram of a data processing and main control module and a camera module provided by an embodiment of the present invention;

[0028] Figure 16 is Figure 15 A specific connection schematic diagram of the data processing and main control module and the camera module shown;

[0029] Figure 17 is Figure 15 Another specific connection schematic diagram of the data processing and main control module and the camera module shown;

[0030] Figure 18 It is a schematic working process diagram of a data processing and main control module provided by an embodiment of the present invention;

[0031] Figure 19 It is another schematic working process diagram of a data processing and main control module provided by an embodiment of the present invention. Specific embodiments

[0032] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. Additionally, it should be noted that for the convenience of description, only parts related to the present invention are shown in the drawings, rather than all the structures.

[0033] The terms used in the embodiments of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. It should be noted that the orientation terms such as "upper", "lower", "left", and "right" described in the embodiments of the present invention are described from the angles shown in the drawings and should not be construed as limiting the embodiments of the present invention. In addition, in the context, it should also be understood that when it is mentioned that an element is formed "on" or "under" another element, it can not only be directly formed "on" or "under" another element, but also be indirectly formed "on" or "under" another element through an intermediate element. The terms "first", "second", etc. are only used for descriptive purposes and do not indicate any order, quantity, or importance, but are only used to distinguish different components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0034] The term "comprising" and its variants used in the present invention are open-ended, that is, "including but not limited to". The term "based on" means "at least partially based on". The term "an embodiment" means "at least one embodiment".

[0035] It should be noted that the concepts such as "first" and "second" mentioned in the present invention are only used to distinguish the corresponding contents and are not used to limit the order or interdependent relationship.

[0036] It should be noted that the modification of "one" and "multiple" mentioned in the present invention is illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0037] Figure 1 is a system architecture diagram of a multifunctional imaging module integrating multiple camera modules for an intelligent door lock provided by an embodiment of the present invention. Refer to Figure 1 , the multifunctional imaging module includes an upper camera imaging sub-module 1 and a lower camera imaging sub-module 2; the upper camera imaging sub-module 1 includes an upper camera light source module 10, an upper camera module 11, a data processing and main control module 12, a first upper camera interface module 13, and a second upper camera interface module 14; the data processing and main control module 12 has a single-chip structure; the upper camera light source module 10, the upper camera module 11, the first upper camera interface module 13, and the second upper camera interface module 14 are respectively electrically connected to the data processing and main control module 12. The lower camera imaging sub-module 2 includes a lower camera light source module 20, a lower camera module 21, and a lower camera interface module 22; the lower camera light source module 20 and the lower camera module 21 are respectively electrically connected to the lower camera interface module 22.

[0038] The second upper camera interface module 14 is used for electrically connecting to a host computer or an external device 3. The lower camera interface module 22 is electrically connected to the first upper camera interface module 13 through a flexible printed circuit (FPC) patch cord, so that the lower camera light source module 20 and the lower camera module 21 in the lower camera imaging sub-module 2 are respectively electrically connected to the data processing and main control module 12 in the upper camera imaging sub-module 1, and the lower camera imaging sub-module 2 is electrically connected to the host computer or the external device 3 through the upper camera imaging sub-module 1. Among them, the FPC patch cord 4 can specifically be a Mobile Industry Processor Interface (MIPI) cable, etc.

[0039] The upper camera module 11 includes a first infrared camera unit and a first visible light camera unit, or includes a first infrared and visible light integrated camera unit; the lower camera module 21 includes a second infrared camera unit and a second visible light camera unit, or includes a second infrared and visible light integrated camera unit. Figure 2 Yes Figure 1 is a schematic structural diagram of the camera module and the light source module in the multifunctional imaging module shown, refer to Figure 1 and Figure 2 In a specific embodiment, optionally, the upper camera module 11 includes a first infrared camera unit 103 and a first visible light camera unit 102, and the lower camera module 21 includes a second infrared and visible light integrated camera unit 105. Figure 2 Only the specific position distribution of each device is exemplified, and it can be set according to actual needs during specific applications, and no limitation is made here.

[0040] Among them, the first infrared camera unit 103 and the second infrared camera unit indicate cameras that are sensitive to light in the infrared band and can collect light in the infrared band for infrared imaging. The first visible light camera unit 102 and the second visible light camera unit indicate cameras that are sensitive to light in the visible light band and can collect light in the visible light band for visible light imaging. The first infrared and visible light integrated camera unit and the second infrared and visible light integrated camera unit 105 indicate cameras that are simultaneously sensitive to light in the infrared band and the visible light band and can respectively collect light in the infrared band and the visible light band for imaging. The specific structures and functions in the upper camera module 11 and the lower camera module 21 are not limited here, and those skilled in the art can make selections and settings according to actual requirements for the wavelength band.

[0041] The upper camera module 11 is used to collect images of the first field of view area below the horizontal plane and at least part of the field of view area above the horizontal plane, and the lower camera module 21 is used to collect images of the second field of view area below the horizontal plane, and image data is obtained respectively; the upper light source module 10 is used to project at least floodlight onto the first field of view area below the horizontal plane and at least part of the field of view area above the horizontal plane, and the lower light source module 20 is used to project at least floodlight onto the second field of view area below the horizontal plane.

[0042] It can be understood that the multifunctional imaging module in the embodiments of the present invention is applied to an intelligent door lock. The basic functions of the upper camera module 11 and the lower camera module 21 therein are to collect images of different regions in the vertical direction, so as to realize the cat's eye imaging function. Specifically, the upper camera module 11 collects images of the first field of view area below the horizontal plane and at least part of the field of view area above the horizontal plane, aiming to monitor the middle and upper field of view areas for the cat's eye; the lower camera module 21 collects images of the second field of view area below the horizontal plane, aiming to monitor the ground and low areas for the cat's eye. Correspondingly, the upper light source module 10 and the lower light source module 20 respectively project at least floodlight onto the image acquisition areas of the corresponding camera modules to supplement light for the cameras and ensure the picture brightness.

[0043] The imaging module includes a cat's eye function mode and a face / handprint function mode; in the cat's eye function mode, the data processing and main control module 12 is used to respectively control the upper light source module 10 and the lower light source module 20 to project at least floodlight when the ambient light brightness is lower than a preset threshold, and is also used to respectively control the upper camera module 11 and the lower camera module 21 to collect images. The data processing and main control module 12 is also used to process the image data collected by the upper camera module 11 and the lower camera module 21, and respectively obtain the cat's eye images of the upper camera module 11 and / or the lower camera module 21, or combine them to obtain a combined cat's eye image or splice them to obtain a spliced cat's eye image; wherein, the field of view areas of the combined cat's eye image and the spliced cat's eye image are both larger than the field of view areas of the upper camera module 11 and the lower camera module 21; in the face / handprint function mode, the data processing and main control module 12 is used to control the upper light source module 10 to project at least floodlight, and is also used to control the upper camera module 11 to collect images; the data processing and main control module 12 is also used to process the image data collected by the upper camera module 11, obtain a face image or a handprint image, and perform live detection and identity authentication based on the face image or the handprint image.

[0044] Specifically, the imaging module is substantially multifunctional, including at least one of a cat-eye function and a face or palm recognition function. The cat-eye function is jointly implemented by multiple camera modules, that is, jointly implemented by the upper camera imaging sub-module 1 and the lower camera imaging sub-module 2. For example, the cat-eye images respectively acquired by the upper camera imaging sub-module 1 and the lower camera imaging sub-module 2 can be stitched to form a stitched cat-eye image, which is displayed on the display screen. It can be understood that the stitched cat-eye image is essentially an integrated and complete monitoring picture. In addition, the cat-eye images respectively acquired by the upper camera imaging sub-module 1 and the lower camera imaging sub-module 2 can also be combined, that is, displayed in different areas on the same display screen, and combined and displayed on the display screen without stitching according to the picture content. It can be understood that the stitched cat-eye image is essentially two separate monitoring pictures, but is synchronously displayed on the same display screen. Through the above imaging module, the imaging field of view area in the cat-eye scenario can be enlarged, effectively matching the upper middle field of view area, as well as the imaging field of view areas of the ground and low areas. Of course, the upper camera imaging sub-module 1 and the lower camera imaging sub-module 2 can also not be combined or stitched with each other, but only output images separately to meet the user's need to view the upper or lower image areas separately.

[0045] First of all, it should be noted that for the cat-eye function, the working process of the upper camera sub-module and the lower camera sub-module also needs to consider the brightness of the ambient light to determine whether the upper camera light source module 10 and the lower camera light source module 20 need to be used for supplementary lighting. Specifically, an ambient light sensor can be set to collect the ambient light, and the brightness of the ambient light is compared with a preset threshold to determine the brightness of the current intelligent door lock. When the ambient light brightness exceeds the preset threshold, it means that the current ambient light is relatively bright and clear imaging can be achieved, so no supplementary lighting is required; when the ambient light brightness is lower than the preset threshold, it means that the current ambient light is relatively dark and clear imaging cannot be achieved, and the camera needs to be supplemented with light.

[0046] Further, when the environment is bright and the cat's eye function mode is enabled, the upper camera module 11 enables the RGB mode, that is, outputs an RGB image, and the lower camera module 21 enables the RGB mode, that is, outputs an RGB image; when the environment is dark and the cat's eye function mode is enabled, the upper camera module 11 enables the IR mode, that is, outputs a black-and-white image, and the lower camera module 21 enables the IR mode, that is, outputs a black-and-white image. Optionally, according to requirements, the images of the upper imaging sub-module 1 and the lower imaging sub-module 2 can be combined or spliced and then output to the cat's eye, or the images of the upper imaging sub-module 1 or the lower imaging sub-module 2 can be separately output to the cat's eye. Optionally, when combining or splicing the images, the resolution of the images output by the upper imaging sub-module 1 and the lower imaging sub-module 2 and the area of the field of view region displayed can be flexibly adjusted according to the customer's requirements for the cat's eye display field of view and image resolution. For example, if the resolution of the finally combined or spliced image is 1280*720, if the upper camera image field of view is to account for a larger proportion, the cat's eye image output of the upper camera module 11 can be 720*720, and the cat's eye image output of the lower camera module 21 can be 560*720, and then the two cat's eye images are combined or spliced; if the lower camera image field of view is to account for a larger proportion, the cat's eye image output of the upper camera module 11 can be appropriately adjusted to 560*720, and the cat's eye image output of the lower camera module 21 can be 720*720, and then the two cat's eye images are combined or spliced.

[0047] Specifically, the fill light function is implemented by the upper light source module 10 and the lower light source module 20. Among them, the upper light source module 10 and the lower light source module 20 are used to project at least floodlight; continue to refer to Figure 2 , optionally, the upper light source module 10 at least includes a first floodlight illumination unit 101, and the lower light source module 20 at least includes a second floodlight illumination unit 104. After the cat's eye function mode is enabled, it is turned on when the environment is dark to fill light for camera image acquisition, and is synchronously turned on after the face brushing or palm brushing function mode is turned on to fill light for camera image acquisition; the fill light wavelengths of the two light source modules are not restricted to be the same. The upper camera module 11 and the lower camera module 21 are used to capture and image the reflected light of the floodlight projected onto the target object and / or the reflected light of the target object under ambient light. Among them, the upper camera module 11 is used for at least one of the cat's eye function mode and the face brushing or palm brushing function mode, and the cat's eye function mode is mainly used for target objects in the upper field of view area in general, such as people. The lower camera module 21 is only used for the cat's eye function mode and is mainly used for target objects on the ground and in low-lying areas, such as pets and express deliveries.

[0048] Specifically, the data processing and main control module 12 is used to control the switching between the cat-eye function mode and the face or palm brushing function mode, and perform image and data processing, etc. The data processing and main control module 12 has a single-chip structure, which means that only a single chip is provided in the upper camera imaging sub-module 1. This chip is fixed to the main board of the upper camera imaging sub-module 1 through surface mount technology, and is respectively on the same side or the opposite side of the main board as the upper camera light source module 10 and the upper camera module 11, and is electrically connected to the main board and electrically connected to other modules through the main board. In other words, the control and data processing functions of each module in the upper camera imaging sub-module 1 and the lower camera imaging sub-module 2 are all realized by this single chip. There is no need to set another chip on the lower camera imaging sub-module 2, nor to set control and data processing chips for the upper camera imaging sub-module 1 and the lower camera imaging sub-module 2 respectively. Instead, a single chip set in the upper camera imaging sub-module 1 is responsible for the functions of the upper and lower cameras, which can save the number of chips, simplify the module structure, and reduce costs.

[0049] Specifically, the first upper camera interface module 13, the second upper camera interface module 14, and the lower camera interface module 22 include all interfaces for communication between modules, and may optionally include interfaces such as MIPI interface, DVP interface, IIC interface, USB interface, UART serial communication interface, MIPI-DSI interface, SPI interface, etc., and can also be provided with a DC socket, an RJ45 network port, etc. The first upper camera interface module 13, the second upper camera interface module 14, and the lower camera interface module 22 are fixed to their respective main boards by welding, are electrically connected to the main board, and are electrically connected to other modules through the main board. Among them, the first upper camera interface module 13 is connected to the lower camera interface module 22 through a patch cord, and the second upper camera interface module 14 is connected to the host computer or the external device 3, and can perform communication transmission and power supply, etc.

[0050] In summary, the technical solution of the embodiment of the present invention has the following technical effects:

[0051] 1. In the traditional solution, usually two imaging modules, namely the upper camera imaging module and the lower camera imaging module, are required. At least one interface and connection line are respectively required for the two imaging modules to connect to the host computer or external device for communication transmission and power supply, etc. In the solution provided by the present invention, there is only one imaging module, and the upper camera imaging sub-module is connected to the host computer or external device through at least one interface and connection line, and then through at least one interface connected to the lower camera imaging sub-module, communication transmission and power supply, etc. can be carried out. Thus, the design of the number of customer host computer interfaces is reduced, and the volume of the imaging module layout is saved.

[0052] 2. In the traditional solution, each of the two imaging modules, namely the upper camera imaging module and the lower camera imaging module, requires at least one main control module for control, image, and data processing, that is, at least two main control chips are needed. In the solution provided by the present invention, the imaging module can realize the control, image, and data processing of the upper camera imaging sub-module and the lower camera imaging sub-module only through the data processing and main control module of the upper camera imaging sub-module, that is, only one main control chip is needed, which greatly saves costs.

[0053] 3. In the solution provided by the present invention, the imaging module has multiple functions, including at least one of a cat's-eye function and a face or palm recognition function. The cat's-eye function can enlarge the imaging field of view area in the cat's-eye scenario by combining or stitching the images of the upper camera imaging sub-module and the lower camera imaging sub-module, effectively matching the upper field of view area, as well as the imaging field of view areas of the ground and low areas.

[0054] In an optional embodiment of the present invention, the length L of the FPC transfer line 4 satisfies: 20 cm ≤ L ≤ 60 cm. Specifically, the above FPC transfer line 4 can be selected in any size from 20 to 60 cm, and its relative position with other modules is not limited. Thus, the FPC transfer line 4 can meet and adapt to the relatively far setting positions of the upper camera imaging sub-module and the lower camera imaging sub-module in the smart door lock, while ensuring that the lower camera imaging sub-module can be controlled, data processed, communication transmitted, and powered through the upper camera imaging sub-module.

[0055] Further optionally, the lower camera module 21 includes an imaging chip, the imaging chip includes a power receiving port, and multiple filter capacitors are connected in parallel to the power receiving port; and the capacitance value of at least one filter capacitor ≥ 5 μF.

[0056] In the traditional solution, the maximum line length of the above FPC transfer line 4 is usually 20 cm. When the line length of the above FPC transfer line 4 is greater than 20 cm, it may cause problems such as impaired transmission signal integrity, signal susceptibility to noise interference, unstable image output, excessive noise points, and unstable power supply for the lower camera imaging module. To solve the problems that may exist in the traditional solution with a longer transfer line, and to ensure the stability of power supply, signal transmission, and image output, multiple filter capacitors can be added near the DOVDD and / or DVDD power supply ports of the imaging chip in the lower camera module 21 for filtering, where the capacitance value of at least one filter capacitor ≥ 5 μF, to reduce the power supply ripple of the camera module, avoid the problems of impaired transmission signal integrity and signal susceptibility to noise interference, ensure the stable output of images and stable power supply of the lower camera imaging sub-module, and reduce the signal and image output noise points of the camera module.

[0057] Optionally, in some embodiments, the upper imaging sub-module 1 further includes an upper imaging main board, and the upper imaging light source module 10, the upper imaging camera module 11, the data processing and main control module 12, and the first upper imaging interface module 13 are all disposed on the upper imaging main board. The upper imaging light source module 10, the upper imaging camera module 11, and the first upper imaging interface module 13 are respectively electrically connected to the data processing and main control module 12 through the upper imaging main board. The lower imaging sub-module 2 further includes a lower imaging main board, and the lower imaging light source module 20, the lower imaging camera module 21, and the lower imaging interface module 22 are all disposed on the lower imaging main board. The lower imaging light source module 20 and the lower imaging camera module 21 are respectively electrically connected to the lower imaging interface module 22 through the lower imaging main board.

[0058] In these embodiments, the imaging module substantially adopts a co-substrate design, that is, the upper imaging sub-module 1 can make the upper imaging light source module 10 and the upper imaging camera module 11 no longer be connected to the main board as independent devices through connectors, but directly attached to the main board, and share a substrate with other modules such as the data processing and main control module 12 in the upper imaging sub-module 1. In the lower imaging sub-module 2, the lower imaging light source module 20 and the lower imaging camera module 21 are no longer connected to the main board as independent devices through connectors, but directly attached to the main board, and share a substrate with other modules in the lower imaging sub-module 2.

[0059] Optionally, in some embodiments, the lower imaging sub-module 2 is usually installed below the lock body of the smart door lock. The lower imaging sub-module 2 can be perpendicular to the lock body itself (that is, the lower panel of the lock body is horizontal and parallel to the ground), or can form a certain angle with the ground (usually the angle between the lower imaging sub-module 2 and the ground is preferably between 0 and 45°). The advantage of this installation position is strong concealment, not easy to be noticed, and a lens with a smaller field of view can be selected to accurately cover the imaging field area of the ground and low-lying areas.

[0060] Optionally, in some embodiments, the upper imaging sub-module 1 may further include a wireless module, which includes WIFI, Bluetooth, 4G, 5G, etc., for network connection and communication.

[0061] Optionally, in some embodiments, the upper imaging sub-module 1 may further include a power supply module for powering all modules.

[0062] Optionally, in some embodiments, the upper imaging sub-module 1 may further include a storage module, such as including DDR and Flash memories, for storing various types of data.

[0063] Optionally, in some embodiments, the upper imaging sub-module 1 may further include a security module for monitoring system security.

[0064] Optionally, in the embodiments of the present invention, the upper camera light source module may include a first floodlighting unit; the first floodlighting unit includes a first light source for emitting light in the infrared band; alternatively, the first floodlighting unit includes a first light source and a second light source, the first light source is for emitting light in the infrared band, and the second light source is for emitting at least one visible light band light and / or at least one infrared band light, and the infrared band light emitted by the second light source is different from the infrared band light emitted by the first light source. The lower camera light source module may include a second floodlighting unit, and the second floodlighting unit includes a third light source for emitting infrared band light or visible light band light.

[0065] Next, with reference to the accompanying drawings, the specific structures of the camera module and the light source module in the multi-functional imaging module according to the embodiments of the present invention will be exemplified and introduced. Figure 3 FIG. is a schematic diagram of the physical structure of a camera module and a light source module in a multi-functional imaging module provided by an embodiment of the present invention. Figure 4 and Figure 5 FIGS. are schematic diagrams of the structures of two camera units provided by an embodiment of the present invention. First, referring to Figures 1 - 5 , the upper camera imaging sub-module of the imaging module includes an upper camera light source module 10 and an upper camera module 11, and the lower camera imaging sub-module includes a lower camera light source module 20 and a lower camera module 21.

[0066] Specifically, the upper camera light source module 10 of this embodiment is composed of a first floodlighting unit 101; the lower camera light source module 20 is composed of a second floodlighting unit 104. The first floodlighting unit 101 at least includes a first light source (not shown in the figure), and the first light source emits light in the infrared band, which may specifically be a light-emitting diode or a laser light source; the first light source may also be a light source integrating multiple required spectra, such as emitting white light and infrared light simultaneously or independently and switchably; the first floodlighting unit 101 may also include a first light source and a second light source (not shown in the figure), the first light source emits light in the infrared band, and the second light source may be a light source including one or more colors, such as white light, red light, green light, blue light, yellow light, etc., and emits at least one visible light band light or at least one infrared band light. The second floodlighting unit 104 at least includes a third light source (not shown in the figure), and the third light source emits infrared band light or visible light band light, which may specifically be a light-emitting diode or a laser light source; the third light source may also be a light source integrating multiple required spectra, such as emitting white light and infrared light simultaneously or independently and switchably.

[0067] Continuing to refer to Figures 1 - 5, optionally, the upper camera module of this embodiment may include a first visible light camera unit 102 and a first infrared camera unit 103; the first visible light camera unit 102 may be composed of an imaging chip 10111, a lens 10112, a filter 10113, a bracket or a base; the imaging chip 10111 is an RGB imaging chip, and the filter 10113 is an infrared cut-off filter that passes visible light; the first infrared camera unit 103 may be composed of an imaging chip 10111, a lens 10112, a filter 10113, a bracket or a base, the imaging chip 10111 is an IR imaging chip, and the filter 10113 is an infrared band-pass filter, and the infrared band-pass filter is used to pass light with a wavelength corresponding to the upper camera light source module 10; the lens 10112 and the filter 10113 are respectively located on the optical path of the imaging chip 10111, and the bracket or the base is used to carry and accommodate the imaging chip 10111, the lens 10112 and the filter 10113.

[0068] Optionally, the lower camera module 21 of this embodiment includes a second infrared and visible light integrated camera unit 105, and the second infrared and visible light integrated camera unit 105 may also be composed of an imaging chip 10111, a lens 10112, a filter 10113, a bracket or a base; the imaging chip 10111 may be an RGB imaging chip or an RGB-IR imaging chip; the filter 10113 may be a dichroic filter that can pass visible light and light with a wavelength corresponding to the lower camera light source module 20; the filter 10113 may also be used in conjunction with a filter switch (IR-CUT), and two filters 10113 with different passband bands are used, one is an infrared cut-off filter that passes visible light, and the other is a full-pass glass sheet or an infrared band-pass filter; when the ambient brightness is higher than a certain threshold, the infrared cut-off filter is switched to the optical path through the IR-CUT to block light other than the visible light band; when the ambient brightness is lower than a certain threshold, the full-pass glass sheet or the infrared band-pass filter is switched to the optical path through the IR-CUT to block light other than the wavelength corresponding to the lower camera light source module 20.

[0069] Specifically, as Figure 4 shown, the lens 10112 may adopt a threaded lens, and the barrel of the lens 10112 is fixed in the base through a thread, and the base is fixed on the main board with glue; it may also be an integrated lens structure, and the barrel of the lens 10112 is fixed in the bracket with AA glue, and the bracket is fixed on the main board with glue. Additionally, optionally, as Figure 5 shown, the lens 10112 may also adopt a metasurface lens, a diffractive lens, a refractive lens or any hybrid form of the above lenses, and is fixed in the barrel of the lens 10112. Optionally, as Figure 4As shown, the above-mentioned filter 10113 can be integrated into the lens barrel of the lens 10112, or directly fixed on a bracket or a base. Additionally, optionally, as Figure 5 shown, when the lens 10112 adopts a metasurface lens, the filter 10113 can also adopt a filter film, which is plated on the upper surface or the lower surface of the microlens structure of the superlens, so that the lens 10112 integrates the imaging function and the filter function of the filter at the same time.

[0070] The advantages of this solution are as follows: 1. The upper camera module can simultaneously implement at least one of the cat-eye function and the face or palm swiping function; 2. The upper camera module is divided into an independent first visible light camera unit and a first infrared camera unit, each with its own filter, which can avoid the difficult control of filter switching and reduce costs; 3. In the upper camera module, the first visible light camera unit adopts an infrared cut-off filter that passes visible light, and the first infrared camera unit adopts an infrared band-pass filter for passing light with a wavelength corresponding to the lower shooting light source module. Therefore, there is no color cast problem in the images output by the upper camera module; 4. The lower camera module adopts a second infrared and visible light integrated camera unit, which can reduce the number of camera units, thereby saving the material and assembly costs of the imaging module.

[0071] Optionally, Figure 6 is a schematic diagram of the physical structure of the camera module and the light source module in another multi-functional imaging module provided by an embodiment of the present invention. As Figure 6 shown, compared with the embodiment shown in Figure 3 In this embodiment, the lower camera module 21 can also include a second visible light camera unit 106 and a second infrared camera unit 107; the second visible light camera unit 106 can be composed of an imaging chip 10111, a lens 10112, a filter 10113, a bracket or a base; the imaging chip 10111 is an RGB imaging chip, and the filter 10113 is an infrared cut-off filter that passes visible light; the second infrared camera unit 107 can be composed of an imaging chip 10111, a lens 10112, a filter 10113, a bracket or a base. The imaging chip 10111 is an IR imaging chip, and the filter 10113 is an infrared band-pass filter, and the infrared band-pass filter is used to pass light with a wavelength corresponding to the lower shooting light source module 20; the lens 10112 and the filter 10113 are respectively located on the optical paths of the imaging chip 10111, and the bracket or the base is used to carry and accommodate the imaging chip 10111, the lens 10112 and the filter 10113.

[0072] The advantages of this solution are as follows: 1. The lower camera module is divided into an independent second visible light camera unit and a second infrared camera unit, each of which also has its own filter, which can avoid the difficult control of filter switching; 2. The second visible light camera unit in the lower camera module uses an infrared cut-off filter that passes visible light, and the second infrared camera unit uses an infrared band-pass filter to pass light with a wavelength corresponding to the lower light source module. Therefore, the images output by the lower camera module do not have a color cast problem.

[0073] Optionally, Figure 7 FIG. is a schematic diagram of the physical structure of the camera module and the light source module in another multi-functional imaging module provided by an embodiment of the present invention. As Figure 7 shown, compared with the Figure 3 embodiment shown, the upper camera module 11 of this embodiment may also include a first infrared and visible light integrated camera unit 108, which may be composed of an imaging chip 10111, a lens 10112, a filter 10113, a bracket or a base; the imaging chip 10111 may be an RGB imaging chip or an RGB-IR imaging chip; the filter 10113 may be a dichroic filter that can pass visible light and light with a wavelength corresponding to the upper light source module 10; the filter 10113 may also be used in combination with IR-CUT, and two filters 10113 with different passband bands are used, one is an infrared cut-off filter that passes visible light, and the other is an infrared band-pass filter; when the ambient brightness is higher than a certain threshold, the infrared cut-off filter is switched to the optical path through IR-CUT to block light outside the visible light band; when the ambient brightness is lower than a certain threshold, the all-pass glass or the infrared band-pass filter is switched to the optical path through IR-CUT to block light outside the wavelength corresponding to the upper light source module 10.

[0074] The advantages of this solution are as follows: Both the upper camera module and the lower camera module adopt infrared and visible light integrated camera units, which can further reduce the number of camera units, thereby saving the material and assembly costs of the imaging module.

[0075] Optionally, Figure 8 FIG. is a schematic diagram of the physical structure of the camera module and the light source module in another multi-functional imaging module provided by an embodiment of the present invention. As Figure 8 shown, compared with the Figure 3 embodiment shown, the upper camera module of this embodiment may be composed of a first infrared and visible light integrated camera unit 108, and the lower camera module 21 may be composed of a second visible light camera unit 106 and a second infrared camera unit 107.

[0076] The advantages of this solution are as follows: 1. The upper camera module adopts the first infrared and visible light integrated camera unit, which can also reduce the number of camera units, thereby saving the material and assembly costs of the imaging module; 2. The lower camera module is divided into an independent second visible light camera unit and a second infrared camera unit, each of which also has its own filter, which can avoid the difficult control of filter switching; 3. In the lower camera module, the second visible light camera unit adopts an infrared cut-off filter that passes visible light, and the second infrared camera unit adopts an infrared band-pass filter for passing light with a wavelength corresponding to the lower light source module. Therefore, the output image of the lower camera module has no color cast problem.

[0077] Figures 3 - 8 Only examples of the specific position distributions of each device are given, and they can be set according to actual needs during specific applications, and are not limited here.

[0078] Optionally, in some embodiments, the upper imaging sub-module 1 may further be provided with an ambient light sensor or a proximity light sensor. The ambient light sensor is used to detect the illuminance of the environment and sense whether the illuminance of the surrounding environment is lower than a certain threshold. Generally, the setting range of this threshold is 5 - 100 Lux. When the surrounding ambient illuminance increases or decreases, other modules can be selectively controlled to make responses. The proximity sensor is used to sense whether there is a target object within the working distance range of the imaging module, and generate a trigger signal when the target object approaches, and other modules can selectively make responses according to the trigger signal.

[0079] Optionally, in the embodiments of the present invention, the upper light source module may further include a structured light projection unit, and the structured light projection unit is used to project structured light; the structured light projection unit includes a fourth light source, and the fourth light source is used to emit light in the infrared band. Hereinafter, with reference to the drawings, the specific structure of the light source module with a structured light projection unit in the embodiments of the present invention will be further exemplified and introduced.

[0080] Figure 9 is a schematic physical structure diagram of a camera module and a light source module in another multifunctional imaging module provided by the embodiments of the present invention, Figure 10 and Figure 11 are schematic structure diagrams of two structured light projection units provided by the embodiments of the present invention. Referring to Figure 9 , compared with Figure 3 shown in the embodiment, a structured light projection unit 109 can be added to the upper light source module 10 in this embodiment for projecting structured light. In the upper light source module 10, the first floodlighting unit 101 and the structured light projection unit 109 can be divided into two independent units. Specifically, as Figure 10 and Figure 11As shown in the figure, the structured light projection unit 109 includes at least a fourth light source 10101 and a dimming element. The fourth light source 10101 is used to emit light in the infrared band, and specifically can be a laser light source such as a VCSEL (Vertical-Cavity Surface-Emitting Laser), an EEL (Edge-Emitting Laser), or an HCSEL (Horizontal-Cavity Surface-Emitting Laser). The dimming element is located on the light-emitting side of the fourth light source 10101, and specifically can be a collimating mirror 10102 and a diffractive optical element 10103, or a collimating diffractive integrated optical element 10104, or a metasurface lens 10105, or a reasonable combination of the above three possible optical elements. After passing through the dimming element, the fourth light source 10101 projects structured light with certain characteristic information onto the imaging target, such as common speckle structured light.

[0081] In this embodiment, structured light can be projected in the face or palm brushing function mode. After forming reflected light on the face or palm, the speckle pattern of the face or palm can be directly collected, or a three-dimensional structured light image carrying the depth information of the face or palm can be collected, and the depth image can be output using this structured light image. Thus, the above-mentioned speckle pattern or depth image is used for liveness judgment, enriching the information input for liveness judgment, thereby increasing the recognition accuracy of the face and palm brushing functions, effectively preventing or avoiding deception or attacks on the recognition system, increasing security, and further improving the usage scenario of the imaging module.

[0082] Figure 12 It is a schematic diagram of the physical structure of the camera module and the light source module in another multi-functional imaging module provided by the embodiment of the present invention. Figure 13 and Figure 14 are schematic diagrams of the structures of two upper-shot light source modules provided by the embodiment of the present invention. Compared with the embodiment shown in Figure 9 In this embodiment, the upper-shot light source module can further adopt a two-in-one solution of the structured light projection unit and the flood illumination unit. As shown in Figure 12 The first flood illumination unit and the structured light projection unit of the upper-shot light source module can be integrated in the same projector 110.

[0083] Specifically, as shown in Figure 13 and Figure 14 The projector 110 includes a first light source 10100, a fourth light source 10101, and a spacer block 10106. The first light source 10100 and the fourth light source 10101 emit light alternately and share a dimming element. The first light source 10100 is used to emit floodlight, and the fourth light source 10101 is used to emit structured light. The spacer block 10106 can generally be a printed circuit board, a ceramic substrate, or a metal conductive block, and can be placed below the first light source 10100 or below the fourth light source 10101 (in the figure, the spacer block 10106 is placed below the fourth light source 10101). The dimming element is located on the light-emitting side of the light source, and specifically can be Figure 13The collimating mirror 10102 and the diffractive optical element 10103 shown can be Figure 14 The integrated collimating and diffractive optical element 10104 shown can be a metasurface lens 10105, or a reasonable combination of the above three possible optical elements. The first light source 10100 projects uniform floodlight towards the imaging target after passing through the dimming element; the fourth light source 10101 projects structured light towards the imaging target after passing through the dimming element.

[0084] Compared with Figure 9 In the shown embodiment, in this embodiment, the structured light projection unit and the first floodlight illumination unit are further integrated in a projector 110, which can improve the usage scenario of the imaging module while simplifying the complexity of the module, and is easy to manufacture and produce.

[0085] Figure 15 It is a schematic diagram of the connection structure between a data processing and main control module and a camera module provided by an embodiment of the present invention. Refer to Figure 15 , in an alternative embodiment of the present invention, an upper camera imaging sub-module may further be provided to include an MIPI switch 5; the MIPI switch 5 includes a first input terminal 51, a second input terminal 52, and an output terminal 53; the first input terminal 51 is electrically connected to the upper camera module 11, the second input terminal 52 is electrically connected to the lower camera module 21, and the output terminal 53 is electrically connected to the data processing and main control module 12; the MIPI switch 5 is configured to switch between a first conduction state and a second conduction state; wherein, the first conduction state is that the first input terminal 51 is in conduction with the output terminal 53, and the second conduction state is that the second input terminal 52 is in conduction with the output terminal 53.

[0086] Wherein, the first conduction state and the second conduction state refer to two MIPI channel conduction states of the MIPI switch 5. By switching between the two MIPI channel conduction states, the data processing and main control module 12 can be connected to different camera units, and receive and process the image data collected by the connected camera units.

[0087] Specifically, in this embodiment, in order to control multiple camera modules and process images and data using only one data processing and main control module 12, the data processing and main control module 12 needs to have multiple MIPI channels. In the upper camera module 11 and the lower camera module 21, each single camera unit corresponds to at least one MIPI channel. Since the more MIPI channels the data processing and main control module 12 requires, the higher its chip cost, in this embodiment, an MIPI switch 5 is used to enable the lower camera module 21 and the upper camera module 11 to share MIPI channels. For example, a signal can be initiated at the end of each frame to start the MIPI switch to achieve the switching of MIPI channels. The data processing and main control module 12 can multiplex the upper camera module 11 and the lower camera module 21 according to the switching of MIPI channels, and correspondingly switch control logics, image processing methods, data processing methods, etc. If it is selected that the camera of the lower camera module 21 and the infrared camera of the upper camera module 11 share one MIPI channel, the frame rate of the visible light camera in the upper camera module 11 can be guaranteed, and the frame rate of the cat's eye function mode implemented by the visible light camera can be guaranteed; if the camera of the lower camera module 21 and the visible light camera of the upper camera module 11 share one MIPI channel, the face brushing or palm brushing function mode implemented by the infrared camera in the upper camera module 11 can be made faster. Thus, using the MIPI switch 5 can save the number of MIPI channels required by the data processing and main control module 12, select a chip with a lower cost, and save costs.

[0088] Figure 16 Yes Figure 15 A specific connection schematic diagram of the data processing and main control module and the camera module shown, refer to Figure 16 , in a specific embodiment, it is optional that the upper camera module 11 includes a first infrared camera unit 103 and a first visible light camera unit 102; the lower camera module 21 includes a second infrared and visible light integrated camera unit 105; the first infrared camera unit 103 or the first visible light camera unit 102 is electrically connected to the first input end 51; the second infrared and visible light integrated camera unit 105 is electrically connected to the second input end 52; the first conduction state is that the first input end 51 is conducted with the output end 53, and the second conduction state is that the second input end 52 is conducted with the output end 53.

[0089] This embodiment means that based on Figure 3In the setting mode of the camera module in the embodiment, the first visible light camera unit 102 or the first infrared camera unit 103 of the upper camera module 11 and the second infrared and visible light integrated camera unit 105 of the lower camera module 21 can share a single MIPI channel, that is, they are connected to the data processing and main control module 12 through the MIPI switch 5, and the visible light images in the upper camera module 11 and the images in the lower camera module 21 are multiplexed time-divisionally, or the infrared images in the upper camera module 11 and the images in the lower camera module 21 are multiplexed time-divisionally.

[0090] Figure 17 Yes Figure 15 Another specific connection schematic diagram of the data processing and main control module and the camera module shown, refer to Figure 17 , in another specific embodiment, it is optional that the upper camera module 11 includes a first infrared camera unit 103 and a first visible light camera unit 102, and the lower camera module 21 includes a second infrared camera unit 107 and a second visible light camera unit 106; the first input end 51 includes a first sub-input end 511 and a second sub-input end 512; the first infrared camera unit 103 is electrically connected to the first sub-input end 511, and the first visible light camera unit 102 is electrically connected to the second sub-input end 512; the second input end 52 includes a third sub-input end 523 and a fourth sub-input end 524; the second infrared camera unit 107 is electrically connected to the third sub-input end 523, and the second visible light camera unit 106 is electrically connected to the fourth sub-input end 524.

[0091] The first conduction state includes a first sub-conduction state and a second sub-conduction state. The first sub-conduction state is that the first sub-input end 511 is conducted with the output end 53, and the second sub-conduction state is that the second sub-input end 512 is conducted with the output end 53; the second conduction state includes a third sub-conduction state and a fourth sub-conduction state. The third sub-conduction state is that the third sub-input end 523 is conducted with the output end 53, and the fourth sub-conduction state is that the fourth sub-input end 524 is conducted with the output end 53; the MIPI switch 5 is configured to switch between the first sub-conduction state, the second sub-conduction state, the third sub-conduction state, and the fourth sub-conduction state.

[0092] This embodiment means that based on Figure 6In the camera module setting method where the upper camera module 11 and the lower camera module 21 in the embodiment are each provided with an infrared camera unit and a visible light camera unit, the first visible light camera unit 102 and the first infrared camera unit 103 of the upper camera module 11 can share a single MIPI channel with the second visible light camera unit 106 and the second visible light camera unit 106 of the lower camera module 21. That is, they are connected to the data processing and main control module 12 through the MIPI switch 5, and the visible light images, infrared images of the upper camera module 11 and the visible light images, infrared images of the lower camera module 21 are multiplexed in time.

[0093] It should be noted here that since the upper camera module 11 and the lower camera module 21 can be provided with more than one camera unit, in addition to connecting the camera unit and the data processing and main control module 12 through the MIPI switch 5 in the above embodiment, some camera units can also be directly connected to the data processing and main control module 12, occupying a single MIPI channel of the data processing and main control module 12 alone. Thus, when the camera unit occupying the MIPI channel alone is working, the collected images can be directly provided to the data processing and main control module 12 for processing and output.

[0094] Figure 18 It is a schematic diagram of the working process of a data processing and main control module provided by an embodiment of the present invention. Refer to Figure 18 , Refer to Figure 15 And Figure 18 , In the embodiment of the present invention, the working process of the multi-functional imaging module may include a first time-division multiplexing stage T1. The first time-division multiplexing stage T1 includes multiple first processing cycles t1. Each first processing cycle t1 includes multiple image output frames, and the number of image output frames in different first processing cycles t1 is the same; each first processing cycle t1 includes M1 first image output frames ta and N1 second image output frames tb. In the figure, the solid waveform represents the first image output frame ta, and the solid waveform represents the second image output frame tb; where M1 and N1 are both positive integers; the MIPI switch 5 is configured to be in a first conduction state during the first image output frame ta and in a second conduction state during the second image output frame tb.

[0095] It can be understood that since the MIPI switching switch 5 has different conduction states, that is, in the two conduction states of the MIPI switching switch 5, namely the first conduction state and the second conduction state, the data processing and main control module 12 can respectively process and obtain different images, realizing the output of images collected by different camera modules in different image output frames, namely the first image output frame ta and the second image output frame tb. Specifically, in the first conduction state, the data processing and main control module 12 is connected to the upper camera module 11, and in the second conduction state, the data processing and main control module 12 is connected to the lower camera module 21. Therefore, it can be known that the M1 first image frames and N1 second image frames included in the first processing cycle t1 indicate that a certain number of images collected by the upper camera module 11 and a certain number of images collected by the lower camera module 21 can be allocated and output in the multiple image output frames in this processing cycle. The specific values of M1 and N1 can control the output ratio of the images collected by the upper camera module 11 and the lower camera module 21 in this processing cycle. For example, when M1 = N1, it means that the data processing and main control module 12 can, through the conduction control of the MIPI switching switch 5, output the images collected by the upper camera module 11 in half of the image output frames and output the images collected by the lower camera module 21 in the other half of the image output frames. And, the optional MIPI switching switch 5 can alternately be in the first conduction state and the second conduction state. Thus, the data processing and main control module 12 alternately outputs the images collected by the upper camera module 11 and the lower camera module 21. Further, by maintaining the cyclic output of images in the first time-division multiplexing stage T1 with the first processing cycle t1, the images of the upper camera module 11 and the lower camera module 21 can be stably output in a fixed ratio, forming a continuous video output to meet the requirements of cat's-eye monitoring.

[0096] Figure 19 is another schematic diagram of the working process of the data processing and main control module provided by the embodiment of the present invention. Refer to Figure 19 , in another embodiment of the present invention, the working process of the multi-functional imaging module may further include a second time-division multiplexing stage T2. The second time-division multiplexing stage T2 includes multiple second processing cycles t2. The second processing cycle t2 includes multiple image output frames, and the number of image output frames in different second processing cycles t2 is the same; the second processing cycle t2 includes M2 first image output frames ta and N2 second image output frames tb. In the figure, the solid waveform also represents the first image output frame ta and the solid waveform represents the second image output frame tb; where M2 and N2 are both positive integers; M2 / N2 ≠ M1 / N1.

[0097] In this embodiment, the working logic of the second time-division multiplexing stage T2 is the same as that of the first time-division multiplexing stage T1. Both control the ratio of the two image output frames, namely the first image output frame ta and the second image output frame tb, in each processing cycle by switching the two conduction states of the mipi switch 5. The only difference is that the ratio of the two image output frames in the second processing cycle t2 is different, that is, M2 / N2 ≠ M1 / N1, thus realizing the time-division multiplexing stage with two output ratios. It can be seen that in the embodiment of the present invention, by controlling the ratio of the conduction states of the mipi switch 5, the output ratio of the images collected by the upper camera module 11 and the lower camera module 21 can be adjusted, so as to adapt to different application scenarios of the intelligent door lock, flexibly increase the image output ratio of the upper camera module 11 or the lower camera module 21, and meet the requirements of cat's eye monitoring.

[0098] It should be noted that, as Figure 18 and Figure 19 show, the time of the first time-division multiplexing stage T1 and the second time-division multiplexing stage T2 can be the same or different, and the number of image output frames in the first processing cycle t1 and the second processing cycle t2 can be the same or different. Those skilled in the art can make selections and adjustments to meet the requirements of specific scenarios.

[0099] The advantages of this embodiment are as follows: First, by adopting the Figure 18 and / or Figure 19 solution of the embodiment shown, it can adapt to different application scenarios of the intelligent door lock, flexibly increase the image output ratio of the upper camera module 11 or the lower camera module 21, and meet the requirements of cat's eye monitoring. Second, by adopting the Figure 18 and / or Figure 19 solution of the embodiment shown, the number of image output frames in the face recognition or palm recognition function mode can be increased according to the scene needs, the recognition speed of face recognition or palm recognition can be improved, and thus the response speed of the face recognition and palm recognition function modes can be improved. Third, by adopting the Figure 18 and / or Figure 19 solution of the embodiment shown, different image processing can be flexibly performed on different frame images according to the scene needs, the image processing flow of some frame images can be simplified, and thus the response speed of the cat's eye or face recognition and palm recognition function modes can be further improved.

[0100] Note that the above is only a preferred embodiment of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments, combinations with each other, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, it can also include more other equivalent embodiments, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A multifunctional imaging module integrating a multi-camera module for an intelligent door lock, characterized in that, It includes an upper camera imaging sub-module and a lower camera imaging sub-module; The upper camera imaging sub-module includes an upper camera light source module, an upper camera module, a data processing and main control module, a first upper camera interface module, and a second upper camera interface module; the data processing and main control module is of a single-chip structure; the upper camera light source module, the upper camera module, the first upper camera interface module, and the second upper camera interface module are respectively electrically connected to the data processing and main control module; The lower camera imaging sub-module includes a lower camera light source module, a lower camera module, and a lower camera interface module; the lower camera light source module and the lower camera module are respectively electrically connected to the lower camera interface module; The second upper camera interface module is used to be electrically connected to a host computer or an external device, and the lower camera interface module is electrically connected to the first upper camera interface module through an FPC transfer line, so that the lower camera light source module and the lower camera module in the lower camera imaging sub-module are respectively electrically connected to the data processing and main control module in the upper camera imaging sub-module, and the lower camera imaging sub-module is electrically connected to the host computer or the external device through the upper camera imaging sub-module; The upper camera module includes a first infrared camera unit and a first visible light camera unit, or includes a first infrared and visible light two-in-one camera unit; the lower camera module includes a second infrared camera unit and a second visible light camera unit, or includes a second infrared and visible light two-in-one camera unit; The upper camera module is used to collect images of a first field of view area below the horizontal plane and at least part of the field of view area above the horizontal plane, and the lower camera module is used to collect images of a second field of view area below the horizontal plane, and respectively obtain image data; The upper camera light source module is used to project at least floodlight on the first field of view area below the horizontal plane and at least part of the field of view area above the horizontal plane, and the lower camera light source module is used to project at least floodlight on the second field of view area below the horizontal plane; The imaging module includes a cat's eye function mode and a face / handprint function mode; In the cat's eye function mode, the data processing and main control module is used to respectively control the upper camera light source module and the lower camera light source module to project at least floodlight when the ambient light brightness is lower than a preset threshold, and is also used to respectively control the upper camera module and the lower camera module to collect images. The data processing and main control module is also used to process the image data collected by the upper camera module and the lower camera module, respectively obtain the cat's eye images of the upper camera module and / or the lower camera module, or perform combination to obtain a combined cat's eye image or perform stitching to obtain a stitched cat's eye image; wherein, the field of view areas of the combined cat's eye image and the stitched cat's eye image are respectively larger than the field of view areas of the upper camera module and the lower camera module; In the face / hand brushing function mode, the data processing and main control module is used to control the upper camera light source module to project at least floodlight, and is also used to control the upper camera module to collect images; the data processing and main control module is further used to process the image data collected by the upper camera module to obtain a face brushing image or a hand brushing image, and perform live detection and identity authentication based on the face brushing image or the hand brushing image.

2. The multi-functional imaging module according to claim 1, wherein The upper camera imaging sub-module further includes an MIPI switch. The MIPI switch includes a first input terminal, a second input terminal, and an output terminal; the first input terminal is electrically connected to the upper camera module, the second input terminal is electrically connected to the lower camera module, and the output terminal is electrically connected to the data processing and main control module. The MIPI switch is configured to switch between a first conduction state and a second conduction state; wherein, the first conduction state is that the first input terminal is in conduction with the output terminal, and the second conduction state is that the second input terminal is in conduction with the output terminal.

3. The multifunctional imaging module according to claim 2, wherein The upper camera module includes a first infrared camera unit and a first visible light camera unit; the lower camera module includes a second infrared and visible light integrated camera unit. The first infrared camera unit or the first visible light camera unit is electrically connected to the first input terminal; the second infrared and visible light integrated camera unit is electrically connected to the second input terminal. The first conduction state is that the first input terminal is in conduction with the output terminal, and the second conduction state is that the second input terminal is in conduction with the output terminal.

4. The multi-functional imaging module according to claim 2, wherein, The upper camera module includes a first infrared camera unit and a first visible light camera unit, and the lower camera module includes a second infrared camera unit and a second visible light camera unit. The first input terminal includes a first sub-input terminal and a second sub-input terminal; the first infrared camera unit is electrically connected to the first sub-input terminal, and the first visible light camera unit is electrically connected to the second sub-input terminal; the second input terminal includes a third sub-input terminal and a fourth sub-input terminal; the second infrared camera unit is electrically connected to the third sub-input terminal, and the second visible light camera unit is electrically connected to the fourth sub-input terminal. The first conduction state includes a first sub-conduction state and a second sub-conduction state, the first sub-conduction state is that the first sub-input terminal is in conduction with the output terminal, and the second sub-conduction state is that the second sub-input terminal is in conduction with the output terminal. The second conduction state includes a third sub-conduction state and a fourth sub-conduction state, the third sub-conduction state is that the third sub-input terminal is in conduction with the output terminal, and the fourth sub-conduction state is that the fourth sub-input terminal is in conduction with the output terminal. The MIPI switch is configured to switch between the first sub-conduction state, the second sub-conduction state, the third sub-conduction state, and the fourth sub-conduction state.

5. The multi-functional imaging module according to claim 2, wherein The working process of the multifunctional imaging module includes a first time-division multiplexing stage, and the first time-division multiplexing stage includes a plurality of first processing cycles. Each first processing cycle includes a plurality of image output frames, and the number of image output frames in different first processing cycles is the same; Each first processing cycle includes M1 first image output frames and N1 second image output frames; where M1 and N1 are both positive integers; The MIPI switch is configured to be in the first conduction state during the first image output frame and in the second conduction state during the second image output frame.

6. The multi-functional imaging module according to claim 5, wherein, The working process of the multifunctional imaging module further includes a second time-division multiplexing stage. The second time-division multiplexing stage includes a plurality of second processing cycles. Each second processing cycle includes a plurality of image output frames, and the number of image output frames in different second processing cycles is the same; Each second processing cycle includes M2 first image output frames and N2 second image output frames; where M2 and N2 are both positive integers; M2 / N2 ≠ M1 / N1.

7. The multi-functional imaging module according to claim 1, wherein The length L of the FPC jumper cable satisfies: 20 cm ≤ L ≤ 60 cm.

8. The multifunctional imaging module according to claim 1, wherein, The lower camera module includes an imaging chip, and the imaging chip includes a power receiving port. A plurality of filter capacitors are connected in parallel to the power receiving port; and the capacitance value of at least one of the filter capacitors ≥ 5 μF.

9. The multifunctional imaging module according to claim 1, wherein The upper camera imaging sub-module further includes an upper camera main board. The upper camera light source module, the upper camera module, the data processing and main control module, and the first upper camera interface module are all arranged on the upper camera main board. The upper camera light source module, the upper camera module, and the first upper camera interface module are respectively electrically connected to the data processing and main control module through the upper camera main board; The lower camera imaging sub-module further includes a lower camera main board. The lower camera light source module, the lower camera module, and the lower camera interface module are all arranged on the lower camera main board. The lower camera light source module and the lower camera module are respectively electrically connected to the lower camera interface module through the lower camera main board.

10. The multi-functional imaging module according to claim 1, wherein The upper camera light source module includes a first omnidirectional illumination unit; the first omnidirectional illumination unit includes a first light source for emitting light in the infrared band; or, the first omnidirectional illumination unit includes a first light source and a second light source. The first light source is used for emitting light in the infrared band, and the second light source is used for emitting at least one visible light band light and / or at least one infrared band light, and the infrared band light emitted by the second light source is different from the infrared band light emitted by the first light source; The lower camera light source module includes a second omnidirectional illumination unit, and the second omnidirectional illumination unit includes a third light source for emitting infrared band light or visible light band light; And / or, the upper camera light source module further includes a structured light projection unit for projecting structured light; The structured light projection unit includes a fourth light source for emitting light in the infrared band.

Images