System for monitoring passengers in automobile cabin based on OMS camera

Abstract

The invention relates to a passenger monitoring system in an automobile cabin based on an OMS camera, which comprises an OMS camera assembly installed near an inner rearview mirror, and the OMS camera assembly comprises a camera shell, a front panel, a rear cover, an OMS lens, an LED lamp and a PCBA board, the PCBA board is also integrated with an alarm module, a remnant monitoring and searching module, an alarm processing module, an RGB-IR module, and an ISP determination module. The PCBA board comprises an image sensor, an image processor, an encoder, a lamp driving module, a connector and a power supply which are arranged on a circuit board; the automobile safety monitoring system has the advantages that the safety performance of an automobile is further improved by monitoring the sensing data of passengers in the cabin, an alarm is given for reminding if dangerous behaviors occur, and remnant objects in the automobile can be monitored, so that pets and children left in the automobile can obtain safety prompts.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile safety assisted driving, and in particular to an automobile cabin occupant monitoring system based on an OMS camera and using RGB-IR. Background Art

[0002] At present, in order to ensure the safety of motor vehicle drivers and passengers, motor vehicles are usually equipped with an occupant monitoring system (OMS) to assist drivers in judging their own status. The occupant monitoring system (OMS) is an extension of the DMS system. It can further improve the safety performance of the car by monitoring the perception data of passengers in the cockpit. If dangerous behavior occurs, an alarm will be issued. The occupant monitoring system (OMS) relies on the image input of the camera and uses deep learning algorithms to monitor the driver and passengers in real time for face recognition, fatigue monitoring, attention monitoring, and behavior monitoring. Image quality has become a key link in the entire cockpit visual monitoring system. In a normal car use environment, there is a relatively dark light, which leads to poor quality of the collected images. How to accurately and real-time obtain the visual ability in the vehicle cockpit under the current environment is a problem that needs to be solved urgently. In addition, the functions of the occupant monitoring systems currently on the market are relatively simple. Summary of the invention

[0003] In view of the above problems, the purpose of the present invention is to provide an automobile cabin occupant monitoring system based on an OMS camera, which uses RGB-IR technology and can simultaneously capture RGB color images and IR images through a sensor device, that is, it has visibility during both daytime and nighttime, so as to overcome the shortcomings of the above-mentioned prior art.

[0004] The present invention provides an automobile cabin occupant monitoring system based on an OMS camera, comprising: an OMS camera assembly installed near an interior rearview mirror, the OMS camera assembly comprising: a camera housing, a front panel, a rear cover, an OMS lens, an LED lamp and a PCBA board, the PCBA board also integrating an alarm module, a leftover monitoring and searching module, an alarm processing module, an RGB-IR module, and an ISP determination module;

[0005] The camera housing has two front-end mounting cavities and one rear-end mounting cavity, the OMS lens and the LED lamp are embedded in the two front-end mounting cavities, the front panel is installed at the front end of the camera housing, and two avoidance holes are provided on the front panel corresponding to the positions of the OMS lens and the LED lamp;

[0006] The PCBA board is embedded in the rear installation cavity, and the rear cover is installed at the rear end of the camera housing and seals the PCBA board in the camera housing;

[0007] The PCBA board includes: an image sensor, an image processor, an encoder, a light driving module, a connector, and a power supply installed on the circuit board. The image sensor is connected to the OMS lens and is used to collect images. The image processor is connected to the image sensor, the encoder, and the light driving module respectively. The image processor is used to receive image information from the image sensor and process and identify it. The encoder is used to compile and convert the data processed by the image processor into a communication signal. The connector is used to transmit the communication signal processed by the encoder to the outside. The light driving module is used to drive the LED light. The power supply is used to power the circuit board.

[0008] The RGB-IR module is used to capture an RGB color image and an IR infrared image through an image sensor and output the RGB color image and the infrared image to an image processor;

[0009] The ISP determination module is used to identify and determine the brightness of the image collected by the image sensor, and automatically switch the RGB color image output or the IR infrared image output of the RGB-IR module according to the brightness of the image, wherein the IR image output is switched to the IR image output when the light is relatively dark, and the RGB color image output is switched to the RGB color image output when the light is sufficient;

[0010] The alarm module is used to determine whether the driver and passengers have dangerous behaviors, and to issue an alarm reminder when dangerous behaviors occur; dangerous behaviors of passengers include whether they wear seat belts, whether their limbs are stretched out of the window, etc.; wherein the alarm reminder includes one or more of the following methods, including: voice reminder, steering wheel vibration reminder, instrument panel display reminder, seat belt vibration reminder;

[0011] The left-behind monitoring and searching module is used to monitor and search for left-behind items and left-behind creatures in the vehicle, wherein the identified left-behind creatures include pets and children, and the identified left-behind items include handbags, keys, documents, etc.;

[0012] The alarm processing module is used for users to receive biological left-behind alarms and users to cancel biological left-behind alarms; wherein, the user receives the biological left-behind alarm in the car, including autonomously detecting whether there are biological left in the car and detecting the danger after the owner turns off the engine, locks the car and leaves the car, notifying the owner of the possible biological left-behind danger alarm by using the APP or CallCenter, and prompting the owner to return to the car as soon as possible to handle it; wherein, the user cancels the biological left-behind alarm in the following ways: after the owner receives the biological left-behind alarm, the owner returns and completes the processing, or the owner remotely processes it on the App (opens the car window) and clicks to cancel the alarm, or calls the CallCenter to confirm the cancellation of the alarm.

[0013] As a preferred embodiment of the present invention, the FOV viewing angle range of the OMS camera assembly is: 130 degrees for horizontal viewing angle and 96 degrees for vertical viewing angle.

[0014] Another object of the present invention is to provide a processing method of an automobile cabin occupant monitoring system based on an OMS camera, which specifically comprises the following steps:

[0015] Step S1: Using the RGB-IR method on the OMS camera assembly, the ISP determines the brightness of the image, and automatically switches to IR image output in dark conditions to enhance the reality effect. Using RGB-IR technology, a sensor device can capture both RGB color images and IR images at the same time, that is, it has both daytime and nighttime visibility; the RGB-IR method allows a sensor to capture RGB color images and infrared images at the same time, which provides dual visibility during the day and at night, and enhances image quality under different lighting conditions.

[0016] Step S2: using the alarm module to determine whether the driver and passengers have dangerous behaviors, and issuing an alarm reminder when dangerous behaviors occur; wherein, the dangerous behaviors of passengers include whether they wear seat belts, whether their limbs are stretched out of the window, and the alarm reminder includes one or more of the following methods, including: voice reminder, steering wheel vibration reminder, instrument panel display reminder, seat belt vibration reminder;

[0017] Step S3: using the left-behind monitoring and searching module to monitor and search for left-behind items and left-behind creatures in the vehicle, wherein the identified left-behind creatures include pets and children, and the identified left-behind items include handbags, keys, and documents;

[0018] Step S4: using the alarm processing module to notify the user of receiving the biological left behind alarm in the car and the user to cancel the biological left behind alarm; wherein, the user receives the biological left behind alarm in the car, including autonomously detecting whether there are biological left in the car and detecting the danger after the owner turns off the engine, locks the car and leaves the car, notifying the owner of the possible biological left behind danger alarm by using the APP or CallCenter, and prompting the owner to return to the car as soon as possible to handle it; wherein, the user cancels the biological left behind alarm in the following ways: after the owner receives the biological left behind alarm, the owner returns and completes the processing, or the owner processes it remotely on the App (opens the car window) and clicks to cancel the alarm, or calls the CallCenter to confirm the cancellation of the alarm.

[0019] The beneficial effects of the present invention are as follows:

[0020] 1. The present invention further improves the safety performance of the car by monitoring the perception data of the passengers in the cabin. If dangerous behavior occurs, an alarm will be issued, and objects left in the car can be monitored so that pets and children left in the car can receive safety reminders.

[0021] 2. The present invention determines whether the driver and passengers have dangerous behaviors, and will issue an alarm reminder when dangerous behaviors occur. In addition, the alarm module adds multiple alarm reminder modes to further improve safety performance.

[0022] 3. The camera of the present invention adopts RGB-IR technology, and determines the brightness of the image through ISP. When the light is relatively dark, it automatically switches to IR image output to enhance the reality effect. Using RGB-IR technology, RGB color images and IR images can be captured by a sensor device at the same time, that is, it has the ability to see both during the day and at night. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more apparent and easy to understand. In the accompanying drawings:

[0024] Figure 1 This is a schematic diagram of the overall structure of Example 1.

[0025] Figure 2 This is a circuit block diagram of the first embodiment.

[0026] Description of the drawings: camera housing 1, front panel 2, back cover 3, OMS lens 4, LED light 5, PCBA board 6, image sensor 7, image processor 8, encoder 9, light driving module 10, connector 11, power supply 12, memory 13, resetter 14. DETAILED DESCRIPTION

[0027] Example 1

[0028] See also Figure 1-2As shown, the embodiment provides an automobile cabin occupant monitoring system based on an OMS camera, including: an OMS camera assembly installed near the interior rearview mirror, the FOV viewing angle range of the OMS camera assembly: horizontal viewing angle 130 degrees, vertical viewing angle 96 degrees. The OMS camera assembly includes: a camera housing 1, a front panel 2, a rear cover 3, an OMS lens 4, an LED light 5 and a PCBA board 6, and the PCBA board 6 also integrates an alarm module, a relic monitoring and search module, an alarm processing module, an RGB-IR module, and an ISP determination module; the camera housing 1 has two front-end mounting cavities and one rear-end mounting cavity, the OMS lens 4 and the LED light 5 are embedded in the two front-end mounting cavities, the front panel 2 is installed at the front end of the camera housing 1, and two avoidance holes are provided on the front panel 2 corresponding to the positions of the OMS lens 4 and the LED light 5; the PCBA board 6 is embedded in the rear-end mounting cavity, the rear cover 3 is installed at the rear end of the camera housing 1 and the PCBA board 6 is sealed in the camera housing 1; the PCBA board includes: an image sensor 7, an image processor 8, an encoder 9, a light driving module 10, and a connector 11 installed on the circuit board , a power supply 12, a memory 13, and a resetter 14. The image sensor 7 is connected to the OMS lens 4 and is used to collect images. The image processor 8 is respectively connected to the image sensor 7, the encoder 9 and the light driving module 10. The image processor 8 is used to receive the image information of the image sensor 7 and process and identify it. The encoder 9 is used to compile and convert the data processed by the image processor 8 into a communication signal. The connector 11 is used to transmit the communication signal processed by the encoder 9 to the outside. The light driving module 10 is used to drive the LED light 5. The power supply 12 is used to power the circuit board and the components on the circuit board. The memory 13 is used to store the processing information of the above components. Among them, the image sensor 7 adopts Sensor-SC2331AT, the image processor 8 adopts ISP-FH8322, and the encoder 9 adopts Serializer-DS90UB935-Q1.

[0029] The RGB-IR module in this embodiment is used to capture RGB color images and IR infrared images through the image sensor 7 and output the RGB color images and infrared images to the image processor 8. The RGB-IR module adopts RGB-IR technology. The RGB-IR technology allows one sensor to capture RGB color images and infrared images at the same time, which provides dual visibility during the day and at night and enhances the image quality under different lighting conditions.

[0030] The ISP determination module in this embodiment is used to identify and determine the brightness of the image collected by the image sensor 7, and automatically switch the RGB color image output or the IR infrared image output of the RGB-IR module according to the brightness of the image, wherein the IR image output is switched to the IR image output when the light is relatively dark, and the RGB color image output is switched to the RGB color image output when the light is sufficient. The ISP determination module adopts the ISP determination technology;

[0031] The alarm module in this embodiment is used to determine whether the driver and passengers have dangerous behaviors, and to issue an alarm reminder when dangerous behaviors occur; dangerous behaviors of passengers include whether they wear seat belts, whether their limbs are stretched out of the window, etc.; wherein the alarm reminder includes one or more of the following methods, including: voice reminder, steering wheel vibration reminder, instrument panel display reminder, seat belt vibration reminder;

[0032] The left-behind monitoring and searching module in this embodiment is used to monitor and search for left-behind items and left-behind creatures in the vehicle, wherein the identified left-behind creatures include pets and children, and the identified left-behind items include handbags, keys, documents, etc.;

[0033] The alarm processing module in this embodiment is used for users to receive biological left-behind alarms and users to cancel biological left-behind alarms; wherein, the user receives the biological left-behind alarm in the car, including autonomously detecting whether there are biological left in the car and detecting the danger after the car owner turns off the engine, locks the car and leaves the car, notifying the car owner of the possible biological left-behind danger alarm by using the APP or CallCenter, and prompting the car owner to return to the car as soon as possible to handle it; wherein, the user cancels the biological left-behind alarm in the following ways: after the car owner receives the biological left-behind alarm, the car owner returns and completes the processing, or the car owner remotely processes it on the App (opens the car window) and clicks to cancel the alarm, or calls the CallCenter to confirm the cancellation of the alarm.

[0034] Example 2

[0035] This embodiment provides a processing method of an automobile cabin occupant monitoring system based on an OMS camera, which specifically includes the following steps:

[0036] Step S1: Using the RGB-IR method on the OMS camera assembly, the ISP determines the brightness of the image, and automatically switches to IR image output in dark conditions to enhance the reality effect. Using RGB-IR technology, a sensor device can capture both RGB color images and IR images at the same time, that is, it has both daytime and nighttime visibility; the RGB-IR method allows a sensor to capture RGB color images and infrared images at the same time, which provides dual visibility during the day and at night, and enhances image quality under different lighting conditions.

[0037] Step S2: using the alarm module to determine whether the driver and passengers have dangerous behaviors, and issuing an alarm reminder when dangerous behaviors occur; wherein, the dangerous behaviors of passengers include whether they wear seat belts, whether their limbs are stretched out of the window, and the alarm reminder includes one or more of the following methods, including: voice reminder, steering wheel vibration reminder, instrument panel display reminder, seat belt vibration reminder;

[0038] Step S3: using the left-behind monitoring and searching module to monitor and search for left-behind items and left-behind creatures in the vehicle, wherein the identified left-behind creatures include pets and children, and the identified left-behind items include handbags, keys, and documents;

[0039] Step S4: using the alarm processing module to notify the user of receiving the biological left behind alarm in the car and the user to cancel the biological left behind alarm; wherein, the user receives the biological left behind alarm in the car, including autonomously detecting whether there are biological left in the car and detecting the danger after the owner turns off the engine, locks the car and leaves the car, notifying the owner of the possible biological left behind danger alarm by using the APP or CallCenter, and prompting the owner to return to the car as soon as possible to handle it; wherein, the user cancels the biological left behind alarm in the following ways: after the owner receives the biological left behind alarm, the owner returns and completes the processing, or the owner processes it remotely on the App (opens the car window) and clicks to cancel the alarm, or calls the CallCenter to confirm the cancellation of the alarm.

[0040] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An automobile cabin occupant monitoring system based on an OMS camera, characterized in that: include: An OMS camera assembly installed near the rearview mirror, the OMS camera assembly comprising: a camera housing, a front panel, a rear cover, an OMS lens, an LED light and a PCBA board, the PCBA board also integrating an alarm module, a leftover monitoring and search module, an alarm processing module, an RGB-IR module, and an ISP determination module; The camera housing has two front-end mounting cavities and one rear-end mounting cavity, the OMS lens and the LED lamp are embedded in the two front-end mounting cavities, the front panel is installed at the front end of the camera housing, and two avoidance holes are provided on the front panel corresponding to the positions of the OMS lens and the LED lamp; The PCBA board is embedded in the rear installation cavity, and the rear cover is installed at the rear end of the camera housing and seals the PCBA board in the camera housing; The PCBA board includes: an image sensor, an image processor, an encoder, a light driving module, a connector, and a power supply installed on the circuit board. The image sensor is connected to the OMS lens and is used to collect images. The image processor is connected to the image sensor, the encoder, and the light driving module respectively. The image processor is used to receive image information from the image sensor and process and identify it. The encoder is used to compile and convert the data processed by the image processor into a communication signal. The connector is used to transmit the communication signal processed by the encoder to the outside. The light driving module is used to drive the LED light. The power supply is used to power the circuit board. The RGB-IR module is used to capture an RGB color image and an IR infrared image through an image sensor and output the RGB color image and the infrared image to an image processor; The ISP determination module is used to identify and determine the brightness of the image collected by the image sensor, and automatically switch the RGB color image output or the IR infrared image output of the RGB-IR module according to the brightness of the image, wherein the IR image output is switched to the IR image output when the light is relatively dark, and the RGB color image output is switched to the RGB color image output when the light is sufficient; The alarm module is used to determine whether the driver and passengers have dangerous behaviors, and issue an alarm reminder when dangerous behaviors occur; the alarm reminder includes one or more of the following methods, including: voice reminder, steering wheel vibration reminder, instrument panel display reminder, seat belt vibration reminder; The left-behind monitoring and searching module is used to monitor and search for left-behind items and left-behind organisms in the vehicle; The alarm processing module is used for users to receive biological left-behind alarms and users to cancel biological left-behind alarms; wherein, the user receives biological left-behind alarms including autonomous detection of whether there are biological left-behind alarms in the car and detecting danger after the car owner turns off the engine, locks the car and leaves the car, notifies the car owner of possible biological left-behind danger alarms through APP or CallCenter, and prompts the car owner to return to the car as soon as possible to handle; wherein, the user cancels biological left-behind alarms including: after the car owner receives the biological left-behind alarm, the car owner returns and completes the processing, or the car owner remotely processes on the App and clicks to cancel the alarm, or calls CallCenter to confirm the cancellation of the alarm.

2. The vehicle cabin occupant monitoring system based on an OMS camera according to claim 1, characterized in that: The FOV viewing angle range of the OMS camera assembly is: 130 degrees horizontally and 96 degrees vertically.

3. The processing method of the vehicle cabin occupant monitoring system based on the OMS camera according to claim 1 is characterized in that: The following steps are involved: Step S1: Using the RGB-IR method on the OMS camera assembly, the ISP determines the brightness of the image, and automatically switches to IR image output in dark conditions to enhance the reality effect. Using RGB-IR technology, a sensor device can capture both RGB color images and IR images, that is, it has both daytime and nighttime visibility; Step S2: using the alarm module to determine whether the driver and passengers have dangerous behaviors, and issuing an alarm reminder when dangerous behaviors occur; wherein, the dangerous behaviors of passengers include whether they wear seat belts, whether their limbs are stretched out of the window, and the alarm reminder includes one or more of the following methods, including: voice reminder, steering wheel vibration reminder, instrument panel display reminder, seat belt vibration reminder; Step S3: using the left-behind monitoring and searching module to monitor and search for left-behind items and left-behind creatures in the vehicle, wherein the identified left-behind creatures include pets and children, and the identified left-behind items include handbags, keys, and documents; Step S4: using the alarm processing module to notify the user of receiving the biological left behind alarm in the car and the user to cancel the biological left behind alarm; wherein, the user receives the biological left behind alarm in the car, including autonomously detecting whether there are biological left in the car and detecting the danger after the car owner turns off the engine, locks the car and leaves the car, notifying the car owner of the possible biological left behind danger alarm by using the APP or CallCenter, and prompting the car owner to return to the car as soon as possible to handle it; wherein, the user cancels the biological left behind alarm in the following ways: after the car owner receives the biological left behind alarm, the car owner returns and completes the processing, or the car owner remotely processes it on the App and clicks to cancel the alarm, or calls the CallCenter to confirm the cancellation of the alarm.

Images