Electronic outside rear-view mirror in-vehicle life auxiliary detection system and method
By combining CMS cameras and controllers, the hardware redundancy problem of in-vehicle vital sign detection systems has been solved, achieving efficient and low-cost vital sign detection and improving the accuracy and safety of detection.
Patent Information
- Application Number
- CN202511204302.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-27
- Publication Date
- 2025-11-21
AI Technical Summary
Existing in-vehicle vital signs detection systems require additional controllers and sensors, resulting in hardware and algorithm redundancy and increased costs.
The system utilizes a CMS camera and controller to detect vital signs inside the vehicle. It achieves this by adjusting the folding angle, compensating for image acquisition, and calculating confidence levels, combined with a Hall sensor and illumination compensation mechanism.
It reduces the number of sensors such as seat pressure sensors and millimeter-wave radar, improving safety and reducing costs, while also improving the accuracy and reliability of detection.
Smart Images
Figure CN120997809A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to an electronic outside rearview mirror in-vehicle life auxiliary detection system and method, belonging to the technical field of in-vehicle vital sign detection. BACKGROUND
[0002] At present, in-vehicle vital sign detection is mainly detected by using an in-vehicle OMS (in-vehicle vital sign detection system), which needs to additionally increase a controller and a camera, or uses a full-vehicle seat pressure sensor and a millimeter wave radar to detect.
[0003] However, with the improvement of vehicle intelligence, whether using an OMS system or using a seat pressure sensor and a millimeter wave radar, additional costs are needed. When there are many controllers and sensors, algorithm and hardware redundancy is easily caused. SUMMARY
[0004] The technical problem to be solved by the present application is to overcome the shortcomings of the prior art, provide an electronic outside rearview mirror in-vehicle life auxiliary detection system and method, which can save the number of in-vehicle sensors, reduce hardware and algorithm redundancy, improve safety, and reduce production cost.
[0005] To solve the above technical problems, the technical scheme of the present application is:
[0006] The present application provides an electronic outside rearview mirror in-vehicle life auxiliary detection method, which comprises the following steps:
[0007] Step S1, controlling the CMS camera to fold according to the vehicle state signal;
[0008] Step S2, detecting whether the folding angle of the CMS camera meets the standard, if yes, locking the folding position and jumping to step S3, if not, fine-tuning the folding angle and then repeating step S2;
[0009] Step S3, collecting in-vehicle images through the CMS camera;
[0010] Step S4, calculating in-vehicle vital sign confidence according to the in-vehicle images;
[0011] Step S5, judging whether the in-vehicle vital sign confidence meets the in-vehicle vital sign confidence threshold, if yes, starting vehicle warning control, if not, the vehicle system enters a dormant state.
[0012] Further, in step S1, the CMS camera is controlled to fold according to the vehicle state signal, which specifically comprises the following steps:
[0013] When the vehicle lock signal is detected, the CMS controller controls the folding motor to drive the CMS camera to fold.
[0014] Further, in the step S2, whether the folding angle of the CMS camera meets the standard is detected, specifically including the following steps:
[0015] The actual folding angle of the CMS camera is detected by a Hall sensor, and whether the error between the actual folding angle and the target folding angle exceeds the error threshold is judged, if yes, the folding angle does not meet the standard, if not, the folding angle meets the standard.
[0016] Further, in the step S3, the in-vehicle image is collected by the CMS camera, specifically including the following steps:
[0017] When the in-vehicle illumination does not reach the illumination threshold or there is a reflection interference, a compensation mechanism for in-vehicle image collection is started.
[0018] Further, the calculation formula of the in-vehicle vital sign confidence is as follows:
[0019]
[0020] Wherein, Confidence is the in-vehicle vital sign confidence;
[0021] K detected is the number of measured human key points;
[0022] K total is the total number of standard human key points;
[0023] α is the weight coefficient of the number of human key points;
[0024] A recognized is the effective life area;
[0025] A standard is the standard human projection area;
[0026] β is the weight coefficient of the effective life area;
[0027] M score is the dynamic feature score;
[0028] γ is the weight coefficient of the dynamic feature score;
[0029] Light Factor is the illumination compensation factor.
[0030] Further, the starting of the vehicle warning control in the step S5 includes controlling the vehicle double flash light to flash, controlling the vehicle to sound the horn and controlling the vehicle APP to push the warning information.
[0031] Another aspect of the present application also provides an electronic outside rearview mirror in-vehicle life auxiliary detection system, which comprises a CMS controller and a CMS camera.
[0032] The CMS camera is used to collect in-vehicle images.
[0033] The CMS controller is used to control the CMS camera to fold according to a vehicle state signal, to determine whether there is a vital sign in the vehicle according to the in-vehicle images collected by the CMS camera, and to take a corresponding control strategy according to the determination result of the in-vehicle vital sign.
[0034] Further, the CMS controller comprises an interface unit, a communication unit, a core computing unit and a storage unit.
[0035] The interface unit is used to provide an external interface of the CMS controller.
[0036] The communication unit is used for communication interaction between the CMS controller and the CMS camera, and communication interaction between the CMS controller and the ego vehicle.
[0037] The core computing unit comprises a real-time computing unit and a performance computing unit.
[0038] The real-time computing unit is used for communication processing between the CMS controller and the ego vehicle and early warning control functions.
[0039] The performance computing unit is used for data fusion, image rendering and image processing on the in-vehicle images collected by the CMS camera.
[0040] The storage unit is used to store software codes, calibration data, data records and internal computing data.
[0041] Further, the interface unit comprises a Fakra interface, a multi-pin interface, an MTD interface and a Debug interface.
[0042] The Fakra interface is used to receive in-vehicle image signals collected by the CMS camera.
[0043] The multi-pin interface is used to receive power and communication signals.
[0044] The MTD interface is used for OTA upgrade.
[0045] The Debug interface is used for program debugging.
[0046] Further, the real-time computing unit is an MCU chip, and the performance computing unit is an SOC chip.
[0047] By adopting the above technical solutions, the present application has the following beneficial effects:
[0048] The CMS camera and the CMS controller are used for in-vehicle life auxiliary detection, the number of related sensors such as seat pressure sensors and millimeter wave radars can be saved, the safety is improved, and the cost is also reduced. When the in-vehicle image is collected through the CMS camera, the compensation mechanism for starting in-vehicle image collection is started according to the illumination scene, the image quality captured by the CMS camera is ensured, and the accuracy of subsequent in-vehicle vital sign detection is improved. BRIEF DESCRIPTION OF DRAWINGS
[0049] Figure 1 The electronic rearview mirror in-vehicle life auxiliary detection method flowchart of the present application;
[0050] Figure 2 The principle block diagram of the electronic rearview mirror in-vehicle life auxiliary detection system of the present application. DETAILED DESCRIPTION
[0051] In order to make the content of the present application more easily understood, the present application will be further described in detail below according to specific embodiments and in combination with the drawings.
[0052] Embodiment one
[0053] As shown in the figure, the present embodiment provides an electronic rearview mirror in-vehicle life auxiliary detection method, which uses a CMS camera and a CMS controller for in-vehicle life auxiliary detection, can save the number of related sensors such as seat pressure sensors and millimeter wave radars, improve safety, and also reduce cost. It includes the following steps: Figure 1 Step S1, control the CMS camera to fold according to the vehicle state signal sent by the body controller. Specifically, when the vehicle lock signal is detected, the CMS controller controls the folding motor to drive the CMS camera to fold, the initial position of the CMS camera folding is the fully unfolded state, and the CMS controller controls the folding motor to drive the CMS camera to rotate 70°±5° horizontally and 30°±3° vertically downward from the fully unfolded state. The CMS camera of the present embodiment can use a fisheye lens, and the field of view angle range can reach 150°, ensuring that the field of view can cover the in-vehicle seat.
[0054]
[0055] Step S2: Detect the actual folding angle of the CMS camera using a Hall sensor. Determine if the error between the actual folding angle and the target folding angle exceeds the error threshold. If so, the folding angle is substandard. In this case, perform PID fine-tuning on the folding angle and repeat step S2. If not, the folding angle meets the standard. Lock the folding position and proceed to step S3. The initial positions of both the actual and target folding angles are the CMS camera in its fully extended state. The target folding angle is the CMS camera rotated horizontally inward by 70°±5° and vertically downward by 30°±3° from its fully extended state.
[0056] Specifically, the Hall sensor is coaxially fixed with the CMS camera's rotating shaft, allowing the Hall sensor to rotate synchronously when the CMS camera folds. The actual folding angle of the CMS camera is then detected by counting Hall pulses, enabling real-time monitoring and ensuring the folding accuracy of the CMS camera. In this embodiment, the error threshold for the folding angle is ±2°. Furthermore, PID fine-tuning refers to optimizing the folding angle of the CMS camera by adjusting the proportional, integral, and derivative parameters.
[0057] Step S3: Acquire three in-vehicle HDR images using the CMS camera at 0.5-second intervals. The image resolution is 1280×720@30fps. When the in-vehicle lighting does not reach the lighting threshold or there is glare interference, the compensation mechanism for in-vehicle image acquisition is activated to ensure the image quality captured by the CMS camera and improve the accuracy of subsequent in-vehicle vital sign confidence calculations.
[0058] Specifically, the interior light threshold is 50 lux. When the external ambient light is weak or the window light transmittance is weak, the interior light may be less than 50 lux. In this case, the 850nm infrared supplement light is activated. The position of the infrared lamp can be referenced from the infrared camera. The reflection of the window glass may cause reflection interference, affecting the image quality captured by the CMS camera. In this case, the polarizing filter is activated to suppress reflection interference. The polarizing filter is set in front of the CMS camera.
[0059] Step S4: Based on the three in-vehicle HDR images, calculate the confidence score for occupants' vital signs. This confidence score is used to determine whether vital signs are present inside the vehicle. The formula for calculating the confidence score for occupants' vital signs is as follows:
[0060]
[0061] Among them, Confidence refers to the confidence level of vital signs inside the vehicle;
[0062] K detected To accurately measure the number of key points on the human body, a multi-scale feature fusion method can be used for extraction.
[0063] K total The total number of key points on the standard human body is a preset value, with 20 points for adults and 15 points for children;
[0064] α is the weighting coefficient for the number of key points on the human body. In this embodiment, the weighting coefficient for the number of key points on the human body is 0.5.
[0065] A recognized The effective life area is calculated by pixel area. The steps for pixel area calculation are as follows: acquiring in-vehicle images, object detection, outputting detection boxes, pixel segmentation, extracting the effective life area, counting the number of pixels, and converting to pixel area.
[0066] A standard The standard human body projection area is preset according to the seat position. The standard human body projection area serves as a benchmark reference value for the human body recognition area and is used to quantitatively evaluate whether the target area (such as the head and torso) captured by the camera belongs to valid vital signs.
[0067] β is the weighting coefficient of the effective life area. In this embodiment, the weighting coefficient of the effective life area is 0.3.
[0068] M score To score dynamic features, the system analyzes continuous image sequences (multi-frame image motion detection) to capture micro-motion features unique to vital signs, such as respiratory micro-movements and blinking frequency, to prevent static misjudgments (such as human model toys).
[0069] γ is the weighting coefficient of the dynamic feature score. In this embodiment, the weighting coefficient of the dynamic feature score is 0.2.
[0070] Light Factor The illumination compensation factor represents the ambient light influence coefficient. Depending on the lighting scenario, different values are assigned to the illumination compensation factor: When the vehicle interior is in an extremely dark environment (such as a tunnel or underground parking garage), the illuminance range is <5 lux. Factor The illuminance ranges from 1.8 to 2; when the vehicle interior is in a dimly lit environment (such as at dusk or in rainy weather), the illuminance ranges from 5 to 50 lux. Factor The illuminance ranges from 1.5 to 1.8; when the vehicle interior is under normal lighting conditions, the illuminance ranges from 50 to 10,000 lux. Factor The value is 1; when the car interior is in a strong light environment (such as glare from car windows on a sunny day), the illuminance range is 10,000 to 50,000 lux. Factor The illuminance is 0.8–0.9; when the vehicle interior is in an extreme glare environment (such as snow or direct sunlight), the illuminance range is >50,000 lux, at which point Light FactorIt ranges from 0.5 to 0.7.
[0071] Step S5: Determine whether the confidence level of the vital signs inside the vehicle has reached the confidence threshold of the vital signs inside the vehicle. If yes, activate the vehicle warning control; otherwise, the vehicle system enters a dormant state.
[0072] Specifically, when two out of three in-vehicle HDR images show a confidence level of occupant vital signs that reaches the in-vehicle vital signs confidence threshold, it indicates the presence of vital signs within the vehicle. At this point, vehicle warning control is activated, including controlling the vehicle's hazard lights to flash, the vehicle's horn to sound, and the vehicle's app to push warning information. Otherwise, it indicates the absence of vital signs within the vehicle, and the vehicle system enters a sleep state. In this embodiment, the in-vehicle vital signs confidence threshold is generally set to 85%, but it can be dynamically optimized based on the environment. For example, in low-light conditions, the in-vehicle vital signs confidence threshold is reduced to 75%, and in rainy or foggy weather, the weight of infrared features is increased.
[0073] Example 2
[0074] like Figure 2 As shown, this embodiment provides an in-vehicle life assistance detection system as described in Embodiment 1, which includes a CMS controller and a CMS camera.
[0075] The CMS camera in this embodiment includes a left CMS camera and a right CMS camera, which are used to collect images inside the vehicle to provide a data basis for subsequent assessment of vital signs inside the vehicle.
[0076] In this embodiment, the CMS controller controls the CMS camera to fold based on vehicle status signals, and determines whether there are signs of life inside the vehicle based on the images captured by the CMS camera. Then, it takes corresponding control strategies based on the determination of signs of life inside the vehicle. When there are no signs of life inside the vehicle, the vehicle system enters a sleep state; when there are signs of life inside the vehicle, the vehicle warning control is activated.
[0077] The CMS controller in this embodiment includes an interface unit, a communication unit, a core computing unit, and a storage unit.
[0078] Specifically, the interface unit provides external interfaces for the CMS controller. The interface unit includes a Fakra interface, a multi-pin interface, an MTD interface, and a debug interface. The Fakra interface receives in-vehicle image signals captured by the CMS camera; the multi-pin interface receives power and communication signals; the MTD interface is used for OTA upgrades; and the debug interface is used for program debugging.
[0079] Specifically, the communication unit is used for communication between the CMS controller and the CMS camera, as well as between the CMS controller and the vehicle. The communication unit mainly includes CAN (FD) communication and Ethernet communication.
[0080] Specifically, the core computing unit includes a real-time computing unit and a performance computing unit. The real-time computing unit can use an MCU chip for communication processing between the CMS controller and the vehicle, as well as for early warning control functions. The performance computing unit can use an ASIL B functional safety level SOC chip for data fusion, image rendering, and image processing of in-vehicle images captured by the CMS camera.
[0081] Specifically, the storage unit includes RAM and ROM, which can use DDR or eMMC, and the capacity can be adjusted according to product requirements. eMMC provides non-volatile storage services, ensuring that data is not lost when power is lost, and is used to store software code, calibration data, and data records; DDR provides high-speed temporary data storage services for storing internal calculation data.
[0082] The working principle of this invention is as follows:
[0083] Based on the vehicle status signal, the system controls the CMS camera to fold; it checks whether the folding angle of the CMS camera meets the standard. If so, it locks the folding position and collects images inside the vehicle through the CMS camera. If not, it fine-tunes the folding angle; it calculates the confidence level of occupants' vital signs based on the images inside the vehicle; it determines whether the confidence level of occupants' vital signs reaches the confidence level threshold. If so, it activates vehicle warning control. If not, the vehicle system enters a dormant state.
[0084] Utilizing CMS cameras and controllers for in-vehicle life assistance detection can reduce the number of related sensors such as seat pressure sensors and millimeter-wave radar, improving safety while also reducing costs. When capturing in-vehicle images via the CMS camera, a compensation mechanism is activated based on the lighting conditions to ensure the image quality captured by the CMS camera and improve the accuracy of subsequent in-vehicle vital sign detection.
[0085] The specific embodiments described above further illustrate the technical problems, technical solutions, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for assisting in the detection of life inside a vehicle using an electronic exterior rearview mirror, characterized in that, It includes the following steps: Step S1: Based on the vehicle status signal, control the CMS camera to fold; Step S2: Check if the folding angle of the CMS camera meets the standard. If yes, lock the folding position and jump to step S3. If no, fine-tune the folding angle and then repeat step S2. Step S3: Acquire in-vehicle images using the CMS camera; Step S4: Calculate the confidence level of vital signs inside the vehicle based on the in-vehicle images; Step S5: Determine whether the confidence level of the vital signs inside the vehicle has reached the confidence threshold of the vital signs inside the vehicle. If yes, activate the vehicle warning control; otherwise, the vehicle system enters a dormant state.
2. The electronic exterior rearview mirror in-vehicle life-assistance detection method according to claim 1, characterized in that, In step S1, the CMS camera is folded according to the vehicle status signal, which specifically includes the following steps: When a vehicle lock signal is detected, the CMS controller controls the folding motor to drive the CMS camera to fold.
3. The electronic exterior rearview mirror in-vehicle life-assistance detection method according to claim 1, characterized in that, In step S2, detecting whether the folding angle of the CMS camera meets the standard specifically includes the following steps: The actual folding angle of the CMS camera is detected by a Hall sensor. It is then determined whether the error between the actual folding angle and the target folding angle exceeds the error threshold. If so, the folding angle does not meet the standard; otherwise, the folding angle meets the standard.
4. The electronic exterior rearview mirror in-vehicle life-assistance detection method according to claim 1, characterized in that, In step S3, the in-vehicle images are captured using the CMS camera, specifically including the following steps: When the interior lighting does not reach the lighting threshold or there is glare interference, the compensation mechanism for interior image acquisition is activated.
5. The electronic exterior rearview mirror in-vehicle life-assistance detection method according to claim 1, characterized in that, The formula for calculating the confidence level of vital signs inside the vehicle is as follows: Among them, Confidence refers to the confidence level of vital signs inside the vehicle; K detected To measure the number of key points on the human body; K total This represents the total number of key points on the standard human body. α is the weighting coefficient for the number of key points on the human body; A recognized The effective life zone area; A standard The standard human body projection area; β is the weighting coefficient for the effective life region area; M score Scoring for dynamic features; γ is the weighting coefficient for dynamic feature scoring; Light Factor This is the illumination compensation factor.
6. The electronic exterior rearview mirror in-vehicle life-assistance detection method according to claim 1, characterized in that, The vehicle warning control in step S5 includes controlling the vehicle's hazard lights to flash, controlling the vehicle's horn to sound, and controlling the vehicle's APP to push warning information.
7. An in-vehicle life-assistance detection system employing the electronic exterior rearview mirror in-vehicle life-assistance detection method as described in any one of claims 1 to 6, characterized in that, It includes a CMS controller and a CMS camera; The CMS camera is used to capture images inside the vehicle. The CMS controller is used to control the CMS camera to fold according to the vehicle status signal, and to determine whether there are signs of life inside the vehicle based on the in-vehicle images captured by the CMS camera. Then, it adopts corresponding control strategies based on the determination of signs of life inside the vehicle.
8. The in-vehicle life assistance detection system according to claim 7, characterized in that, The CMS controller includes an interface unit, a communication unit, a core computing unit, and a storage unit; The interface unit is used to provide an external interface for the CMS controller; The communication unit is used for communication and interaction between the CMS controller and the CMS camera, as well as for communication and interaction between the CMS controller and the vehicle. The core computing unit includes a real-time computing unit and a performance computing unit; The real-time computing unit is used for communication processing and early warning control functions between the CMS controller and the vehicle. The performance computing unit is used to perform data fusion, image rendering, and image processing on the in-vehicle images captured by the CMS camera; The storage unit is used to store software code, calibration data, data records, and internal calculation data.
9. The in-vehicle life assistance detection system according to claim 8, characterized in that, The interface unit includes a Fakra interface, a multi-pin interface, an MTD interface, and a Debug interface; The Fakra interface is used to receive in-vehicle image signals captured by the CMS camera; The multi-pin interface is used to receive power and communication signals; The MTD interface is used for OTA upgrades; The Debug interface is used for program debugging.
10. The in-vehicle life assistance detection system according to claim 8, characterized in that, The real-time computing unit is an MCU chip, and the performance computing unit is a SOC chip.