In-vehicle passenger intelligent occupation detection method based on multi-sensor fusion

Abstract

The invention discloses an intelligent occupying detection method for passengers in a vehicle based on multi-sensor fusion. The intelligent occupying detection method comprises the steps that the fault conditions of seat pressure sensors and visual detection sensor assemblies of a main driver and a non-main driver are judged; if no fault occurs, obtaining a main driver occupying state and a passenger attribute according to the detection results of the two; if the main driver seat pressure sensor and the visual detection sensor assembly both break down, the main driver occupying state is obtained according to the fastening state of a main driver safety belt or the state signal of a brake pedal; if one of the seat pressure sensors and the visual detection sensor assemblies of the main driver and the non-main driver breaks down, the occupying states and the passenger attributes of the main driver and the non-main driver are obtained according to the detection results of the sensors which do not break down; and if the seat pressure sensor and the visual detection sensor assembly at the non-main-driving corresponding position fail, the non-main-driving corresponding position does not occupy the space. According to the invention, the visual identification scheme is fused, so that the cockpit occupation identification precision and reliability can be improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle occupant occupancy detection technology, and more specifically, to an intelligent occupant occupancy detection method based on multi-sensor fusion. Background Technology

[0002] Currently, the automotive industry is undergoing a dramatic transformation towards electrification and intelligentization. As the market evolves, users' demands for intelligent features in new energy vehicles are increasing, particularly in intelligent scenarios such as seamless vehicle start, intelligent massage, intelligent dual-zone climate control, and intelligent audio zones. The realization of these features all relies on the detection results of occupant occupancy.

[0003] Currently, the mainstream method for detecting occupancy in the industry involves placing pressure sensors under the seat cushions in the vehicle. When someone is seated, pressure is applied to the sensor, causing a change in its resistance. The controller detects this change in resistance to determine if someone is seated. However, this method has significant drawbacks. It can also misidentify passengers with lighter weights, leading to issues such as a lighter female driver being unable to activate the vehicle or rear passengers being unable to activate functions like smart air conditioning. Furthermore, if the pressure sensors malfunction, they will completely fail to detect occupancy, resulting in a poor user experience, complaints, and ultimately, damage to the automaker's product image.

[0004] Therefore, there is an urgent need for an intelligent occupant occupancy detection method based on multi-sensor fusion. Summary of the Invention

[0005] The purpose of this invention is to provide a method for intelligent occupant occupancy detection in vehicles based on multi-sensor fusion, so as to solve the problems in the prior art.

[0006] This invention provides a method for intelligent occupant occupancy detection in vehicles based on multi-sensor fusion, comprising:

[0007] Step S1: Determine the fault status of the driver's seat pressure sensor, the seat pressure sensor in the corresponding non-driver's seat position, and the vision detection sensor assembly;

[0008] Step S2: If neither the driver's seat pressure sensor nor the vision detection sensor assembly malfunctions, proceed to step S3. If both the driver's seat pressure sensor and the vision detection sensor assembly malfunction, proceed to step S4. If neither the seat pressure sensor nor the vision detection sensor assembly in the non-driver's seat position malfunctions, proceed to step S5. If the seat pressure sensor in the driver's seat position nor the non-driver's seat position malfunctions, and not all vision detection sensor assemblies malfunction, proceed to step S6. If neither the seat pressure sensor in the driver's seat position nor the non-driver's seat position malfunctions, and all vision detection sensor assemblies malfunction, proceed to step S7. If both the seat pressure sensor and the vision detection sensor assembly in the non-driver's seat position malfunction, proceed to step S8.

[0009] Step S3: When the driver opens the car door and sits in the driver's seat, the left front area controller collects the signal from the driver's seat pressure sensor and identifies the driver information through the vision detection sensor component. The left front area controller obtains the driver's seat occupancy status and driver's seat occupant attributes based on the detection results of the driver's seat pressure sensor and the vision detection sensor component.

[0010] Step S4: The left front area controller obtains the driver's seat occupancy status based on the driver's seat belt fastening status or the brake pedal status signal.

[0011] Step S5: When someone occupies the non-driver's seat, the left front area controller obtains the occupancy status and occupant attributes of the corresponding non-driver's seat based on the detection results of the visual detection sensor assembly and the pressure sensor at the corresponding position.

[0012] Step S6: The left front area controller obtains the occupancy status and occupant attributes of the corresponding positions of the driver's seat and non-driver's seat based on the detection results of the visual detection sensor components.

[0013] Step S7: The left front area controller obtains the occupancy status of the corresponding positions of the driver's seat and non-driver's seat based on the detection results of the pressure sensors at the corresponding positions;

[0014] Step S8: The left front area controller determines that there is no occupied space in the corresponding position of the non-driver's seat.

[0015] The in-vehicle occupant intelligent occupancy detection method based on multi-sensor fusion, as described above, preferably involves the following step S3: when the driver opens the door and sits in the driver's seat, the left front area controller collects the signal from the driver's seat pressure sensor and simultaneously identifies the driver information through the visual detection sensor component, provided that neither the driver's seat pressure sensor nor the visual detection sensor component malfunctions. The left front area controller then obtains the driver's seat occupancy status and driver occupant attributes based on the detection results from the driver's seat pressure sensor and the visual detection sensor component, including:

[0016] The driver's seat pressure sensor collects the resistance change of the driver's seat position before and after pressure is applied through electrical signals in order to detect whether a person or heavy object is occupying the seat.

[0017] The driver's seat pressure sensor transmits the detection results to the left front zone controller;

[0018] The integrated braking system sends the brake pedal depress status signal to the left front zone controller via the CAN bus.

[0019] The driver monitoring sensor and the cabin monitoring sensor collect images of the person occupying the driver's seat.

[0020] The driver monitoring sensors and cabin monitoring sensors transmit the collected images to the cockpit domain controller;

[0021] The cockpit domain controller processes the images collected by the driver monitoring sensor and the cabin monitoring sensor to obtain the image recognition results, and then transmits the image recognition results to the left front area controller via Ethernet.

[0022] The left front area controller determines the driver's seat occupancy status based on the detection results of the driver's seat pressure sensor, the image recognition results of the cabin area controller, and the status signal of the brake pedal being pressed.

[0023] The left front area controller determines the attributes of the driver / occupant based on the image recognition results from the cockpit area controller.

[0024] The in-vehicle occupant intelligent occupancy detection method based on multi-sensor fusion, as described above, preferably involves the left front area controller determining the driver's seat occupancy status based on the detection results of the driver's seat pressure sensor, the image recognition results of the cabin domain controller, and the brake pedal depressing status signal, including:

[0025] If any of the following conditions are met, it is determined that the driver's seat is occupied: the cockpit domain controller determines that the driver's seat is occupied, the driver's seat pressure sensor determines that the driver's seat is occupied, or the brake pedal is pressed.

[0026] If any of the following conditions are met, it is determined that the driver's seat is not occupied: the cockpit domain controller determines that the driver's seat is unoccupied, the driver's seat pressure sensor determines that the driver's seat is unoccupied, and the brake pedal is not pressed.

[0027] The in-vehicle occupant intelligent occupancy detection method based on multi-sensor fusion, as described above, preferably includes the following step S4: when both the driver's seat pressure sensor and the visual detection sensor assembly malfunction, the left front area controller determines the driver's seat occupancy status based on the driver's seatbelt fastening status or the brake pedal status signal, including:

[0028] The left front zone controller acquires the seat belt fastening status via electrical signals;

[0029] The integrated braking system sends the brake pedal depress status signal to the left front zone controller via the CAN bus.

[0030] The left front zone controller determines the driver's seat occupancy status based on the driver's seatbelt fastening status and brake pedal status signals.

[0031] The in-vehicle occupant intelligent occupant detection method based on multi-sensor fusion, as described above, preferably involves the left front area controller determining the driver's seat occupancy status based on the driver's seatbelt fastening status and brake pedal status signals, including:

[0032] If any of the following conditions are met, it is determined that the driver's seat is occupied: the pressure sensor is faulty and the vision sensor is malfunctioning, and the driver's seat belt is fastened and the brake pedal is depressed.

[0033] If the pressure sensor malfunctions, the vision detection sensor fails, the driver's seatbelt is not fastened, and the brake pedal is not depressed, then it is determined that the driver's seat is not occupied.

[0034] The in-vehicle occupant intelligent occupancy detection method based on multi-sensor fusion, as described above, preferably, in step S5, when someone occupies the non-driver's seat, the left front area controller obtains the occupancy status and occupant attributes of the non-driver's seat based on the detection results of the visual detection sensor component and the corresponding pressure sensor, provided that neither of the seat pressure sensor nor the visual detection sensor component at the corresponding position malfunctions. This includes:

[0035] The number of seats is pre-stored in the left front area controller to identify the number of seats in the corresponding vehicle model;

[0036] If there is a seat in the corresponding position, the seat pressure sensor in the corresponding position will collect the resistance change before and after the corresponding seat position is pressed through electrical signal to detect whether a person or heavy object occupies the seat.

[0037] The non-driver's seat pressure sensor at the corresponding location sends the detection result to the area controller at the corresponding location;

[0038] The corresponding area controller transmits the detection results from the seat pressure sensor to the left front area controller via Ethernet.

[0039] The driver monitoring sensor and the cabin monitoring sensor collect images of people occupying the corresponding seat positions;

[0040] The driver monitoring sensors and cabin monitoring sensors transmit the collected images to the cockpit domain controller;

[0041] The cockpit domain controller processes the images collected by the driver monitoring sensor and the cabin monitoring sensor to obtain the image recognition results, and then transmits the image recognition results to the left front area controller via Ethernet.

[0042] The left front area controller determines the occupancy status of the corresponding position based on the detection results of the seat pressure sensor at the corresponding position and the image recognition results of the cockpit domain controller.

[0043] The left front area controller determines the occupant attributes at the corresponding location based on the image recognition results from the cockpit domain controller.

[0044] The in-vehicle occupant intelligent occupancy detection method based on multi-sensor fusion, as described above, preferably involves the left front area controller determining the occupancy status of the corresponding position based on the detection results of the seat pressure sensor at the corresponding location and the image recognition results of the cabin domain controller, including:

[0045] If the cockpit domain controller determines that someone is in the seat or the corresponding seat pressure sensor determines that someone is in the seat, then the seat position is considered to be occupied.

[0046] If the cockpit domain controller determines that no one is present and the pressure sensor also determines that no one is present, then it is determined that the seat position is not occupied.

[0047] The in-vehicle occupant intelligent occupancy detection method based on multi-sensor fusion, as described above, preferably involves the following step S6: when the seat pressure sensors at the driver's seat and the non-driver's seat corresponding positions malfunction, but not all visual detection sensor components malfunction, the left front area controller obtains the occupancy status and occupant attributes at the driver's seat and the non-driver's seat corresponding positions based on the detection results of the visual detection sensor components.

[0048] The integrated braking system sends the brake pedal depress status signal to the left front zone controller via the CAN bus.

[0049] At least one of the driver monitoring sensor and the cabin monitoring sensor acquires images of people occupying positions in the driver's seat and non-driver's seat positions;

[0050] At least one of the pilot monitoring sensor and the cabin monitoring sensor transmits the acquired images to the cockpit domain controller;

[0051] The cockpit domain controller processes images acquired by at least one of the driver monitoring sensors and the cabin monitoring sensors to obtain image recognition results, and transmits the image recognition results to the left front area controller via Ethernet.

[0052] The left front area controller determines the driver's seat occupancy status and the non-driver's seat occupancy status based on the image recognition results from the cockpit domain controller and the brake pedal depressing status signal.

[0053] The left front area controller determines the attributes of the driver and non-driver occupants based on the image recognition results from the cockpit domain controller.

[0054] The left front area controller determines the driver's seat occupancy status and the non-driver's seat occupancy status based on the image recognition results from the cockpit domain controller and the brake pedal depressing status signal, including:

[0055] If any of the following conditions are met, it is determined that the driver's seat is occupied: the cockpit domain controller determines that someone is in the driver's seat, or the brake pedal is pressed.

[0056] If the cockpit domain controller determines that the driver's seat is unoccupied and the brake pedal is not pressed, it will determine that the driver's seat is not occupied.

[0057] If the cockpit domain controller determines that there is someone in the non-driver's seat, it will determine that the non-driver's seat is occupied.

[0058] If the cockpit domain controller determines that the non-driver's seat is unoccupied, it will determine that the non-driver's seat is not occupied.

[0059] The in-vehicle occupant intelligent occupancy detection method based on multi-sensor fusion, as described above, preferably involves the following steps: In the case that the seat pressure sensors at the driver's seat and the corresponding non-driver's seat positions are functioning correctly, and all visual detection sensor components are malfunctioning, step S7, where the left front area controller obtains the occupancy status of the driver's seat and the corresponding non-driver's seat positions based on the detection results of the pressure sensors at the corresponding positions, includes:

[0060] The driver's seat pressure sensor collects the resistance change of the driver's seat before and after pressure is applied through electrical signals in order to detect whether there is a person or heavy object occupying the driver's seat.

[0061] The non-driver's seat pressure sensor in the non-driver's seat position collects the resistance change of the seat in the non-driver's seat position before and after being pressed through electrical signals, so as to detect whether there is a person or heavy object occupying the non-driver's seat position.

[0062] The driver's seat pressure sensor transmits the detection results to the left front zone controller;

[0063] The non-driver's seat pressure sensor at the corresponding location sends the detection result to the area controller at the corresponding location;

[0064] The corresponding area controller transmits the detection results from the seat pressure sensor to the left front area controller via Ethernet.

[0065] The integrated braking system sends the brake pedal depress status signal to the left front zone controller via the CAN bus.

[0066] The left front zone controller determines the occupancy status of the driver's seat and the corresponding non-driver's seat based on the detection results of the driver's seat pressure sensor, the non-driver's seat pressure sensor, and the brake pedal depress status signal.

[0067] The in-vehicle occupant intelligent occupancy detection method based on multi-sensor fusion, as described above, preferably involves the left front area controller determining the occupancy status of the driver's seat and the corresponding non-driver's seat based on the detection results of the driver's seat pressure sensor, the non-driver's seat pressure sensor, and the brake pedal depressing status signal. This includes:

[0068] If any of the following conditions are met, it is determined that the driver's seat is occupied: the driver's seat pressure sensor detects that someone is in the driver's seat, or the brake pedal is pressed.

[0069] If the driver's seat pressure sensor determines that the driver's seat is unoccupied and the brake pedal is not pressed, then it is determined that the driver's seat is not occupied.

[0070] If the non-driver's seat pressure sensor at the corresponding position detects that someone is in the corresponding non-driver's seat, then it is determined that the corresponding non-driver's seat is occupied.

[0071] If the non-driver's seat pressure sensor at the corresponding position determines that no one is in the corresponding non-driver's seat, then it is determined that the corresponding non-driver's seat is not occupied.

[0072] This invention provides a multi-sensor fusion-based intelligent occupant occupant detection method for vehicles. It utilizes multiple sensors, including visual and pressure sensors, to comprehensively determine and output the occupant occupant status. When all sensors are functioning normally, the visual sensor's ability to more accurately distinguish between objects and occupant types is leveraged. The main controller processes the information collected by both types of sensors and outputs the final judgment signal, improving the accuracy and reliability of cabin occupant recognition and overcoming the bottleneck of pressure sensors' tendency to misidentify occupants. This provides more accurate and stable occupant condition judgment for intelligent scenarios. The fusion of visual recognition increases the flexibility of vehicle development; when the number of seats changes, visual recognition can effectively cover different areas without requiring additional hardware for occupant detection, reducing overall vehicle costs. Redundancy can be implemented between different types of sensors; if one sensor fails, another can continue to provide occupant occupant detection. The system's location information acquisition capability enables redundant processing for occupancy recognition, providing assurance for upstream applications with functional safety requirements. For example, when the pressure sensor malfunctions, it switches to outputting an occupancy signal from the visual sensor. When the visual sensor malfunctions or is manually turned off or obstructed, the main controller switches to relying solely on the traditional pressure sensor for occupancy recognition, maintaining the original basic level of occupancy recognition. Furthermore, the visual sensor can also differentiate occupants by their attributes, such as whether they are adults, children, or pets, providing more diverse signal outputs for the intelligent occupancy recognition system. This offers richer occupant information expansion functions for intelligent scenarios that rely on occupancy information, such as child safety protection and pet modes, thereby improving the user experience. The driver's side also integrates the processing of input signals from the brake pedal, seatbelt, etc., providing a more stable output for driver's side occupancy recognition and preventing scenarios such as lighter female drivers failing to recognize occupancy, leading to vehicle malfunction during seamless start-up. Attached Figure Description

[0073] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings, wherein:

[0074] Figure 1 A flowchart illustrating an embodiment of the intelligent occupant occupant detection method based on multi-sensor fusion provided by the present invention;

[0075] Figure 2 This is a schematic diagram illustrating the interaction between the left front area controller, the cockpit domain controller, the seat pressure sensor, the driver monitoring sensor, and the cabin monitoring sensor. Detailed Implementation

[0076] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. The descriptions of the exemplary embodiments are merely illustrative and are in no way intended to limit the present disclosure or its application or use. The present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided so that the present disclosure will be thorough and complete, and will fully express the scope of the disclosure to those skilled in the art. It should be noted that, unless specifically stated otherwise, the relative arrangement of components and steps, the composition of materials, numerical expressions, and values ​​set forth in these embodiments should be interpreted as exemplary only and not as limiting.

[0077] The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different parts. Terms such as “including” or “contains” mean that the element preceding the term encompasses the element listed after it, and do not exclude the possibility of encompassing other elements as well. Terms such as “above” and “below” are used only to indicate relative positional relationships; when the absolute position of the described object changes, this relative positional relationship may also change accordingly.

[0078] In this disclosure, when a specific component is described as being located between a first component and a second component, an intermediary component may or may not be present between the specific component and the first or second component. When a specific component is described as connecting to other components, the specific component may be directly connected to the other components without having an intermediary component, or it may not be directly connected to the other components but may have an intermediary component.

[0079] All terms used in this disclosure (including technical or scientific terms) have the same meaning as understood by one of ordinary skill in the art to which this disclosure pertains, unless otherwise specifically defined. It should also be understood that terms defined in a general dictionary, such as a dictionary, should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or highly formalized meaning, unless expressly defined herein.

[0080] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0081] like Figure 1 and Figure 2 As shown, the intelligent occupant occupancy detection method based on multi-sensor fusion provided in this embodiment includes the following steps in actual execution:

[0082] Step S1: Determine the fault status of the driver's seat pressure sensor, the corresponding seat pressure sensor in the non-driver's seat position, and the vision detection sensor assembly.

[0083] The visual inspection sensor assembly includes a driver monitoring sensor (DMS) and a cabin monitoring sensor (CMS).

[0084] Step S2: If neither the driver's seat pressure sensor nor the vision detection sensor assembly malfunctions, proceed to step S3. If both the driver's seat pressure sensor and the vision detection sensor assembly malfunction, proceed to step S4. If neither the seat pressure sensor nor the vision detection sensor assembly in the non-driver's seat position malfunctions, proceed to step S5. If the seat pressure sensors in the driver's seat and the non-driver's seat position malfunction, and not all vision detection sensor assemblies malfunction, proceed to step S6. If neither the seat pressure sensor in the driver's seat nor the non-driver's seat position malfunctions, and all vision detection sensor assemblies malfunction, proceed to step S7. If both the seat pressure sensor and the vision detection sensor assembly in the non-driver's seat position malfunction, proceed to step S8.

[0085] Step S3: When the driver opens the car door and sits in the driver's seat, the left front area controller (VIU0) collects the signal from the driver's seat pressure sensor and identifies the driver information through the vision detection sensor component. The left front area controller obtains the driver's seat occupancy status and driver's seat occupant attributes based on the detection results of the driver's seat pressure sensor and the vision detection sensor component.

[0086] In one embodiment of the intelligent occupant occupancy detection method based on multi-sensor fusion of the present invention, step S3 may specifically include:

[0087] Step S31: The driver's seat pressure sensor collects the resistance change of the driver's seat position before and after being pressed by an electrical signal in order to detect whether a person or heavy object is occupying the seat.

[0088] Step S32: The driver's seat pressure sensor transmits the detection results to the left front zone controller.

[0089] Step S33: The integrated braking system sends the brake pedal depressing status signal to the left front zone controller via the CAN bus.

[0090] Step S34: The driver monitoring sensor and the cabin monitoring sensor collect images of the person occupying the driver's seat.

[0091] Step S35: The pilot monitoring sensor and the cabin monitoring sensor transmit the acquired images to the cockpit domain controller (CDC).

[0092] Step S36: The cockpit domain controller processes the images collected by the driver monitoring sensor and the cabin monitoring sensor to obtain the image recognition result, and transmits the image recognition result to the left front area controller via Ethernet.

[0093] The image processing of the cockpit domain controller includes: modeling using at least 50% facial images and 50% body images to collect rules for identifying people, and incorporating smoothing processing to solve the problems of filtering out faces floating outside the window and fixing the number and position of people inside the cabin. The detailed processing of the cockpit domain controller in this invention can be referred to the prior art, and will not be repeated here.

[0094] Step S37: The left front area controller determines the driver's seat occupancy status based on the detection results of the driver's seat pressure sensor, the image recognition results of the cabin area controller, and the brake pedal depressing status signal.

[0095] Specifically, the driver's seat is determined to be occupied if any of the following conditions are met: the cockpit domain controller determines that the driver's seat is occupied, the driver's seat pressure sensor determines that the driver's seat is occupied, or the brake pedal is pressed.

[0096] If any of the following conditions are met, it is determined that the driver's seat is not occupied: the cockpit domain controller determines that the driver's seat is unoccupied, the driver's seat pressure sensor determines that the driver's seat is unoccupied, and the brake pedal is not pressed.

[0097] Furthermore, in one embodiment of the present invention, the occupancy recognition processing result can be uploaded to the car manufacturer's cloud server and forwarded to the mobile APP to provide richer in-vehicle information.

[0098] Step S38: The left front area controller determines the attributes of the driver / occupant based on the image recognition results from the cockpit domain controller.

[0099] In this invention, the occupant attribute determination strategy is as follows: When the visual detection sensor component is working normally, the left front area controller determines that a certain position is occupied based on the input information from the visual detection sensor component. Then, based on the occupant image processing results collected by the visual detection sensor component, it determines whether the occupant is an adult, child, or pet, and encapsulates the occupant attribute information in an Ethernet packet for use by other functions. In one embodiment of this invention, a reinforcement learning training model can be used to make facial recognition more accurate, capable of distinguishing specific genders and multiple age groups, thereby providing richer occupant attribute information for use in intelligent scenarios.

[0100] Step S4: The left front area controller obtains the driver's seat occupancy status based on the driver's seat belt fastening status or the brake pedal status signal.

[0101] In one embodiment of the intelligent occupant occupancy detection method based on multi-sensor fusion of the present invention, step S4 may specifically include:

[0102] Step S41: The left front area controller acquires the seat belt fastening status through electrical signals.

[0103] Step S42: The integrated braking system sends the brake pedal depress status signal to the left front zone controller via the CAN bus.

[0104] Step S43: The left front area controller determines the driver's seat occupancy status based on the driver's seatbelt fastening status and brake pedal status signals.

[0105] Specifically, if any of the following conditions are met, it is determined that the driver's seat is occupied: the pressure sensor is faulty and the vision sensor is malfunctioning, and the driver's seat belt is fastened and the brake pedal is depressed.

[0106] If the pressure sensor malfunctions, the vision detection sensor fails, the driver's seatbelt is not fastened, and the brake pedal is not depressed, then it is determined that the driver's seat is not occupied.

[0107] Step S5: When someone occupies the non-driver's seat, the left front area controller obtains the occupancy status and occupant attributes of the corresponding non-driver's seat based on the detection results of the visual detection sensor component and the pressure sensor at the corresponding position.

[0108] In one embodiment of the intelligent occupant occupancy detection method based on multi-sensor fusion of the present invention, step S5 may specifically include:

[0109] Step S51: Pre-store the configuration word for the number of seats in the left front area controller to identify the number of seats for the corresponding vehicle model.

[0110] Step S52: If there is a seat at the corresponding position, the seat pressure sensor at the corresponding position collects the resistance change before and after the corresponding seat position is pressed through an electrical signal to detect whether a person or heavy object occupies the seat.

[0111] Different car models have different numbers of seats, such as four-seat executive versions, five-seat and six-seat models. This invention takes into account the scalability of rear seat occupancy recognition, and stores the seat number configuration word in the left front area controller to identify the number of seats in different car models. The configuration word representing the number of seats is used as a prerequisite for seat occupancy recognition other than the driver's seat. When a seat exists in a certain position, the occupancy logic judgment of the corresponding position is made according to the seat pressure sensor input information of the corresponding position.

[0112] Step S53: The non-driver's seat pressure sensor at the corresponding location sends the detection result to the area controller at the corresponding location.

[0113] like Figure 2 As shown, the corresponding area controllers are the front left area controller, the front right area controller (VIU1), the rear left area controller (VIU2), and the rear right area controller (VIU3).

[0114] Step S54: The area controller at the corresponding location transmits the detection results of the seat pressure sensor to the left front area controller via Ethernet.

[0115] Step S55: The driver monitoring sensor and the cabin monitoring sensor collect images of the person occupying the corresponding seat position.

[0116] Step S56: The pilot monitoring sensor and the cabin monitoring sensor transmit the collected images to the cockpit domain controller.

[0117] Step S57: The cockpit domain controller processes the images collected by the driver monitoring sensor and the cabin monitoring sensor to obtain the image recognition result, and transmits the image recognition result to the left front area controller via Ethernet.

[0118] Step S58: The left front area controller determines the occupancy status of the corresponding position based on the detection results of the seat pressure sensor at the corresponding position and the image recognition results of the cockpit domain controller.

[0119] Specifically, if the cockpit domain controller determines that someone is there or the seat pressure sensor at the corresponding location determines that someone is there, then it is determined that the seat position is occupied.

[0120] If the cockpit domain controller determines that no one is present and the pressure sensor also determines that no one is present, then it is determined that the seat position is not occupied.

[0121] Step S59: The left front area controller determines the occupant attributes at the corresponding position based on the image recognition results from the cockpit domain controller.

[0122] Step S6: The left front area controller obtains the occupancy status and occupant attributes of the corresponding positions of the driver's seat and non-driver's seat based on the detection results of the vision detection sensor components.

[0123] In one embodiment of the intelligent occupant occupancy detection method based on multi-sensor fusion of the present invention, step S6 may specifically include:

[0124] Step S61: The integrated braking system sends the brake pedal depressing status signal to the left front zone controller via the CAN bus.

[0125] Step S62: At least one of the driver monitoring sensor and the cabin monitoring sensor acquires images of people occupying positions in the driver's seat and non-driver's seat positions.

[0126] Step S63: At least one of the pilot monitoring sensor and the cabin monitoring sensor transmits the acquired images to the cockpit domain controller.

[0127] Step S64: The cockpit domain controller processes the images collected by at least one of the driver monitoring sensor and the cabin monitoring sensor to obtain the image recognition result, and transmits the image recognition result to the left front area controller via Ethernet.

[0128] Step S65: The left front area controller determines the driver's seat occupancy status and the non-driver's seat occupancy status based on the image recognition results of the cockpit domain controller and the brake pedal depressing status signal.

[0129] Specifically, if any of the following conditions are met, it is determined that the driver's seat is occupied: the cockpit domain controller determines that someone is in the driver's seat, or the brake pedal is pressed.

[0130] If the cockpit domain controller determines that the driver's seat is unoccupied and the brake pedal is not pressed, it will determine that the driver's seat is not occupied.

[0131] If the cockpit domain controller determines that there is someone in the non-driver's seat, it will determine that the non-driver's seat is occupied.

[0132] If the cockpit domain controller determines that the non-driver's seat is unoccupied, it will determine that the non-driver's seat is not occupied.

[0133] Step S66: The left front area controller determines the attributes of the driver and non-driver based on the image recognition results of the cockpit domain controller.

[0134] Step S7: The left front area controller obtains the occupancy status of the corresponding positions of the driver's seat and non-driver's seat based on the detection results of the pressure sensors at the corresponding positions.

[0135] In one embodiment of the intelligent occupant occupancy detection method based on multi-sensor fusion of the present invention, step S7 may specifically include:

[0136] Step S71: The driver's seat pressure sensor collects the resistance change of the driver's seat position before and after pressure is applied through electrical signals to detect whether there is a person or heavy object occupying the driver's seat position.

[0137] Step S72: The non-driver's seat pressure sensor in the non-driver's seat position collects the resistance change of the seat in the non-driver's seat position before and after being pressed by an electrical signal, so as to detect whether there is a person or heavy object occupying the non-driver's seat position.

[0138] Step S73: The driver's seat pressure sensor transmits the detection results to the left front zone controller.

[0139] Step S74: The non-driver's seat pressure sensor at the corresponding location sends the detection result to the area controller at the corresponding location.

[0140] Step S75: The area controller at the corresponding location transmits the detection results of the seat pressure sensor to the left front area controller via Ethernet.

[0141] Step S76: The Integrated Braking System (IBS) sends the brake pedal depress status signal to the left front zone controller via the CAN bus.

[0142] Step S77: The left front area controller determines the occupancy status of the driver's seat and the corresponding non-driver's seat based on the detection results of the driver's seat pressure sensor, the non-driver's seat pressure sensor, and the brake pedal depressing status signal.

[0143] Specifically, if any of the following conditions are met, it is determined that the driver's seat is occupied: the driver's seat pressure sensor detects that someone is in the driver's seat, or the brake pedal is pressed.

[0144] If the driver's seat pressure sensor determines that the driver's seat is unoccupied and the brake pedal is not pressed, then it is determined that the driver's seat is not occupied.

[0145] If the non-driver's seat pressure sensor at the corresponding position detects that someone is in the corresponding non-driver's seat, then it is determined that the corresponding non-driver's seat is occupied.

[0146] If the non-driver's seat pressure sensor at the corresponding position determines that no one is in the corresponding non-driver's seat, then it is determined that the corresponding non-driver's seat is not occupied.

[0147] When the seat pressure sensor malfunctions, the left front area controller switches to the recognition result of the vision detection sensor assembly. When the vision detection sensor assembly malfunctions, is turned off, or is obstructed, the left front area controller switches to the recognition result of the seat pressure sensor.

[0148] Step S8: The left front area controller determines that there is no occupied space in the corresponding position of the non-driver's seat.

[0149] In summary, the driver's seat occupancy logic is as follows: if any of the following conditions are met, the left front zone controller (VIU0) determines that the driver's seat is occupied:

[0150] A1. The visual sensor determines that someone is in the driver's seat, or the driver's seat pressure sensor determines that someone is in the driver's seat;

[0151] A2. A single visual or pressure sensor fails, and the unfailed sensors determine that someone is in the driver's seat;

[0152] A3. The pressure sensor is faulty, the vision sensor is malfunctioning, and the driver's seatbelt is fastened.

[0153] A4. The brake pedal was pressed.

[0154] The left front zone controller (VIU0) determines that the driver's seat is not occupied if any of the following conditions are met:

[0155] B1. The visual sensor determines that the driver's seat is unoccupied, and the driver's seat pressure sensor also determines that the driver's seat is unoccupied and the brake pedal is not pressed.

[0156] B2. A single visual or pressure sensor fails, and the non-failed sensor determines that the driver is not in the driver's seat and the brake pedal is not pressed.

[0157] B3. The pressure sensor is faulty, the vision sensor is malfunctioning, the driver's seatbelt is not fastened, and the brake pedal is not depressed.

[0158] The occupancy detection logic for all other non-driver's seat positions is the same, as follows:

[0159] If any of the following conditions are met, VIU0 determines that the position is occupied:

[0160] C1. A visual sensor or a pressure sensor detects the presence of someone;

[0161] C2. A single visual or pressure sensor fails, and the unfailed sensor determines that someone is present.

[0162] If any of the following conditions are met, VIU0 determines that there is no placeholder at that position:

[0163] D1. The visual sensor determines that no one is present, and the pressure sensor also determines that no one is present.

[0164] D2. A single visual or pressure sensor fails, and the unfailed sensors determine that no one is present.

[0165] D3, vision, and pressure sensors all failed.

[0166] This invention utilizes existing components in new energy vehicles, such as Driver Monitoring Sensor (DMS), Visual Inspection Sensor (CMS), and seat pressure sensors, to achieve comprehensive identification and processing of occupant occupancy. For the driver's seat, signals from the brake pedal and seatbelt supplement the identification criteria, improving the success rate of driver occupancy recognition and ensuring normal vehicle startup even in the event of sensor failure. Furthermore, the image processing capabilities of visual sensors like the CMS are used to identify passenger attributes, differentiate occupant types, and provide richer personnel information.

[0167] The intelligent occupant occupancy detection method based on multi-sensor fusion provided in this invention utilizes multiple sensors, including vision and pressure sensors, to comprehensively judge and output the occupant occupancy status in new energy vehicles. When all sensors are working normally, the vision sensor's ability to more accurately distinguish between object or occupant types is leveraged. The main controller comprehensively processes the information collected by the two types of sensors and outputs the final judgment signal, improving the accuracy and reliability of cabin occupancy recognition and overcoming the bottleneck of pressure sensor's tendency to misidentify. This provides a more accurate and stable occupancy condition judgment for intelligent scenarios. The fusion of vision recognition increases the flexibility of vehicle development. When the number of seats changes, vision recognition can better cover different areas without requiring additional hardware configuration to identify occupancy, reducing the overall vehicle cost. Redundancy can be implemented between different types of sensors, so that if one sensor fails, another sensor can continue to provide information. The system's occupancy information collection capability enables redundant processing for occupancy recognition, providing assurance for upstream applications requiring functional safety. For example, when the pressure sensor malfunctions, it switches to outputting an occupancy signal from the visual sensor. When the visual sensor malfunctions or is manually turned off or obstructed, the main controller switches to relying solely on the traditional pressure sensor for occupancy recognition, maintaining the original basic level of occupancy recognition. Furthermore, the visual sensor can differentiate occupants by their attributes, such as whether they are adults, children, or pets, providing more diverse signal outputs for the intelligent occupancy recognition system. This offers richer occupant information extensions for intelligent scenarios relying on occupancy information, such as child safety protection and pet modes, thereby enhancing the user experience. The driver's side also integrates the processing of input signals from the brake pedal and seatbelt, providing a more stable output for driver's occupancy recognition and preventing scenarios such as lighter female drivers failing to recognize occupancy, leading to vehicle malfunction during seamless start-up.

[0168] The embodiments of this disclosure have now been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein based on the above description.

[0169] While specific embodiments of this disclosure have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments or equivalent substitutions can be made to some technical features without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for intelligent occupant occupancy detection in vehicles based on multi-sensor fusion, characterized in that, include: Step S1: Determine the fault status of the driver's seat pressure sensor, the seat pressure sensor in the corresponding non-driver's seat position, and the vision detection sensor assembly; Step S2: If neither the driver's seat pressure sensor nor the vision detection sensor assembly malfunctions, proceed to step S3. If both the driver's seat pressure sensor and the vision detection sensor assembly malfunction, proceed to step S4. If neither the seat pressure sensor nor the vision detection sensor assembly in the non-driver's seat position malfunctions, proceed to step S5. If the seat pressure sensor in the driver's seat position nor the non-driver's seat position malfunctions, and not all vision detection sensor assemblies malfunction, proceed to step S6. If neither the seat pressure sensor in the driver's seat position nor the non-driver's seat position malfunctions, and all vision detection sensor assemblies malfunction, proceed to step S7. If both the seat pressure sensor and the vision detection sensor assembly in the non-driver's seat position malfunction, proceed to step S8. Step S3: When the driver opens the car door and sits in the driver's seat, the left front area controller collects the signal from the driver's seat pressure sensor and identifies the driver's information through the visual detection sensor component. The left front zone controller obtains the driver's seat occupancy status and driver occupant attributes based on the detection results of the driver's seat pressure sensor and vision detection sensor components; Step S4: The left front area controller obtains the driver's seat occupancy status based on the driver's seat belt fastening status or the brake pedal status signal. Step S5: When someone occupies the non-driver's seat, the left front area controller obtains the occupancy status and occupant attributes of the corresponding non-driver's seat based on the detection results of the visual detection sensor assembly and the pressure sensor at the corresponding position. Step S6: The left front area controller obtains the occupancy status and occupant attributes of the corresponding positions of the driver's seat and non-driver's seat based on the detection results of the visual detection sensor components. Step S7: The left front area controller obtains the occupancy status of the corresponding positions of the driver's seat and non-driver's seat based on the detection results of the pressure sensors at the corresponding positions; Step S8: The left front area controller determines that there is no occupied space in the corresponding position of the non-driver's seat.

2. The intelligent occupant occupancy detection method based on multi-sensor fusion according to claim 1, characterized in that, If neither the driver's seat pressure sensor nor the vision detection sensor assembly malfunctions, in step S3, when the driver opens the car door and sits in the driver's seat, the left front area controller collects the signal from the driver's seat pressure sensor and simultaneously identifies the driver's information through the vision detection sensor assembly. The left front zone controller obtains the driver's seat occupancy status and driver occupant attributes based on the detection results from the driver's seat pressure sensor and vision detection sensor assembly, including: The driver's seat pressure sensor collects the resistance change of the driver's seat position before and after pressure is applied through electrical signals in order to detect whether a person or heavy object is occupying the seat. The driver's seat pressure sensor transmits the detection results to the left front zone controller; The integrated braking system sends the brake pedal depress status signal to the left front zone controller via the CAN bus. The driver monitoring sensor and the cabin monitoring sensor collect images of the person occupying the driver's seat. The driver monitoring sensors and cabin monitoring sensors transmit the collected images to the cockpit domain controller; The cockpit domain controller processes the images collected by the driver monitoring sensor and the cabin monitoring sensor to obtain the image recognition results, and then transmits the image recognition results to the left front area controller via Ethernet. The left front area controller determines the driver's seat occupancy status based on the detection results of the driver's seat pressure sensor, the image recognition results of the cabin area controller, and the status signal of the brake pedal being pressed. The left front area controller determines the attributes of the driver / occupant based on the image recognition results from the cockpit area controller.

3. The intelligent occupant occupancy detection method based on multi-sensor fusion according to claim 2, characterized in that, The left front area controller determines the driver's seat occupancy status based on the detection results from the driver's seat pressure sensor, the image recognition results from the cockpit area controller, and the brake pedal depress status signal, including: If any of the following conditions are met, it is determined that the driver's seat is occupied: the cockpit domain controller determines that the driver's seat is occupied, the driver's seat pressure sensor determines that the driver's seat is occupied, or the brake pedal is pressed. If any of the following conditions are met, it is determined that the driver's seat is not occupied: the cockpit domain controller determines that the driver's seat is unoccupied, the driver's seat pressure sensor determines that the driver's seat is unoccupied, and the brake pedal is not pressed.

4. The intelligent occupant occupancy detection method based on multi-sensor fusion according to claim 1, characterized in that, In the event that both the driver's seat pressure sensor and the vision detection sensor assembly malfunction, step S4, where the left front area controller determines the driver's seat occupancy status based on the driver's seatbelt fastening status or the brake pedal status signal, includes: The left front zone controller acquires the seat belt fastening status via electrical signals; The integrated braking system sends the brake pedal depress status signal to the left front zone controller via the CAN bus. The left front zone controller determines the driver's seat occupancy status based on the driver's seatbelt fastening status and brake pedal status signals.

5. The intelligent occupant occupancy detection method based on multi-sensor fusion according to claim 4, characterized in that, The left front area controller determines the driver's seat occupancy status based on the driver's seatbelt fastening status and brake pedal status signals, including: If any of the following conditions are met, it is determined that the driver's seat is occupied: the pressure sensor is faulty and the vision sensor is malfunctioning, and the driver's seat belt is fastened and the brake pedal is depressed. If the pressure sensor malfunctions, the vision detection sensor fails, the driver's seatbelt is not fastened, and the brake pedal is not depressed, then it is determined that the driver's seat is not occupied.

6. The intelligent occupant occupancy detection method based on multi-sensor fusion according to claim 1, characterized in that, If neither the seat pressure sensor nor the visual detection sensor assembly in the non-driver's seat position malfunctions, in step S5, when the non-driver's seat is occupied, the left front area controller obtains the occupancy status and occupant attributes of the non-driver's seat position based on the detection results of the visual detection sensor assembly and the corresponding pressure sensor, including: The number of seats is pre-stored in the left front area controller to identify the number of seats in the corresponding vehicle model; If there is a seat in the corresponding position, the seat pressure sensor in the corresponding position will collect the resistance change before and after the corresponding seat position is pressed through electrical signal to detect whether a person or heavy object occupies the seat. The non-driver's seat pressure sensor at the corresponding location sends the detection result to the area controller at the corresponding location; The corresponding area controller transmits the detection results from the seat pressure sensor to the left front area controller via Ethernet. The driver monitoring sensor and the cabin monitoring sensor collect images of people occupying the corresponding seat positions; The driver monitoring sensors and cabin monitoring sensors transmit the collected images to the cockpit domain controller; The cockpit domain controller processes the images collected by the driver monitoring sensor and the cabin monitoring sensor to obtain the image recognition results, and then transmits the image recognition results to the left front area controller via Ethernet. The left front area controller determines the occupancy status of the corresponding position based on the detection results of the seat pressure sensor at the corresponding position and the image recognition results of the cockpit domain controller. The left front area controller determines the occupant attributes at the corresponding location based on the image recognition results from the cockpit domain controller.

7. The intelligent occupant occupancy detection method based on multi-sensor fusion according to claim 6, characterized in that, The left front area controller determines the occupancy status of the corresponding position based on the detection results of the seat pressure sensor at the corresponding position and the image recognition results of the cockpit domain controller, including: If the cockpit domain controller determines that someone is in the seat or the corresponding seat pressure sensor determines that someone is in the seat, then the seat position is considered to be occupied. If the cockpit domain controller determines that no one is present and the pressure sensor also determines that no one is present, then it is determined that the seat position is not occupied.

8. The intelligent occupant occupancy detection method based on multi-sensor fusion according to claim 1, characterized in that, When the seat pressure sensors in the driver's seat and the corresponding non-driver's seat positions malfunction, and not all of the visual detection sensor components malfunction, step S6, where the left front area controller obtains the occupancy status and occupant attributes of the driver's seat and the corresponding non-driver's seat positions based on the detection results of the visual detection sensor components, includes: The integrated braking system sends the brake pedal depress status signal to the left front zone controller via the CAN bus. At least one of the driver monitoring sensor and the cabin monitoring sensor acquires images of people occupying positions in the driver's seat and non-driver's seat positions; At least one of the pilot monitoring sensor and the cabin monitoring sensor transmits the acquired images to the cockpit domain controller; The cockpit domain controller processes images acquired by at least one of the driver monitoring sensors and the cabin monitoring sensors to obtain image recognition results, and transmits the image recognition results to the left front area controller via Ethernet. The left front area controller determines the driver's seat occupancy status and the non-driver's seat occupancy status based on the image recognition results from the cockpit domain controller and the brake pedal depressing status signal. The left front area controller determines the attributes of the driver and non-driver occupants based on the image recognition results from the cockpit domain controller. The left front area controller determines the driver's seat occupancy status and the non-driver's seat occupancy status based on the image recognition results from the cockpit domain controller and the brake pedal depressing status signal, including: If any of the following conditions are met, it is determined that the driver's seat is occupied: the cockpit domain controller determines that someone is in the driver's seat, or the brake pedal is pressed. If the cockpit domain controller determines that the driver's seat is unoccupied and the brake pedal is not pressed, it will determine that the driver's seat is not occupied. If the cockpit domain controller determines that there is someone in the non-driver's seat, it will determine that the non-driver's seat is occupied. If the cockpit domain controller determines that the non-driver's seat is unoccupied, it will determine that the non-driver's seat is not occupied.

9. The intelligent occupant occupancy detection method based on multi-sensor fusion according to claim 1, characterized in that, If the seat pressure sensors at the driver's seat and the corresponding non-driver's seat positions are not faulty, and all visual detection sensor components are faulty, step S7, where the left front area controller determines the occupancy status of the driver's seat and the corresponding non-driver's seat positions based on the detection results of the pressure sensors at the corresponding positions, includes: The driver's seat pressure sensor collects the resistance change of the driver's seat before and after pressure is applied through electrical signals in order to detect whether there is a person or heavy object occupying the driver's seat. The non-driver's seat pressure sensor in the non-driver's seat position collects the resistance change of the seat in the non-driver's seat position before and after being pressed through electrical signals, so as to detect whether there is a person or heavy object occupying the non-driver's seat position. The driver's seat pressure sensor transmits the detection results to the left front zone controller; The non-driver's seat pressure sensor at the corresponding location sends the detection result to the area controller at the corresponding location; The corresponding area controller transmits the detection results from the seat pressure sensor to the left front area controller via Ethernet. The integrated braking system sends the brake pedal depress status signal to the left front zone controller via the CAN bus. The left front zone controller determines the occupancy status of the driver's seat and the corresponding non-driver's seat based on the detection results of the driver's seat pressure sensor, the non-driver's seat pressure sensor, and the brake pedal depress status signal.

10. The intelligent occupant occupancy detection method based on multi-sensor fusion according to claim 9, characterized in that, The left front zone controller determines the occupancy status of the driver's seat and the corresponding non-driver's seat based on the detection results of the driver's seat pressure sensor, the non-driver's seat pressure sensor, and the brake pedal depress status signal. This includes: If any of the following conditions are met, it is determined that the driver's seat is occupied: the driver's seat pressure sensor detects that someone is in the driver's seat, or the brake pedal is pressed. If the driver's seat pressure sensor determines that the driver's seat is unoccupied and the brake pedal is not pressed, then it is determined that the driver's seat is not occupied. If the non-driver's seat pressure sensor at the corresponding position detects that someone is in the corresponding non-driver's seat, then it is determined that the corresponding non-driver's seat is occupied. If the non-driver's seat pressure sensor at the corresponding position determines that no one is in the corresponding non-driver's seat, then it is determined that the corresponding non-driver's seat is not occupied.

Images