Multi-sensor body measurement detection and restraint device control

Abstract

The camera measurement module is configured to determine a first measurement of an occupant of a seat within a passenger cabin of a vehicle based on images captured using a camera within the passenger cabin of the vehicle. The radar measurement module is configured to determine a second measurement of the occupant of the seat within the passenger cabin of the vehicle based on radar signals from a radar sensor. The measurement module is configured to determine a third measurement of the occupant of the seat within the passenger cabin based on at least one of: the first measurement of the occupant of the seat; and the second measurement of the occupant of the seat. The actuator control module is configured to selectively actuate an actuator of a restraint device associated with the seat based on the one or more third measurements of the occupant of the seat.

Description

[0001] Cross-references to related applications

[0002] This disclosure is a PCT international application claiming priority to U.S. Patent Application No. 18 / 411,530, filed January 12, 2024. The entire disclosure of the above-cited applications is incorporated herein by reference. Technical Field

[0003] This disclosure relates to a monitoring system and method for passenger compartments in vehicles, and more specifically to body measurement detection and restraint device control using multiple different inputs. Background Technology

[0004] The background description provided herein is for the purpose of presenting the overall context of this disclosure. The work of the currently attributed inventor (to the extent described in this background section) and aspects of the description that may not otherwise be considered prior art at the time of filing are neither expressly nor implicitly acknowledged as prior art to this disclosure.

[0005] Transportation can be used for personal purposes (e.g., by the same person or multiple people) or shared by many different people. Ride-sharing systems allow users to request transportation from their boarding location to their alighting location.

[0006] Vehicles can be human-operated or autonomous (e.g., cars, trucks, buses, bicycles, motorcycles, etc.). Examples of autonomous vehicles include semi-autonomous and fully autonomous vehicles. Human-operated vehicles are controlled by humans using input devices such as steering wheels, accelerator pedals, and brake pedals. In some cases, some vehicles can be remotely controlled. Summary of the Invention

[0007] In one feature, a vehicle occupant classification system includes: a camera measurement module configured to determine one or more first measurements of occupants of seats in the passenger compartment of the vehicle based on images captured by a camera inside the passenger compartment; a radar measurement module configured to determine one or more second measurements of occupants of seats in the passenger compartment of the vehicle based on radar signals from radar sensors inside the passenger compartment of the vehicle; a measurement module configured to determine one or more third measurements of occupants of seats in the passenger compartment based on at least one of: one or more first measurements of the occupants of seats; and one or more second measurements of the occupants of seats; and an actuator control module configured to selectively actuate actuators of restraint devices associated with seats based on one or more third measurements of the occupants of seats.

[0008] In a further feature, the measurement module is configured to determine one or more third measurements of the occupant of the seat based on both one or more first measurements and one or more second measurements.

[0009] In a further feature, the measurement module is configured to determine one or more third measurements of the occupant of the seat based on one or more average values ​​of one or more first measurements and one or more second measurements, respectively.

[0010] In a further feature, the measurement module is configured to further determine one or more third measurements of the occupant based on a first confidence value and a second confidence value from the camera measurement module and the radar measurement module, respectively.

[0011] In a further feature, the measurement module is configured to set one or more third measurements as one or more first measurements when the first confidence value is greater than the second confidence value.

[0012] In a further feature, the measurement module is configured to set one or more third measurements as one or more second measurements when the second confidence value is greater than the first confidence value.

[0013] In a further feature, the measurement module is configured to determine one or more third measurements of the occupant of the seat based on one or more weighted averages of one or more first measurements and one or more second measurements, respectively.

[0014] In a further feature, the measurement module is configured to set one or more first weight values ​​for one or more first measurements based on a first confidence value, and to set one or more second weight values ​​for one or more second measurements based on a second confidence value.

[0015] In a further feature, the measurement module is configured to increase one or more first weight values ​​as the first confidence value increases.

[0016] In a further feature, the measurement module is configured to reduce one or more first weight values ​​as the first confidence value decreases.

[0017] Among the further features, the first, second, and third measurements include the weight of the occupants.

[0018] Among the further features, the first, second, and third measurements include the height of the occupants.

[0019] Among the further features, the camera is a time-of-flight camera.

[0020] Among the further features, the restraint device is an airbag.

[0021] In a further feature, the keypoint module is configured to determine occupant keypoints based on images captured using a camera, wherein the camera measurement module is configured to determine one or more first measurements of the occupant of a seat in the passenger compartment of the vehicle based on at least two of the occupant's keypoints.

[0022] In a further feature, the radar measurement module is configured to determine one or more second measurements of the occupants of seats in the passenger compartment of a vehicle based on the average value of the sum of the energy of radar signals from radar sensors.

[0023] In a further feature, the learning module is configured to selectively adjust one or more parameters of the camera measurement module based on one or more differences between one or more first measurements and one or more second measurements.

[0024] In a further feature, the learning module is configured to selectively adjust one or more parameters of the radar measurement module based on one or more differences between one or more first measurements and one or more second measurements.

[0025] In a further feature, the actuator control module is configured to actuate the actuator of the restraint device associated with the seat based on one or more third measurements of the occupant of the seat when a collision between the vehicle and an object is detected.

[0026] In one feature, an occupant classification method includes: determining one or more first measurements of an occupant of a seat in the passenger compartment of a vehicle based on images captured by a camera inside the passenger compartment; determining one or more second measurements of the occupant of the seat in the passenger compartment of a vehicle based on radar signals from radar sensors inside the passenger compartment; determining one or more third measurements of the occupant of the seat in the passenger compartment based on at least one of the following: one or more first measurements of the seat occupant; and one or more second measurements of the seat occupant; and selectively actuating an actuator of a restraint device associated with the seat based on one or more third measurements of the seat occupant.

[0027] Other applications of this disclosure will become apparent from the detailed description, claims, and drawings. The detailed description and specific examples are intended for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0028] This disclosure will be more fully understood from the specific embodiments and accompanying drawings, wherein:

[0029] Figures 1 to 3 This is a functional block diagram of an example transportation system;

[0030] Figure 4 This is a functional block diagram illustrating a specific implementation of the constraint device control module;

[0031] Figure 5 Includes example illustrations of key points of the human body determined from images;

[0032] Figure 6 Examples include radar sensors and vehicle occupants;

[0033] Figure 7 Example plot of occupant weight as a function of the average of the sum of energy from the Doppler Fast Fourier Transform (FFT);

[0034] Figure 8 include Figure 7 Example table of occupant height, weight, and gender (male or female);

[0035] Figure 9 This is a flowchart depicting an example method for controlling the actuation of a restraint device based on body measurements determined from multiple input devices; and

[0036] Figure 10 This is a flowchart depicting an example method of learning.

[0037] In the accompanying drawings, reference numerals may be used repeatedly to identify similar and / or identical elements. Detailed Implementation

[0038] The driver and other occupants of a vehicle have bodies of varying heights and weights. One or more features of the vehicle can be controlled based on the height and weight of the occupants. For example, the airbag deployment force and / or timing of a seat can be set based on the height and / or weight of the occupants.

[0039] This application relates to a system and method for determining occupant height and weight using inputs from more than two sensors in the passenger compartment, such as images from a camera and inputs from radar sensors. This improves the accuracy and reliability of the determined height and weight measurements. Consequently, characteristic control based on occupant height and / or weight is also improved in terms of accuracy and reliability.

[0040] Figure 1 This is a functional block diagram of an example system of vehicle 100. Vehicle 100 includes a passenger compartment 104. Vehicle 100 also includes one or more propulsion devices, such as one or more electric motors and / or engines. Vehicle 100 may include a transmission and / or other types of gear transmission devices configured to transmit torque from the engine and / or electric motor to one or more wheels of vehicle 100.

[0041] One or more seats 108 are located within the passenger compartment 104. Occupants of the vehicle 100 may sit in the seats 108. While an example of a vehicle 100 including four seats has been provided, this application is also applicable to vehicles with more or fewer seats. The vehicle 100 may be a car, van, truck, sedan, multi-purpose vehicle, boat, aircraft, or other suitable type of land, air, or water-based vehicle. This application is also applicable to vehicles 100 used as public transport vehicles, such as buses, trains, trams, streetcars, or other suitable forms of transport.

[0042] The driver sits in a driver's seat (such as 108-1). The driver can actuate the accelerator pedal to control the acceleration of the vehicle 100. The driver can actuate the brake pedal to control the application of the brakes of the vehicle 100. The driver can actuate the steering wheel to control the steering of the vehicle 100. In various specific embodiments, the vehicle 100 can be an autonomous vehicle or a semi-autonomous vehicle. In autonomous and semi-autonomous vehicles, acceleration, braking, and steering can be controlled at least sometimes by one or more control modules of the vehicle 100.

[0043] Camera 112 is configured to capture images including the eyes, head, face, and upper torso of the user (occupant) of vehicle 100 (such as the driver). Camera 112 has a predetermined field of view (FOV). Example FOV is shown in... Figure 1 The image is illustrated by 116. The driver's seat (e.g., 108-1) is positioned within a predetermined field of view (FOV) of camera 112. One or more seats in seats 108 may also be positioned within the predetermined FOV of camera 112. In various embodiments, camera 112 may be positioned vertically above the steering wheel. Camera 112 may be a time-of-flight (ToF) camera or another suitable type of camera. In various embodiments, camera 112 may include a depth component, such as a red-green-blue depth (RGB-D) camera.

[0044] Although an example of a camera is provided, one camera can capture images of users in the front seats of a vehicle, and another camera can capture images of users in the rear seats of vehicle 100, such as... Figure 2 As shown in the example. Alternatively, a camera can be provided for each seat to capture an image of the user in that seat, such as... Figure 3 As shown in the example.

[0045] In various specific implementations, one or more additional cameras may be included, for example, to detect and locate the user, head, face, eyes, etc. While an example of a passenger seated is provided, this application is also applicable to passengers standing in a vehicle and in other orientations, such as having their feet in a footrest space.

[0046] It also includes one or more other types of sensors. For example, a radar sensor 122 may also be included. The radar sensor 122 may output radar signals toward the driver's seat (e.g., 108-1) and receive signals reflected back to the radar sensor 122. One or more parameters of the driver (e.g., height and weight) may be determined based on the received signals. The radar sensor may be, for example, a 77 GHz radar sensor or have one or more other suitable frequencies (such as between 57 GHz and 64 GHz). In various specific embodiments, for each seat (e.g., as shown in the image), a radar sensor may be used. Figure 3 In the example of camera 112) or each row (e.g., as shown in ... Figure 2 (e.g., camera 112 in the example) or for the entire vehicle (e.g., such as...) Figure 1 The example camera 112) provides a radar sensor.

[0047] The restraint device control module 124 controls the deployment of the restraint devices of the vehicle, such as when a collision of the vehicle is detected. Figure 2 An example restraint device 128 associated with the driver's seat 108 is illustrated. One or more restraint devices may be provided for each seat. Examples of restraint devices include airbags and other types of restraint devices.

[0048] The restraint actuator 132 actuates the restraint device 128 in response to input from the restraint device control module 124. For simplicity, the restraint device 128 and the restraint actuator 132 are not shown in the figures. Figure 1 and Figure 3 The following are shown but will be included: Furthermore, although only restraint device 128 is shown, one or more restraint devices may be included for each seat. Additional restraint devices for two or more seats may also be implemented.

[0049] The restraint control module 124 controls the deployment of the seat's restraints (e.g., force, timing, etc.) based on at least one of the occupant's height and weight. As discussed further below, the body module 136 determines the occupant's height and weight based on images from camera 112 and radar signals from radar sensor 122. Although the body module 136 is illustrated within the restraint control module 124, it may be implemented separately or in a separate module.

[0050] Figure 4 This is a functional block diagram of an example specific implementation of the restraint device control module 124. The body keypoint module 404 receives images from the camera 112. The camera 112 can capture images at a predetermined rate (such as 60 Hz) or at another suitable frequency. The body keypoint module 404 determines the key points of the occupant's body based on the images 112. Figure 5 Examples include key points of a human body determined from an image. In various specific implementations, the image may be preprocessed (e.g., by a preprocessing module) before being used to determine key points. Preprocessing may include, for example, removing data about movement points and / or other preprocessing. Key points may correspond to the positions of the occupant's joints, respectively. The body key point module 404 may, for example, use a key point detection algorithm to determine key points from the image.

[0051] The camera measurement module 408 determines the occupant's body measurements based on key points. For example, the camera measurement module 408 can determine the occupant's height based on key points 2 and 5 of the occupant. Figure 5 The vertical position of the keypoint (such as corresponding to the occupant's shoulder) is used to determine the height. The camera measurement module 408 may determine the height, for example, using one of a lookup table and an equation that correlates keypoint data (e.g., the average vertical height of keypoints 2 and 5) with height.

[0052] Camera measurement module 408 can determine the occupant's weight based on the horizontal positions of key points 2 and 5 and key points 8 and 11. Figure 5 The horizontal position (such as corresponding to the occupant's shoulders and hips) is determined. For example, camera measurement module 408 may determine the occupant's chest width based on the horizontal distance between key points 2 and 5, such as using an equation or lookup table that correlates horizontal distance with chest width. Camera measurement module 408 may determine the occupant's hip width based on the horizontal distance between key points 8 and 11, such as using an equation or lookup table that correlates horizontal distance with hip width.

[0053] The camera measurement module 408 can determine the occupant's weight based on the occupant's chest width and hip width, for example, by using an equation or lookup table that correlates chest width and hip width with weight.

[0054] The feature module 416 receives radar signals from the radar sensor 122. Based on the radar signals, the feature module 416 determines characteristics of the occupant's body in the seat. Examples of features include, for example, the energy of the received signal at various locations. The location from which the signal output by the radar sensor 122 is reflected may have higher energy than a location where the occupant is closer to the radar sensor 122 than a location where the occupant is closer to the radar sensor 122 than other objects. In various specific embodiments, the radar signal may be preprocessed (e.g., by a preprocessing module) before being used to determine the features. Preprocessing may include, for example, removing data about movement points and / or other preprocessing.

[0055] Normalization module 420 can normalize features. For example, normalization module 420 can normalize features (e.g., scale, adjust, etc.) for later fusion of the information with body measurements determined based on images from the camera.

[0056] The radar measurement module 424 determines the occupant's body measurements based on features (with or without normalization). For example, the radar measurement module 424 may determine the occupant's height (e.g., seated height) based on the vertical position of the occupant's shoulders. These could be, for example, the vertical position where energy transitions from a higher to a lower value. The radar measurement module 424 may determine the height, for example, using a lookup table and an equation that correlates radar data (e.g., average vertical height of the shoulders) with height.

[0057] like Figure 6 As illustrated in the example where the radar sensor 122 is mounted vertically above the occupant (e.g., on or near the top of a vehicle), the radar measurement module 424 can determine the height of the occupant's torso (h2) and the height of the occupant's head (h1) based on features. The height of the seat the occupant is sitting on (h3) can be a predetermined value. The radar measurement module 424 can determine the occupant's height based on the sum of the torso height (h2) plus the head height (h1) plus the seat height (h3), or determine the occupant's height as equal to these heights.

[0058] The radar measurement module 424 can determine the occupant's weight based on characteristics (with or without normalization). For example, the radar measurement module 424 can determine the occupant's weight based on the average of the sum of the energy of the signals reflected by the occupant. The radar measurement module 424 can determine the weight, for example, using either an equation relating the average of the sum of energy to the weight or a lookup table.

[0059] Figure 7 Example plot including occupant weight (X-axis) as a function of the average of the sum of energies from multiple Doppler FFTs (Y-axis). Figure 7 Each point in the dataset represents a different occupant. For each adult, multiple datasets are recorded and collected. The average of the sum of energies from the Doppler FFT can be calculated for each record. The average of the averages of each average from each record can be determined. Figure 8 include Figure 7 An example table of the occupant's height, weight, and gender (male or female). The radar measurement module 424 and camera measurement module 408 also determine and output indicators of the confidence level in their respective body measurements. The confidence level indicator can be, for example, a value between 0 and 100, where 0 indicates no confidence and 100 indicates complete confidence. The radar measurement module 424 and camera measurement module 408 can determine their confidence levels separately based on one or more characteristics of the input used to determine their body weight.

[0060] Measurement module 428 determines the occupant's final body measurements (e.g., height, weight, classification (e.g., adult or child), gender (e.g., male or female)) based on body measurements from camera measurement module 408 and radar measurement module 424. For example, measurement module 428 may set the final body measurement based on the average of the body measurements from camera measurement module 408 and radar measurement module 424 (e.g., the average height and the average weight) or determine the final body measurement as equal to the average of the body measurements.

[0061] Measurement module 428 can further determine the final body measurements of the occupant based on confidence levels. For example, measurement module 428 can set the final body measurements based on a weighted average of body measurements from camera measurement module 408 and radar measurement module 424 (e.g., average height and average weight) and set weights based on corresponding confidence values. For example, measurement module 428 can set weights to be applied to a corresponding set of body measurements based on confidence values ​​from that measurement module. By way of example only, when the confidence level of camera measurement module 408 increases, measurement module 428 can increase the weights applied to the body measurements of camera measurement module 408, and vice versa. The same applies to radar measurement module 424.

[0062] As another example, measurement module 428 can set the final body measurement to either the body measurement of camera measurement module 408 or radar measurement module 424 based on confidence levels. For instance, when the confidence level of camera measurement module 408 is greater than that of radar measurement module 424, measurement module 428 can set the final body measurement based on the body measurement of camera measurement module 408. When the confidence level of radar measurement module 424 is greater than that of camera measurement module 408, measurement module 428 can set the final body measurement based on the body measurement of radar measurement module 424 or determine the final body measurement to be equal to the body measurement of radar measurement module 424.

[0063] The measurement module 428 can determine the gender of the seat occupant based on the occupant's final body measurements. For example, when the occupant's weight and height exceed a predetermined weight and predetermined height, the measurement module 428 can determine that the occupant is male. In various specific embodiments, gender determination may be omitted or determined in another manner.

[0064] The measurement module 428 can determine whether the occupant of the seat is an adult or a child based on the occupant's final body measurements. For example, when the occupant's weight and height are greater than a predetermined weight and a predetermined height, the measurement module 428 can determine that the occupant is an adult. When at least one of the occupant's weight is less than a predetermined weight and the occupant's height is less than a predetermined height occurs, the measurement module 428 can determine that the occupant is a child. In various specific embodiments, the determination of whether the occupant is an adult or a child may be omitted or determined in another way.

[0065] Measurement module 428 can determine the percentile (e.g., grouping interval) of the occupant's body measurements (e.g., an adult) based on the occupant's final body measurements. The percentile can be set based on whether the occupant is an adult or a child. For example, when height and / or weight are within a first predetermined height and weight range, measurement module 428 can set the occupant's percentile to the 5th percentile (e.g., for an adult). Figure 7 An example of a first predetermined weight range is illustrated. When the height and / or weight are within a second predetermined height and weight range, the measurement module 428 can set the occupant's percentile to the 50th percentile (e.g., for an adult). Figure 7 An example of a second predetermined weight range is illustrated. The second predetermined height and weight range is greater than the first predetermined height and weight range. When the height and / or weight is within a third predetermined height and weight range, the measurement module 428 can set the occupant's percentile to the 95th percentile (e.g., for an adult). Figure 7 The example illustrates a third predetermined weight range. The third predetermined height and weight range are greater than the second predetermined height and weight range.

[0066] When a collision is detected, the actuator control module 432 controls the actuation (e.g., timing, force, etc.) of the actuator 132 of the restraint device 128 based on final body measurements. For example, when the occupant percentile is at the 5th percentile, the actuator control module 432 may actuate the actuator 132 with a first force at a first time after the collision is detected. When the occupant percentile is at the 50th percentile, the actuator control module 432 may actuate the actuator 132 with a second force at a second time after the collision is detected. The second time may be earlier than the first time (after the collision is detected), and the second force may be greater than the first force. When the occupant percentile is at the 95th percentile, the actuator control module 432 may actuate the actuator 132 with a third force at a third time after the collision is detected. The third time may be earlier than the second time (after the collision is detected), and the third force may be greater than the second force. The above examples apply to both adult and child occupants, but for adult occupants, the force and time may be higher and earlier than for child occupants.

[0067] In various specific implementations, the collision module 436 can detect and indicate the presence of a collision condition of the vehicle. For example, the collision module 436 can detect the presence of a collision condition when one or more accelerations of the vehicle (e.g., lateral, longitudinal) are greater than a predetermined acceleration indicating a collision between the vehicle and an object. When the acceleration is less than the corresponding predetermined acceleration, the collision module 436 may not detect a collision condition. While examples of detecting collision conditions exist, this application is also applicable to detecting collision conditions based on one or more other parameters.

[0068] In various specific implementations, the error module 440 can determine the error between the body measurement determined by the camera measurement module 408 and the body measurement determined by the radar measurement module 424. For example, the error module 440 can determine a weight error based on the difference (subtraction) between the occupant weight determined by the camera measurement module 408 and the occupant weight determined by the radar measurement module 424. The error module 440 can determine a height error based on the difference (subtraction) between the occupant height determined by the camera measurement module 408 and the occupant height determined by the radar measurement module 424.

[0069] The learning module 444 can selectively adjust one or more parameters of the camera measurement module 408 based on weight error and / or height error. The learning module 444 can adjust one or more parameters of the camera measurement module 408 to align or adjust to the body measurement of the radar measurement module 424, such as when the confidence level of the radar measurement module 424 is greater than the confidence level of the camera measurement module 408.

[0070] Additionally or alternatively, the learning module 444 may selectively adjust one or more parameters of the radar measurement module 424 based on weight error and / or height error. The learning module 444 may adjust one or more parameters of the radar measurement module 424 to align or adjust to the body measurement of the camera measurement module 408, such as when the confidence level of the camera measurement module 408 is greater than the confidence level of the radar measurement module 424.

[0071] Figure 9 This is a flowchart depicting an example method for determining the occupant's body measurements in the seat and controlling the actuation of the restraint devices associated with the seat based on those measurements.

[0072] Control can begin at 904, where the feature module 416 receives radar input from the radar sensor 122, and the body keypoint module 404 receives images from the camera 112. At 908, preprocessing can be performed, or preprocessing can be performed by the radar sensor 122 and the camera 112 prior to 904.

[0073] At position 912, the body keypoint module 404 determines the keypoints of the occupant of the seat based on the image. These keypoints can be two-dimensional (2D) or three-dimensional (3D) keypoints (positions). The feature module 416 also determines features based on radar input.

[0074] At 916, normalization module 420 normalizes the features, such as for later fusion. At 920, camera measurement module 408 determines the occupant's body measurements based on key points. Radar measurement module 424 determines the occupant's body measurements based on radar features.

[0075] At 924, the measurement module 428 determines the final body measurement based on at least one of the body measurements determined by the camera measurement module 408 and the body measurements determined by the radar measurement module 424, as discussed above.

[0076] At 928, actuator control module 432 can determine whether one or more collision conditions exist. If 928 is false, then at 932, actuator control module 432 does not actuate actuator 132 of restraint device 128, and control can return to 904. If 928 is true, actuator control module 432 actuates actuator 132 based on the final body measurements of the occupant of the seat, and thus actuates restraint device 128 (e.g., deploys the airbag associated with the seat).

[0077] Figure 10 This is a flowchart depicting an example method of learning. Control begins at 1004, where camera measurement module 408 and radar measurement module 424 determine camera body measurements and radar body measurements, respectively, as described above. At 1008, error module 440 determines the error between the camera body measurements and the radar body measurements. At 1012, learning module 444 selectively adjusts at least one of camera measurement module 408 and radar measurement module 424, such as to reduce one or more errors in the error set.

[0078] The foregoing description is merely illustrative in nature and is in no way intended to limit this disclosure, its application, or its use. The broad teachings of this disclosure can be implemented in various forms. Therefore, while this disclosure includes specific examples, its true scope should not be limited thereto, as other modifications will become apparent upon examination of the drawings, specification, and appended claims. It should be understood that one or more steps within the method may be performed in different orders (or concurrently) without altering the principles of this disclosure. Furthermore, although each of the embodiments described above is described as having certain features, any one or more of those features described with respect to any embodiment of this disclosure may be implemented in features of any of the other embodiments and / or in combination with features of any of the other embodiments, even if such combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and substitution of one or more embodiments for each other remains within the scope of this disclosure.

[0079] Various terms are used to describe spatial and functional relationships between elements (e.g., between modules, circuit elements, semiconductor layers, etc.), including “connection,” “joint,” “coupled,” “adjacent,” “next to,” “on top,” “above,” “below,” and “set.” Unless explicitly described as “direct,” when describing a relationship between a first element and a second element in the foregoing disclosure, the relationship can be a direct relationship in which no other intermediate elements exist between the first element and the second element, or it can be an indirect relationship in which one or more intermediate elements exist between the first element and the second element (spatially or functionally). As used herein, at least one of the phrases A, B, and C should be interpreted as using a non-exclusive logical OR to represent logic (A or B or C), and should not be interpreted as representing “at least one of A, at least one of B, and at least one of C.”

[0080] In the accompanying drawings, the direction of the arrows, as indicated by the arrows, generally represents the flow of information of interest (such as data or instructions). For example, when components A and B exchange various types of information, but the information sent from component A to component B is relevant to the illustration, the arrow may point from component A to component B. This unidirectional arrow does not mean that no other information is being sent from component B to component A. Furthermore, for information transmitted from component A to component B, component B may send a request for or confirmation of receipt of that information to component A.

[0081] In this application, the term "module" or "controller" may be replaced by the term "circuit" as defined below. The term "module" may refer to, be a part of, or include the following: application-specific integrated circuit (ASIC); digital, analog, or mixed-signal analog / digital discrete circuit; digital, analog, or mixed-signal analog / digital integrated circuit; combinational logic circuit; field-programmable gate array (FPGA); processor circuitry (shared, dedicated, or grouped) that executes code; memory circuitry (shared, dedicated, or grouped) that stores code executed by the processor circuitry; other suitable hardware components that provide the described functionality; or combinations of some or all of the foregoing, such as in a system-on-a-chip.

[0082] A module may include one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces that connect to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functionality of any given module in this disclosure may be distributed among multiple modules connected via the interface circuits. For example, multiple modules may allow for load balancing. In a further example, a server (also referred to as a remote or cloud) module may perform a function on behalf of a client module.

[0083] As used above, the term "code" can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, data structures, and / or objects. The term "shared processor circuit" encompasses a single processor circuit that executes some or all of the code from multiple modules. The term "group processor circuit" includes processor circuits that, in combination with additional processor circuits, execute some or all of the code from one or more modules. The reference to multiple processor circuits encompasses multiple processor circuits on a discrete die, multiple processor circuits on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above. The term "shared memory circuit" encompasses a single memory circuit that stores some or all of the code from multiple modules. The term "group memory circuit" encompasses memory circuits that, in combination with additional memory, store some or all of the code from one or more modules.

[0084] The term memory circuit is a subset of the term computer-readable medium. As used herein, the term computer-readable medium does not cover transient electrical or electromagnetic signals propagating through a medium (such as on a carrier wave); therefore, the term computer-readable medium can be considered tangible and non-transient. Non-limiting examples of non-transient tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital magnetic tape or hard disk drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0085] The apparatus and methods described in this application can be implemented, in part or in whole, by a special-purpose computer created by configuring a general-purpose computer to perform one or more specific functions embodied in a computer program. The function blocks, flowchart components, and other elements described above serve as software specifications that can be routinely translated into computer programs by skilled technicians or programmers.

[0086] A computer program includes processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. A computer program may also include or depend on stored data. A computer program may encompass a basic input / output system (BIOS) that interacts with the hardware of a special-purpose computer, device drivers that interact with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0087] Computer programs may include: (i) descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation); (ii) assembly code; (iii) object code generated from source code by a compiler; (iv) source code for execution by an interpreter; and (v) source code for compilation and execution by a just-in-time (JIT) compiler, etc. As an example only, source code may come from languages ​​including C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, and Java. ® , Fortran, Perl, Pascal, Curl, OCaml, Javascript ® HTML5 (Hypertext Markup Language 5th Edition), Ada, ASP (Dynamic Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash ® Visual Basic ® Lua, MATLAB, SIMULINK, and Python ® It is written using the syntax of the language.

Claims

1. A passenger classification system for a means of transport, the passenger classification system comprising: A camera measurement module configured to determine one or more first measurements of occupants of seats in the passenger compartment of the vehicle based on images captured by a camera in the passenger compartment of the vehicle. A radar measurement module configured to determine one or more second measurements of the occupant of the seat in the passenger compartment of the vehicle based on radar signals from radar sensors in the passenger compartment of the vehicle. A measurement module configured to determine one or more third measurements of the occupant of the seat in the passenger compartment based on at least one of the following: The one or more first measurements of the occupant of the seat; and The one or more second measurements of the occupant of the seat; and An actuator control module is configured to selectively actuate actuators of restraint devices associated with the seat based on one or more third measurements of the occupant of the seat.

2. The occupant classification system of claim 1, wherein the measurement module is configured to determine the one or more third measurements of the occupant of the seat based on both the one or more first measurements and the one or more second measurements.

3. The occupant classification system of claim 2, wherein the measurement module is configured to determine the one or more third measurements of the occupant of the seat based on one or more average values ​​of the one or more first measurements and the one or more second measurements, respectively.

4. The occupant classification system of claim 1, wherein the measurement module is configured to further determine the one or more third measurements of the occupant based on a first confidence value and a second confidence value of the camera measurement module and the radar measurement module, respectively.

5. The occupant classification system of claim 4, wherein the measurement module is configured to set the one or more third measurements as the one or more first measurements when the first confidence value is greater than the second confidence value.

6. The occupant classification system of claim 5, wherein the measurement module is configured to set the one or more third measurements as the one or more second measurements when the second confidence value is greater than the first confidence value.

7. The occupant classification system of claim 4, wherein the measurement module is configured to determine the one or more third measurements of the occupant of the seat based on one or more weighted averages of the one or more first measurements and the one or more second measurements, respectively.

8. The occupant classification system of claim 7, wherein the measurement module is configured to set one or more first weight values ​​of the one or more first measurements based on the first confidence value, and to set one or more second weight values ​​of the one or more second measurements based on the second confidence value.

9. The occupant classification system of claim 8, wherein the measurement module is configured to increase the one or more first weight values ​​as the first confidence value increases.

10. The occupant classification system of claim 9, wherein the measurement module is configured to decrease the one or more first weight values ​​as the first confidence value decreases.

11. The occupant classification system of claim 1, wherein the first measurement, the second measurement, and the third measurement include the weight of the occupant.

12. The occupant classification system of claim 1, wherein the first measurement, the second measurement, and the third measurement include the height of the occupant.

13. The occupant classification system according to claim 1, wherein the camera is a time-of-flight camera.

14. The occupant classification system according to claim 1, wherein the restraint device is an airbag.

15. The occupant classification system of claim 1, further comprising a keypoint module configured to determine keypoints of the occupant based on the image captured using the camera. The camera measurement module is configured to determine one or more first measurements of the occupants of the seats in the passenger compartment of the vehicle based on at least two of the key points of the occupants.

16. The occupant classification system of claim 1, wherein the radar measurement module is configured to determine the one or more second measurements of the occupant of the seat in the passenger compartment of the vehicle based on the average value of the sum of the energy of the radar signals from the radar sensor.

17. The occupant classification system of claim 1, further comprising a learning module configured to selectively adjust one or more parameters of the camera measurement module based on one or more differences between the one or more first measurements and the one or more second measurements.

18. The occupant classification system of claim 1, further comprising a learning module configured to selectively adjust one or more parameters of the radar measurement module based on one or more differences between the one or more first measurements and the one or more second measurements.

19. The occupant classification system of claim 1, wherein the actuator control module is configured to, upon detecting a collision between the vehicle and an object, actuate the actuator of the restraint device associated with the seat based on one or more third measurements of the occupant of the seat.

20. A method for classifying occupants, the method comprising: One or more first measurements are used to determine the occupants of seats in the passenger compartment of the vehicle based on images captured by cameras inside the passenger compartment. One or more second measurements are taken to determine the occupant of the seat in the passenger compartment of the vehicle based on radar signals from radar sensors in the passenger compartment of the vehicle. One or more third measurements of the occupant of the seat in the passenger compartment are determined based on at least one of the following: The one or more first measurements of the occupant of the seat; and The one or more second measurements of the occupant of the seat; as well as The actuator of the restraint device associated with the seat is selectively actuated based on one or more third measurements of the occupant of the seat.

Images