Vehicle occupancy detection
By calculating transfer functions and training neural networks using microphones and speakers in the vehicle, the problem of vehicle occupancy detection in the prior art requires additional hardware, and a cost-effective vehicle occupancy detection method is achieved.
Patent Information
- Application Number
- CN202410634903.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-22
- Filing Date
- 2024-05-22
- Publication Date
- 2024-11-22
AI Technical Summary
Existing vehicle occupancy detection systems require additional dedicated hardware, which adds cost and complexity, and may interfere with other systems or have an impact on passengers.
Using existing microphones and speakers in the vehicle, the neural network is trained to predict the seat occupancy state by playing audio signals and calculating transfer functions, reducing the need for dedicated transmitters or sensors.
The function of vehicle occupation detection is realized without increasing hardware costs and does not interfere with other systems, and can be suitable for different types of vehicles.
Smart Images

Figure CN118991663A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to vehicle occupancy detection, and more particularly to vehicle occupancy detection using audio signals. Background Art
[0002] Knowledge about vehicle occupancy is important for various safety mechanisms of the vehicle. For example, vehicle occupancy can be used to adapt collision mitigation systems (such as airbags) or forgotten child reminders. In order for the vehicle to perform these operations, the vehicle must detect the occupancy of passengers in the vehicle seats.
[0003] Various systems for vehicle occupancy detection have been proposed. For example, detection using weight sensors, infrared sensors, ultrasonic waves, and various radio waves such as ultra-wideband keyless entry and frequency modulated continuous wave (FMCW) radar have been proposed. While each of these systems has various advantages for detecting passengers within a vehicle, they all have a common factor, namely, additional dedicated hardware such as sensors installed in the vehicle. Therefore, a vehicle occupancy detection system that does not require additional dedicated hardware would be desirable. Summary of the invention
[0004] A simplified overview of one or more aspects is given below to provide a basic understanding of these aspects. This overview is not an extensive overview of all contemplated aspects, and is neither intended to identify the key or critical elements of all aspects, nor to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to a more detailed description presented later.
[0005] In some aspects, the technology described herein relates to a method for detecting seat occupancy of a vehicle, the vehicle comprising a plurality of seats, one or more microphones, and one or more speakers, comprising: determining a first set of transfer functions between the one or more microphones and the one or more speakers based on a first audio signal played from the one or more speakers when a passenger occupies one or more of the plurality of seats; and applying at least the first set of transfer functions and a second set of transfer functions of the vehicle to a neural network trained on transfer functions of the vehicle type to predict seat occupancy of each of the plurality of seats based on the transfer functions.
[0006] In some aspects, the technology described herein relates to a method of training a neural network to detect vehicle occupancy, the vehicle comprising a plurality of seats, one or more microphones, and one or more speakers, the method comprising: playing an audio signal in the vehicle via the one or more speakers when the plurality of seats are in each of a plurality of occupancy states including an empty vehicle state; recording, by the one or more microphones, respective received audio signals for each of the one or more speakers for each occupancy state; calculating a set of transfer functions between each pair of speakers and microphones for each occupancy state; and training the neural network to predict occupancy of each of the plurality of seats based on a first set of transfer functions for the empty vehicle state and a second set of transfer functions for a current occupancy state.
[0007] In some aspects, the technology described herein relates to an apparatus for detecting seat occupancy of a vehicle, comprising a plurality of seats, one or more microphones, and one or more speakers, comprising: a memory storing computer-executable instructions; and a digital signal processor coupled to the memory and configured to execute the instructions to: determine a first set of transfer functions between the one or more microphones and the one or more speakers based on a first audio signal played from the one or more speakers when a passenger occupies one or more of the plurality of seats; and apply at least the first set of transfer functions and a second set of transfer functions of the vehicle to a neural network trained on transfer functions of the vehicle type to predict seat occupancy of each of the plurality of seats based on the transfer functions.
[0008] To accomplish the above-mentioned and related purposes, one or more aspects include the features hereinafter fully described and particularly pointed out in the claims. The following description and the accompanying drawings set forth in detail certain illustrative features of one or more aspects. However, these features are indicative of only some of the various ways in which the principles of the various aspects may be employed, and the present description is intended to include all such aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] To provide a more complete understanding of the present disclosure and its features and advantages, reference may be made to the following description taken in conjunction with the accompanying drawings, wherein like reference numerals represent like parts, and wherein:
[0010] Figure 1 is a diagram of an example vehicle including an audio processing unit that implements a vehicle occupancy detection component.
[0011] Figure 2 This is a conceptual diagram of the vehicle occupancy detection component.
[0012] Figure 3is a logic flow diagram of an example method for collecting training data for training an artificial neural network.
[0013] Figure 4 is a logic flow chart of an example method for training an artificial neural network.
[0014] Figure 5 is a flow chart of an example method for detecting vehicle occupancy.
[0015] Figure 6 is a flow chart of another example method of training a neural network to detect vehicle occupancy.
[0016] Figure 7 is a flow chart of another example method of detecting vehicle occupancy. DETAILED DESCRIPTION
[0017] Traditional vehicle occupancy detection uses pressure sensors on each seat. Such systems can detect occupants, but can also generate false positives based on other objects placed on the seats. Pressure sensors also add component cost and complexity to the vehicle. Alternative vehicle occupancy detection systems that use infrared, ultrasonic, or radio frequency (including radar) require adding additional transmitters and sensors to the vehicle. There may also be concerns that additional electromagnetic radiation in the vehicle may interfere with other systems and / or have an effect on passengers.
[0018] Passenger vehicles are often equipped with an entertainment system that includes multiple speakers. In addition, a microphone is often included in the vehicle to provide hands-free applications for voice control of certain vehicle systems (such as entertainment, telephone or navigation). Such voice control systems implement acoustic echo cancellation for hands-free applications, where the transfer function between the speaker and the microphone is known to vary with the passenger load (and temperature) of the vehicle.
[0019] In one aspect, the present disclosure provides systems and methods for detecting vehicle occupancy by applying machine learning to transfer functions between vehicle speakers and microphones. The transfer function is calculated based on an audio signal played through a speaker and picked up by one or more microphones in the vehicle. Because the microphone and speaker are standard components that every vehicle is equipped with, no additional acoustic hardware is required. A machine learning model such as a neural network can be trained to predict the occupancy state of each seat in the vehicle based on a set of transfer functions. The machine learning model can be pre-trained for a vehicle type. For example, a machine learning model can be applied to all vehicles of the same model. In some cases, model changes that affect seats (e.g., an optional third row of seats) may use a separate pre-trained model.
[0020] In some embodiments, the transfer function may be affected by characteristics other than vehicle occupancy. For example, temperature is known to affect the transfer function. In addition, individual characteristics of the acoustic components (speakers and microphones and their tolerances) may also affect the transfer function. In some implementations, the machine learning model can take such characteristics into account. For example, the machine learning model can be trained based on a first set of transfer functions measured when the vehicle is empty and a second set of transfer functions measured when the vehicle is occupied. For example, the training set and / or feature vector of the machine learning model may include the difference between the first set of transfer functions and the second set of transfer functions. Therefore, the machine learning model is trained and evaluated on the transfer functions rather than the changes in the transfer functions themselves. In some embodiments, the second set of transfer functions for an empty vehicle is measured shortly before the vehicle is occupied (e.g., when the vehicle is unlocked). In some embodiments, the feature vector may also include the current temperature to take into account the impact of temperature changes.
[0021] In some embodiments, the present disclosure describes a method for training a neural network to detect vehicle occupancy, the vehicle occupancy comprising a plurality of seats, one or more microphones, and one or more speakers. The method comprises playing an audio signal in the vehicle via the one or more speakers when the plurality of seats are in each of a plurality of occupancy states including an empty vehicle state. The method comprises recording, by the one or more microphones, respective received audio signals for each of the one or more speakers for each occupancy state. The method comprises calculating a set of transfer functions between each pair of speakers and microphones for each occupancy state. The method comprises training a neural network to predict the occupancy of each of the plurality of seats based on a first set of transfer functions for the empty vehicle state and a second set of transfer functions for the current occupancy state.
[0022] In one aspect, the present disclosure can provide one or more of the following technical benefits. The technology disclosed herein provides vehicle occupancy detection using a microphone and speaker, and can be performed by the vehicle's digital signal processor, thereby reducing the need for and associated costs of dedicated transmitters or sensors. In addition, the disclosed technology can be used in conjunction with any additional sensors to verify vehicle occupancy. In some embodiments, although the machine learning model is pre-trained for one type of vehicle (e.g., model), the input (e.g., feature vector) is based on recent measurements of empty vehicles to account for changes in component characteristics and environmental factors. Therefore, the same machine learning model can be applied to different vehicles of the same type without separate training.
[0023] Figure 1is a diagram of an example vehicle 100 including an audio processing unit 110 that implements a vehicle occupancy detection component 120. The vehicle 100 can be any type of passenger vehicle, such as an internal combustion engine (ICE) vehicle, an electric vehicle (EV), or a hybrid vehicle. The vehicle 100 includes a plurality of seats 130. The seats 130 can be arranged in one or more rows. For example, seats 130a and 130b can be located in the front row for the driver and front seat passenger, and seats 130c, 130d, and 130e can be located in the second row. The vehicle 100 can include additional seats or rows. In some embodiments, a type of vehicle can be a model with seats (and other cabin structures) in a known configuration. In some embodiments, different seat configuration options (e.g., optional seats or folding seats) can be treated as different types of vehicles, or different machine learning models can be used for each seat configuration.
[0024] The vehicle 100 includes one or more speakers 140. Each speaker 140 may be capable of outputting sound based on a received audio signal. For example, the speaker 140 may be part of an entertainment system of the vehicle 100. For example, as shown, speakers 140a, 140b, 140c, and 140d may be arranged around the seat 130. In some embodiments, a speaker 140e outside the entertainment system may be used for occupancy detection. The speaker 140 is configured to output sound within a frequency band based on a speaker audio signal L. For n speakers, the speaker audio signal may be represented as L1...Ln. For example, the frequency band may be approximately 250Hz–12kHz. The frequency band may be within the range of human hearing. The speaker 140 may not be able to generate ultrasonic signals.
[0025] The vehicle 100 includes one or more microphones 150. For example, the microphones 150 may be arranged in a microphone array. Each microphone 150 may record a separate received audio signal M. The received audio signal may be represented as M1 ...M m In some embodiments, microphones 150 may be arranged in an array, for example, where the microphones are also used for voice control. In some embodiments, microphones 150a and / or 150b may be placed in strategic locations (e.g., behind seat 130) to help detect occupancy. In addition, the received audio signal may be associated with one of the speakers. Thus, the received audio signal may be transmitted in the form of a microphone. m,n In the form of M m,n indicates the signal received from the nth speaker at the mth microphone. When multiple sets of received audio signals are collected, these sets can be in M X,m,n It is expressed in the form of , where X is a set (eg, corresponding to vehicle occupancy conditions).
[0026] In some embodiments, the vehicle includes a thermometer 160. The thermometer 160 can measure the current temperature in the vehicle cabin and provide a temperature signal T to the audio signal processing unit 110. The temperature may affect the transfer function of the audio signal (eg, due to associated changes in air pressure).
[0027] In one aspect, the transfer function can represent the change from the speaker audio signal L output by the speaker 140 to the received audio signal M received by the microphone 150. In some embodiments, a set of transfer functions can include a transfer function for each pair of speakers 140 and microphones 150. In addition, the transfer function can be analyzed over a frequency band. For example, a frequency band of approximately 250 Hz–12 kHz can be sampled at a frequency resolution of 50-150 Hz to generate a transfer function. Other frequency bands within the human hearing range and / or that can be produced by the speaker 140 can be used.
[0028] In one aspect, to provide a robust measurement of the transfer function, the audio signal processing unit 110 may play a speaker audio signal L via the speaker 140 and record a corresponding received audio signal M via the microphone 150. In one aspect, vehicle occupancy detection may be based on the difference in the transfer function when the vehicle is empty and when the vehicle is occupied. Since the transfer function of an empty vehicle may vary between individual vehicles (e.g., based on individual audio components) and based on temperature, the measurement of the empty vehicle transfer function may be performed shortly before the vehicle is loaded. In some embodiments, the unlocking audio signal processing unit 110 may play a speaker audio signal L in response to the vehicle 100 being unlocked to generate a new transfer function for an empty vehicle.
[0029] In some embodiments, the speaker audio signal L is an engineered audio signal designed to span a frequency band. For example, the speaker audio signal L may be a buzzer or chirp with a frequency variation. In some embodiments, the audio signal processing unit 110 may play the speaker audio signal L from each speaker 140 separately. Each individual speaker audio signal L may be a fraction of a second. In some embodiments, the speaker audio signal may be another audio signal output by a vehicle entertainment system. For example, the speaker audio signal may be music or a conversation output by the entertainment system. In the case where the individual speaker audio signals L do not overlap in time, each received audio signal may correspond to only one speaker 140, and the transfer function may be calculated directly from the measured signal. In some embodiments, the speaker audio signal L may not be separated in time. The audio signal processing unit 110 may, for example, decorrelate the speaker audio signal L using different frequencies. For example, each speaker may play a different frequency range so that the received audio signal M is different for each speaker.
[0030] Figure 2 2 is a conceptual diagram of a vehicle occupancy detection component 120. In some implementations, the vehicle occupancy detection component 120 can be implemented on a digital signal processor 202 and an associated memory 204. The memory 204 can store computer executable instructions that define the vehicle occupancy detection component 120. The digital signal processor 202 can execute the computer executable instructions. The vehicle occupancy detection component 120 can include a transfer function calculator 210, a feature vector generator 220, and an artificial neural network 230.
[0031] The transfer function calculator 210 may be configured to calculate a set of transfer functions H based on the speaker audio signal L and the microphone audio signal M. When the speaker audio signal L is output to the speaker 140, the transfer function calculator 210 may obtain the speaker audio signal L from the audio signal processing unit 110. The transfer function calculator 210 may obtain the microphone audio signal M as input from the microphone 150. The transfer function H is a complex-valued function of an audio frequency. For example, the transfer function is described in, for example, U.S. Patent No. 6,683,961. A set of transfer functions may include a transfer function for each pair of the microphone signal M and the speaker signal L. For example, the transfer function H 1,1 is a transfer function between a first speaker (e.g., speaker 140a) and a first microphone 150. The vehicle 100 may include m speakers and n microphones for a total of m×n transfer functions. In one aspect, the transfer function calculator 210 may calculate a first set of transfer functions 212 for an occupancy condition when a passenger is present in the vehicle, and calculate a second set of transfer functions 214 for an empty vehicle occupancy condition.
[0032] The feature vector generator 220 is configured to generate a feature vector F 222 based on the first set of transfer functions 212 and the second set of transfer functions 214. The feature vector generator 220 outputs the feature vector F 222 to the artificial neural network 230. The transfer function calculator 210 can output the first set of transfer functions 212 for the occupied condition. For example, the first set of transfer functions 212 can be measured once the doors have been locked and the vehicle 100 is started. The transfer function calculator 210 can output the second set of transfer functions 214 for the empty vehicle. In some embodiments, the transfer function calculator 210 measures the second set of transfer functions before loading the vehicle 100, for example, when the vehicle 100 is unlocked. In other embodiments, the feature vector generator 220 can store the second set of transfer functions 214 previously generated for the empty vehicle state. In some embodiments, the feature vector generator 220 also receives the temperature T from the thermometer 160. The temperature T can be associated with the first set of transfer functions 212 and / or the second set of transfer functions 214. For example, the interior temperature of the vehicle 100 may vary significantly when the doors are open and passengers are loaded, so the temperature associated with each set of transfer functions may improve predictions by accounting for changes due to temperature changes rather than seat occupancy changes.
[0033] In some embodiments, the eigenvector F 222 includes the first set of transfer functions 212 and the second set of transfer functions 214. For example, the eigenvector F can simply be a cascade of the first set of transfer functions and the second set of transfer functions. In some embodiments, the eigenvector F can include the magnitude differences between corresponding pairs of the first set of transfer functions and the second set of transfer functions at multiple frequencies within the analysis frequency band. For example, the eigenvector generator 220 can sample the transfer functions for each of the multiple frequencies to generate vectors, and then subtract the vectors to determine the magnitude differences. The eigenvector F 222 can optionally include a single temperature, or a temperature associated with each set of transfer functions.
[0034] The artificial neural network 230 is configured to predict a seat occupancy 240 for each of the plurality of seats 130 based on the feature vector F 222. The artificial neural network 230 is trained based on a training set that includes feature vectors that have been labeled with the correct seat occupancy 240. In some embodiments, the seat occupancy 240 is a binary or Boolean value indicating whether the corresponding seat 130 is occupied. For example, 0 may indicate unoccupied and 1 may indicate occupied. In some embodiments, the seat occupancy 240 is an indication of the size of the occupant in the corresponding seat 130. For example, 0 may indicate unoccupied and 1 may indicate occupied by an adult, but a value of 0.25 may indicate an infant or toddler, and a value of 0.75 may indicate a child. In the case of using such a continuous value, the value may correspond to the size of the occupant (e.g., using weight as an approximation). The artificial neural network 230 may output the predicted seat occupancy 240 to one or more systems of the vehicle, such as a collision mitigation system or a child reminder system.
[0035] Figure 3 is a logic flow diagram of an example method 300 for collecting training data for training the artificial neural network 230. In one aspect, the method 300 may be performed by an external computer in communication with the audio signal processing unit 110 including the vehicle occupancy detection component 120. For example, the audio signal processing unit 110 and / or the vehicle occupancy detection component may be placed in a training mode or connected to an external computer to control the training process and store a training database.
[0036] Training data collection begins at box 310 in an empty vehicle, where m microphone signals M0 are recorded. 1,1 …M0 m,n , and the n speakers output audio signals L1 ... Ln. For example, the speaker signal may be a suitably composed chime or jingle. Examples of such chimes or jingles include chirps played by one speaker after another. The recorded microphone signal is added to the training database. Optionally, the current cabin air temperature is associated with the recorded microphone signal.
[0037] Next, at block 320, one or more people of different sizes enter the vehicle at random seating positions of the z vehicle seats. X 1…P X In z, 0 means the seat is empty and 1 means the seat is occupied. The resulting seating pattern is recorded and used to label the training set.
[0038] After the door is closed, at block 330, the speaker 140 plays the signals L1 ... Ln. Preferably, the signals L1 ... Ln are the same as those in block 310. The audio signal processing unit 110 converts the microphone signal M x,1,1 ...MX,m,n Recorded in the training database. Optionally, the cabin air temperature T x is associated with a training data set, which includes loudspeaker signals L1 ... Ln and seat patterns P x 1...P x The microphone signal M is marked with z X11 …M xmn .
[0039] At decision block 340, if training data collection is complete, method 300 proceeds to block 350 where the training database is saved. If training data collection is not complete, blocks 320, 330, and 340 are repeated.
[0040] Figure 4 is a logic flow diagram of an example method 400 for training an artificial neural network 230. The training phase utilizes a state-of-the-art machine learning training algorithm to convert entries of a training database into weights of an artificial neural network 230. For example, the training algorithm may utilize back propagation. The method 400 may be performed by an external computer. For example, the external computer may include hardware suitable for machine learning, such as a neural network processor or a graphics processing unit (GPU), or may be a personal computer, server, or cloud network utilizing general computing resources. The method 400 may be performed on a training database having a training set collected from a specific type of vehicle (e.g., a model), and a trained neural network may be generated for the same type of vehicle. For example, a trained neural network may be loaded onto the vehicle occupancy detection component 120 as an artificial neural network 230 prior to delivery. Alternatively, the trained neural network may be loaded over the air as an update at a service appointment or via the telecommunications system of the vehicle 100.
[0041] At block 410 , an empty vehicle dataset is retrieved from the training database and the transfer function H is transformed into 0,m,n (f) is calculated as a complex-valued function of the audio frequency f. For example, the training phase may use the same algorithm as the transfer function calculator 210 to calculate the transfer function.
[0042] At block 420, a dataset X of one or more occupied seats is retrieved from the training database, and correspondingly from the records M X,1,1 ...MX ,m,n and L1.Ln to calculate the transfer function H X,m,n .
[0043] At block 430, a feature vector F is calculated as an input to a neural network, and the feature vector F is used to train the feature vector F to predict the seating pattern P. x 1..P xz, for example, the components of the eigenvector are H at each frequency f within the analyzed frequency band (e.g., 250 Hz–12 kHz, with a frequency resolution of, for example, 125 Hz). X,m,n (f) and H 0,m,n In some embodiments, the feature vector F optionally includes temperature data.
[0044] At decision block 440, if the training phase is over, the method 400 proceeds to block 450 where the training weights of the artificial neural network 230 are saved for later use in production. Otherwise, the method 400 repeats blocks 420, 430, and 440.
[0045] Figure 5 is a flow chart of an example method 500 for detecting vehicle occupancy. The method 500 may be performed by the audio signal processing unit 110 including the vehicle occupancy detection component 120. For vehicle operation, the trained weights of the artificial neural network 230 are used to predict vehicle occupancy, ie, seat occupancy 240 for each seat.
[0046] At block 510 , before the vehicle door is opened, the audio signal processing unit 110 plays the speaker signals L1 . . . Ln and records the microphone signals M in an empty vehicle. 0,1,1 ...M 0,m,n Block 510 may be similar to block 310 but performed while the vehicle is in use.
[0047] At block 520 , after the vehicle is entered and the door is closed, the audio signal processing unit 110 plays the speaker signals L1 . . . L n And record the microphone signal M 0,1,1 ...M 0,m,n .
[0048] At block 530 , the vehicle occupancy detection component 120 and / or the feature vector generator 220 calculates the feature vector F 222 based on the previously acquired records. Depending on the implementation and training of the artificial neural network 230 , as described in block 430 , the cabin air temperature may be optionally included in the feature vector F 222 .
[0049] At block 540, with the feature vector F 222 as input, the artificial neural network 230 predicts a seat occupancy pattern. The artificial neural network 230 may output a seat occupancy 240 (e.g., P1 ... Pz) for each seat 130. In some embodiments, the seat occupancy 240 may indicate whether it is occupied (P = 0 or 1). In other embodiments, a seat occupancy 240 value between 0 and 1 may indicate a prediction of a small person, child, infant seat, or bag on the seat 130.
[0050] In some embodiments, blocks 520 , 530 , and 540 may be repeated at any time during vehicle operation to confirm seat occupancy patterns.
[0051] Figure 6 is a flow chart of another example method 600 for training a neural network to detect vehicle occupancy. The method 600 can provide an artificial neural network 230 that can be used in a vehicle including multiple seats, one or more microphones, and one or more speakers.
[0052] In box 610, method 600 includes: when the plurality of seats are in each of a plurality of occupancy states including an empty vehicle state, playing an audio signal via the one or more speakers in the vehicle. For example, in some embodiments, under the control of the vehicle occupancy detection component 120 and / or an external computer, the audio signal processing unit 110 plays the speaker audio signals L1...Ln while the plurality of seats 130 are in each of a plurality of occupancy states including an empty vehicle state. In some embodiments, a plurality of occupancy conditions may be randomly selected. In some embodiments, a plurality of occupancy conditions may include each combination of seats for a maximum of z passengers (i.e., one in each seat). In some embodiments, at subbox 612, box 610 optionally includes playing a segment of the audio signal from each speaker 140 separately.
[0053] In block 620, the method 600 includes recording, by the one or more microphones, respective received audio signals for each of the one or more speakers for each occupancy state. For example, in some embodiments, the one or more microphones 150 record respective received audio signals (M ) for each of the n speakers 140 for each occupancy condition X. 1,1 …M m,n ).
[0054] In block 630, the method 600 includes calculating a set of transfer functions between each pair of speakers and microphones for each occupancy state. For example, in some embodiments, the transfer function calculator 210 calculates a set of transfer functions H between each pair of speakers 140 and microphones 150 for each occupancy condition X. 1,1 …H m,n In some embodiments, each set of transfer functions is also associated with a current temperature, and the feature vector also includes the current temperature.
[0055] In block 640, the method 600 includes training a neural network to predict the occupancy of each of the plurality of seats based on the first set of transfer functions for the empty state and the second set of transfer functions for the current occupied state. In some embodiments, for example, the digital signal processor 202 or an external computer may train the artificial neural network 230 to predict the occupancy of each of the plurality of seats 130 based on the first set of transfer functions 212 for the empty state and the second set of transfer functions 214 for the current occupied state. In some embodiments, the neural network is configured to output an estimate of the size of a person occupying each of the plurality of seats.
[0056] In some embodiments, at sub-block 642, block 640 may optionally include generating a plurality of training sets, each training set including a feature vector of magnitude differences between the first set of transfer functions 212 and the second set of transfer functions 214 and a label of the current occupancy condition X. In some embodiments, each set of transfer functions is also associated with a current temperature, and the feature vector also includes the current temperature. In some embodiments, the feature vector includes magnitude differences between corresponding pairs of the first set of transfer functions 212 and the second set of transfer functions 214 at a plurality of frequencies within an analysis frequency band. For example, the frequency band analyzed may be approximately 250 Hz–12 kHz, with a frequency resolution of 50-150 Hz.
[0057] Figure 7 is a flow chart of an example method 700 for detecting vehicle occupancy. The method 700 may be performed by a vehicle 100 including a vehicle occupancy detection component 120. The vehicle 100 also includes a plurality of seats 130, one or more speakers 140, and one or more microphones 150. The vehicle 100 may optionally include a thermometer 160.
[0058] At block 710, the method 700 optionally includes determining a second set of transfer functions between the one or more microphones and the one or more speakers based on a second audio signal played from the one or more speakers when the vehicle is empty. In some embodiments, for example, the audio signal processing unit 110 and / or the transfer function calculator 210 may determine a second set of transfer functions 214 between the one or more microphones 150 and the one or more speakers 140 based on a second audio signal played from the one or more speakers when the vehicle 100 is empty.
[0059] At block 720, the method 700 includes determining a first set of transfer functions between the one or more microphones and the one or more speakers based on the first audio signal played from the one or more speakers when the passenger occupies one or more of the plurality of seats. In one embodiment, for example, the audio signal processing unit 110 and / or the transfer function calculator 210 may determine a first set of transfer functions 212 between the one or more microphones 150 and the one or more speakers 140 based on the first audio signal played from the one or more speakers 140 when the passenger occupies one or more of the plurality of seats 130.
[0060] In some embodiments, at sub-block 722, block 720 (or block 710) optionally includes playing the first audio signal or the second audio signal (L) via one or more speakers 140. For example, the audio signal processing unit 110 may output signals L1 ... Ln to individually play a segment of the first audio signal or the second audio signal from each speaker 140. In some embodiments, at sub-block 726, block 720 (or block 710 or sub-block 722) optionally includes decorrelating the first audio signal or the second audio signal of each speaker. For example, sub-block 726 may be performed when multiple speakers 140 play audio signals simultaneously to generate a separate audio signal for each speaker 140. In some embodiments, the first audio signal may be played in response to the vehicle being loaded, for example, when the vehicle 100 begins to move. In some embodiments, the second audio signal may be played when the vehicle is empty, in response to the vehicle 100 being unlocked. In some embodiments, the first audio signal or the second audio signal is a preconfigured beep across the analyzed frequency band. In some embodiments, the first audio signal is output from an entertainment system, such as music or speech.
[0061] At sub-block 724, block 720 (or block 710) optionally includes recording, by each of the one or more microphones 150, a respective received audio signal (M) for each of the speakers. At sub-block 728, block 720 (or block 710) optionally includes calculating a first set of transfer functions 212 or a second set of transfer functions 214 between each pair of speakers 140 and microphones 150 based on the first audio signal or the second audio signal and the respective received audio signal.
[0062] In block 730, the method 700 includes applying at least the first set of transfer functions and the second set of transfer functions for the vehicle to a neural network trained on the transfer functions for the vehicle type to predict a seat occupancy feature vector for each of the plurality of seats based on the transfer functions. In some embodiments, for example, the audio signal processing unit 110 and / or the feature vector generator 220 applies at least the first set of transfer functions 212 and the second set of transfer functions 214 for the vehicle 100 to an artificial neural network 230 trained on the transfer functions for the vehicle type to predict a seat occupancy 240 for each of the plurality of seats 130 based on the transfer functions.
[0063] In some embodiments, at sub-block 732, block 730 optionally includes calculating a feature vector F 222 based on at least the first set of transfer functions 212 and the second set of transfer functions 214 of the vehicle 100. In some embodiments, the feature vector includes a current temperature. In some embodiments, sub-block 732 includes determining a magnitude difference between corresponding pairs of the first set of transfer functions 212 and the second set of transfer functions 214 at a plurality of frequencies within an analysis frequency band. For example, the frequency band analyzed may be approximately 250 Hz–12 kHz with a frequency resolution of 50-150 Hz. In some embodiments, at sub-block 734, block 730 optionally includes inputting the feature vector F 222 into an artificial neural network 230 to predict a seat occupancy 240 for each of the plurality of seats 130 based on the feature vector F 222. For example, in some embodiments, the artificial neural network 230 is configured to output the seat occupancy 240 as an estimate of the size of a person occupying each of the plurality of seats (e.g., as a value between 0 and 1). Seat occupancy 240 may be output to another vehicle system, such as a safety system for collision mitigation or child alert.
[0064] Numerous other aspects emerge from the above detailed description and the accompanying drawings. These aspects are embodied in the following items.
[0065] Item 1. A method for detecting seat occupancy of a vehicle, the vehicle comprising a plurality of seats, one or more microphones and one or more speakers, the method comprising: determining a first set of transfer functions between the one or more microphones and the one or more speakers based on a first audio signal played from the one or more speakers when a passenger occupies one or more of the plurality of seats; applying at least the first set of transfer functions and a second set of transfer functions of the vehicle to a neural network trained on the transfer functions of the vehicle type to predict seat occupancy of each of the plurality of seats based on the transfer functions.
[0066] Item 2. The method of Item 1, further comprising: determining the second set of transfer functions between the one or more microphones and the one or more speakers based on a second audio signal played from the one or more speakers when the vehicle is empty.
[0067] Item 3. A method according to Item 2, wherein determining the first set of transfer functions or the second set of transfer functions includes: playing the first audio signal or the second audio signal via the one or more speakers; recording, by each of the one or more microphones, a respective received audio signal for each of the speakers; and calculating the first set of transfer functions or the second set of transfer functions between each pair of speakers and microphones based on the first audio signal or the second audio signal and the respective received audio signals.
[0068] Clause 4. The method of clause 3, wherein playing the first audio signal or the second audio signal via the one or more speakers comprises individually playing a segment of the first audio signal or the second audio signal from each speaker.
[0069] Clause 5. The method of clause 3, wherein playing the first audio signal or the second audio signal via the one or more speakers comprises decorrelating the first audio signal or the second audio signal for each speaker.
[0070] Clause 6. According to any of Clauses 2-5, wherein the second audio signal played from the one or more speakers is played in response to the vehicle being unlocked when the vehicle is vacant.
[0071] Item 7. According to any one of Items 1-6, wherein applying at least the first set of transfer functions and the second set of transfer functions of the vehicle to the neural network includes: calculating a feature vector from at least the first set of transfer functions and the second set of transfer functions of the vehicle; and inputting the feature vector into the neural network to predict seat occupancy of each of the multiple seats based on the feature vector.
[0072] Item 8. According to Item 7, wherein the feature vector also includes the current temperature.
[0073] Clause 9. According to clause 7 or 8, computing the eigenvector comprises determining magnitude differences between corresponding pairs of the first set of transfer functions and the second set of transfer functions at a plurality of frequencies within an analysis frequency band.
[0074] Item 10. According to item 9, the frequency band analyzed is approximately 250 Hz-12 kHz and the frequency resolution is 50-150 Hz.
[0075] Clause 11. According to any of Clauses 1-10, wherein the neural network is configured to output an estimate of the size of a person occupying each seat in the plurality of seats.
[0076] Clause 12. According to any of Clauses 1-11, wherein the first audio signal is a preconfigured beep across the analyzed frequency band.
[0077] Clause 13. According to any of Clauses 1-11, wherein the first audio signal is an output from an entertainment system.
[0078] Item 14. A method for training a neural network to detect vehicle occupancy, the vehicle comprising a plurality of seats, one or more microphones and one or more speakers, the method comprising: playing an audio signal in the vehicle via the one or more speakers when the plurality of seats are in each of a plurality of occupancy states including an empty vehicle state; recording, by the one or more microphones, respective received audio signals for each of the one or more speakers for each occupancy state; calculating a set of transfer functions between each pair of speakers and microphones for each occupancy state; and training the neural network to predict the occupancy of each of the plurality of seats based on a first set of transfer functions for the empty vehicle state and a second set of transfer functions for the current occupancy state.
[0079] Clause 15. The method of clause 14, wherein playing the audio signal via the one or more speakers comprises individually playing a segment of the audio signal from each speaker.
[0080] Item 16. The method of Item 14, wherein training the neural network comprises generating a plurality of training sets, each training set comprising a feature vector of a magnitude difference between the first set of transfer functions and the second set of transfer functions and a state of the current occupancy condition.
[0081] Clause 17. The method of clause 16, wherein each set of transfer functions is further associated with a current temperature, and wherein the feature vector further includes the current temperature.
[0082] Clause 18. According to clause 16 or 17, wherein the feature vector comprises magnitude differences between corresponding pairs of the first set of transfer functions and the second set of transfer functions at a plurality of frequencies within the analyzed frequency band.
[0083] Item 19. According to item 18, the frequency band analyzed is approximately 250 Hz-12 kHz and the frequency resolution is 50-150 Hz.
[0084] Item 20. According to any of Items 14-19, wherein the neural network is configured to output an estimate of the size of a person occupying each seat in the plurality of seats.
[0085] Item 21. A device for detecting seat occupancy of a vehicle, the vehicle comprising a plurality of seats, one or more microphones and one or more speakers, the device comprising: a memory storing computer executable instructions; a digital signal processor coupled to the memory and configured to execute the instructions to: determine a first set of transfer functions between the one or more microphones and the one or more speakers based on a first audio signal played from the one or more speakers when a passenger occupies one or more of the plurality of seats; apply at least the first set of transfer functions and a second set of transfer functions of the vehicle to a neural network trained on the transfer functions of the vehicle type to predict a seat occupancy rate of each of the plurality of seats based on the transfer functions.
[0086] Item 22. The apparatus of Item 21, wherein the digital signal processor is configured to determine the second set of transfer functions between the one or more microphones and the one or more speakers based on a second audio signal played from the one or more speakers when the vehicle is empty.
[0087] Item 23. The device of Item 22, wherein, in order to determine a first set of transfer functions or a second set of transfer functions, the digital signal processor is configured to: output the first audio signal or the second audio signal to the one or more speakers via the one or more speakers; receive a respective received audio signal for each of the speakers from each of the one or more microphones; and calculate the first set of transfer functions or the second set of transfer functions between each pair of speakers and microphones based on the first audio signal or the second audio signal and the respective received audio signals.
[0088] Item 24. The device of Item 23, wherein, to output the first audio signal or the second audio signal via the one or more speakers, the digital signal processor is configured to output segments of the first audio signal or the second audio signal separately to each speaker.
[0089] Item 25. The device of Item 23, wherein, to output the first audio signal or the second audio signal via the one or more speakers, the digital signal processor is configured to decorrelate the first audio signal or the second audio signal for each speaker.
[0090] Clause 26. The device of any of Clauses 22-25, wherein the digital signal processor is configured to output the second audio signal to the one or more speakers when the vehicle is vacant, in response to the vehicle being unlocked.
[0091] Item 27. An apparatus as described in any of Items 21-26, wherein, in order to apply at least a first set of transfer functions and a second set of transfer functions of the vehicle to a neural network, the digital signal processor is configured to: calculate a feature vector from at least the first set of transfer functions and the second set of transfer functions of the vehicle; and input the feature vector into the neural network to predict seat occupancy of each of the plurality of seats based on the feature vector.
[0092] Item 28. The device of Item 27, wherein the feature vector also includes a current temperature.
[0093] Item 29. The apparatus of Item 27 or 28, wherein, to calculate the eigenvector, the digital signal processor is configured to determine magnitude differences between corresponding pairs of the first set of transfer functions and the second set of transfer functions at a plurality of frequencies within the analyzed frequency band.
[0094] Item 30. The apparatus of Item 29, wherein the frequency band analyzed is approximately 250 Hz-12 kHz and the frequency resolution is 50-150 Hz.
[0095] Item 31. The apparatus of any of Items 21-30, wherein the neural network is configured to output an estimate of the size of a person occupying each of the plurality of seats.
[0096] Clause 32. The device of any of Clauses 21-31, wherein the first audio signal is a preconfigured beep across the analyzed frequency band.
[0097] Clause 33. The device of any of Clauses 21-31, wherein the first audio signal is output via an entertainment system.
[0098] Item 34. A vehicle comprising the apparatus of any one of Items 21-33, the vehicle further comprising: the plurality of seats; the one or more microphones; and one or more speakers.
[0099] For example, an element, any part of an element, or any combination of elements can be implemented with a "processing system" including one or more processors. Examples of processors include microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to perform various functions described throughout the present disclosure. One or more processors in a processing system can execute software. Software should be broadly interpreted as instructions, instruction sets, codes, code segments, program codes, programs, subroutines, scripts, software modules, applications, software applications, packages, routines, subroutines, objects, executable files, execution threads, processes, functions, etc., whether referring to software, firmware, middleware, microcode, hardware description languages, or other.
[0100] Therefore, in one or more aspects, one or more of the functions described can be implemented in hardware, software, firmware or any combination thereof. If implemented in software, these functions can be stored on a computer-readable medium, or encoded as one or more instructions or codes in a computer-readable medium. Computer-readable media include computer storage media. Storage media can be any available medium that a computer can access. As an example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer. Disks and disks as used herein include compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs) and floppy disks, wherein disks generally reproduce data magnetically, and disks reproduce data optically with lasers. The above combination should also be included in the scope of computer-readable media. Non-transient computer-readable media do not include transient signals.
Claims
1. A method for detecting seat occupancy of a vehicle, the vehicle comprising a plurality of seats, one or more microphones and one or more speakers, the method comprising: determining a first set of transfer functions between the one or more microphones and the one or more speakers based on a first audio signal played from the one or more speakers when a passenger occupies one or more of the plurality of seats; and Applying at least the first and second sets of transfer functions for the vehicle to a neural network trained on transfer functions for the vehicle type to predict seat occupancy for each of the plurality of seats based on the transfer functions.
2. The method according to claim 1, further comprising: The second set of transfer functions between the one or more microphones and the one or more speakers is determined based on a second audio signal played from the one or more speakers while the vehicle is empty.
3. The method of claim 2, wherein determining the first set of transfer functions or the second set of transfer functions comprises: playing the first audio signal or the second audio signal via the one or more speakers; recording, by each of the one or more microphones, a respective received audio signal for each of the speakers; and The first set of transfer functions or the second set of transfer functions between each pair of loudspeaker and microphone is calculated based on the first audio signal or the second audio signal and the respective received audio signal. 4 . The method of claim 3 , wherein playing the first audio signal or the second audio signal via the one or more speakers comprises individually playing a segment of the first audio signal or the second audio signal from each speaker. 5 . The method of claim 3 , wherein playing the first audio signal or the second audio signal via the one or more speakers comprises decorrelating the first audio signal or the second audio signal for each speaker. 6 . The method of claim 2 , wherein the second audio signal played from the one or more speakers is played in response to the vehicle being unlocked when the vehicle is vacant.
7. The method of claim 1 , wherein applying at least the first set of transfer functions and the second set of transfer functions of the vehicle to the neural network comprises: calculating a feature vector from at least a first set of transfer functions and a second set of transfer functions of the vehicle; and The feature vector is input into the neural network to predict seat occupancy of each of the plurality of seats based on the feature vector. The method according to claim 7 , wherein the feature vector further comprises a current temperature.
9. The method of claim 7, wherein calculating the eigenvector comprises determining magnitude differences between corresponding pairs of the first and second sets of transfer functions at a plurality of frequencies within an analysis frequency band.
10. The method of claim 9, wherein the frequency band analyzed is approximately 250 Hz - 12 kHz with a frequency resolution of 50-150 Hz.
11. The method of claim 1 , wherein the neural network is configured to output an estimate of a size of a person occupying each of the plurality of seats.
12. The method of claim 1, wherein the first audio signal is a preconfigured beep across the analyzed frequency band.
13. The method of claim 1, wherein the first audio signal is an output from an entertainment system.
14. A method of training a neural network to detect occupancy of a vehicle, the vehicle comprising a plurality of seats, one or more microphones, and one or more speakers, the method comprising: playing an audio signal via the one or more speakers in the vehicle when the plurality of seats are in each of a plurality of occupancy states including a vacant state; recording, by the one or more microphones, respective received audio signals of each of the one or more speakers for each occupancy state; calculating a set of transfer functions between each pair of loudspeakers and microphones for each occupancy state; and A neural network is trained to predict occupancy of each of the plurality of seats based on a first set of transfer functions for the vacant state and a second set of transfer functions for a current occupancy state.
15. The method of claim 14, wherein playing the audio signal via the one or more speakers comprises individually playing a segment of the audio signal from each speaker.
16. The method of claim 14, wherein training the neural network comprises generating a plurality of training sets, each training set comprising a feature vector of a magnitude difference between the first set of transfer functions and the second set of transfer functions and a state of the current occupancy condition.
17. The method of claim 16, wherein each set of transfer functions is also associated with a current temperature, and the feature vector further includes the current temperature.
18. The method of claim 16, wherein the eigenvector comprises magnitude differences between corresponding pairs of the first and second sets of transfer functions at a plurality of frequencies within the analyzed frequency band.
19. The method of claim 18, wherein the frequency band analyzed is approximately 250 Hz - 12 kHz with a frequency resolution of 50-150 Hz.
20. The method of claim 14, wherein the neural network is configured to output an estimate of a size of a person occupying each seat in the plurality of seats.
21. An apparatus for detecting seat occupancy of a vehicle, the vehicle comprising a plurality of seats, one or more microphones and one or more speakers, the apparatus comprising: a memory storing computer executable instructions; and a digital signal processor coupled to the memory and configured to execute the instructions to: determining a first set of transfer functions between the one or more microphones and the one or more speakers based on a first audio signal played from the one or more speakers when a passenger occupies one or more of the plurality of seats; and Applying at least the first and second sets of transfer functions for the vehicle to a neural network trained on transfer functions for the vehicle type to predict seat occupancy for each of the plurality of seats based on the transfer functions.
22. The apparatus of claim 21, wherein the digital signal processor is configured to: The second set of transfer functions between the one or more microphones and the one or more speakers is determined based on a second audio signal played from the one or more speakers while the vehicle is empty.
23. The apparatus of claim 22, wherein to determine the first set of transfer functions or the second set of transfer functions, outputting the first audio signal or the second audio signal to the one or more speakers via the one or more speakers; receiving, from each of the one or more microphones, a respective received audio signal for each of the speakers; and The first set of transfer functions or the second set of transfer functions between each pair of loudspeaker and microphone is calculated based on the first audio signal or the second audio signal and the respective received audio signal.
24. The apparatus of claim 23, wherein to output the first audio signal or the second audio signal via the one or more speakers, the digital signal processor is configured to output segments of the first audio signal or the second audio signal to each speaker individually.
25. The apparatus of claim 23, wherein to output the first audio signal or the second audio signal via the one or more speakers, the digital signal processor is configured to decorrelate the first audio signal or the second audio signal for each speaker.
26. The device of claim 22, wherein the digital signal processor is configured to output the second audio signal to the one or more speakers when the vehicle is vacant, in response to the vehicle being unlocked.
27. The apparatus of claim 21, wherein to apply at least a first set of transfer functions and a second set of transfer functions of the vehicle to a neural network, the digital signal processor is configured to: calculating a feature vector from at least a first set of transfer functions and a second set of transfer functions for the vehicle; and The feature vector is input into the neural network to predict seat occupancy of each of the plurality of seats based on the feature vector.
28. The apparatus of claim 27, wherein the feature vector further comprises a current temperature.
29. The apparatus of claim 27, wherein to compute the eigenvectors, the digital signal processor is configured to determine magnitude differences between corresponding pairs of the first and second sets of transfer functions at a plurality of frequencies within an analyzed frequency band.
30. The apparatus of claim 29, wherein the frequency band analyzed is approximately 250 Hz - 12 kHz with a frequency resolution of 50-150 Hz.
31. The apparatus of claim 21 , wherein the neural network is configured to output an estimate of a size of a person occupying each of the plurality of seats.
32. The apparatus of claim 21, wherein the first audio signal is a preconfigured beep spanning the analyzed frequency band.
33. The apparatus of claim 21, wherein the first audio signal is via an output of an entertainment system.
34. A vehicle comprising the apparatus of claim 21, the vehicle further comprising: the plurality of seats; the one or more microphones; and One or more speakers.
Citation Information
Cited By
Method and apparatus for detecting presence of occupant in seat
WO2026152684A1
