Systems and procedures for vehicle occupancy management

Abstract

Method comprising: Receiving a current vehicle image representing a current interior of a vehicle (100); Detecting, by means of an occupancy management system (104), of at least one passenger (F) in the vehicle (100) based on the current vehicle image, further comprising: Accessing an image of an empty vehicle (100) representing the interior of the vehicle (100) without passengers (F); and Subtracting the image of the empty vehicle (100) from the current vehicle image using a background subtraction algorithm, wherein the background subtraction algorithm is one of the following: Weighted Moving Mean, Adaptive Background Learning, Fuzzy Gaussian, Gaussian Mixture Model, and Multi-Layer BGS; Determining, by means of the occupancy management system (104), a seat (S) of the passenger (F);Generating, through the occupancy management system (104), a seat card that identifies the seat (S) of the passenger (F) in the vehicle (100); and estimating the age, weight, height, gender, and emotion of the passenger (F).

Description

TECHNICAL AREA The present disclosure relates to vehicle systems and in particular to systems and methods that manage the occupants in a vehicle. GENERAL STATE OF THE ART Automobiles and other vehicles provide a significant portion of transportation for commercial, governmental, and private entities. Vehicles, including autonomous vehicles, travel on roads, parking lots, and other surfaces when transporting passengers or goods from one location to another. One example of an autonomous vehicle's application is operating as a taxi or shuttle service, picking up one or more passengers in response to a transportation request. When operating as a taxi or shuttle, the autonomous vehicle travels to a designated pickup point, allowing one or more passengers requesting the service to board. The vehicle then proceeds to one or more destinations, allowing the passengers to disembark. When an autonomous vehicle is operated as a taxi or shuttle service, managing vehicle occupancy is crucial. For example, the autonomous vehicle needs to know which seats are occupied and how many additional passengers can board. Furthermore, the autonomous vehicle may need to ensure that the correct passenger disembarks at the designated destination. Several solutions for seat occupancy detection in vehicles are already known. These employ image-based methods, such as the evaluation of reflected light from coated seat surfaces (DE 10 2015 010 282 A1), the automatic repositioning of seats for collision prevention (DE 10 2016 213 040 A1), the analysis of body contour, position, and posture for the detection of occupants or child seats (DE 10 2016 003 315 A1), seat occupancy detection through image comparison for controlling audio and monitoring functions (DE 10 2005 031 338 A1), and depth mapping using stereoscopic image acquisition for the demand-based activation of restraint systems (DE 198 52 653 A1). Although these methods provide reliable occupancy data and partially capture passenger-related parameters, no further feature evaluation is performed for personalized system adaptation or identification of individual occupants. The object of the invention is to provide improved solutions for seat occupancy detection. This problem is solved by the subject matter of the independent claims. Preferred embodiments of the present invention are the subject matter of the dependent claims. BRIEF DESCRIPTION OF THE DRAWINGS Non-restrictive and non-exhaustive embodiments of the present disclosure are described with reference to the following figures, wherein in the different figures the same reference numerals refer to the same parts unless otherwise indicated. Fig. 1 is a block diagram illustrating an embodiment of a vehicle control system that includes an occupancy management system. Fig. 2 is a block diagram illustrating an embodiment of an occupancy management system. Fig. 3 illustrates an embodiment of a vehicle with multiple interior cameras that record different aspects of the vehicle's interior. Fig. 4 illustrates an embodiment of a method for identifying a passenger in a vehicle. Fig. 5 illustrates an embodiment of a method for managing the occupancy of a vehicle when passengers enter or exit the vehicle.Figures 6A and 6B illustrate exemplary images of the interior of a vehicle with four passenger seats. Figures 7A and 7B illustrate exemplary images of the interior of a vehicle with five passengers. Figure 8 illustrates an exemplary image of the interior of the vehicle shown in Figures 7A and 7B after some of the passengers have changed their seats. DETAILED DESCRIPTION The following disclosure refers to the accompanying drawings, which form part thereof and in which specific implementations of the disclosure can be carried out are shown for illustration. It is understood that other implementations may be used and structural changes may be made without deviating from the scope of the present disclosure. References in the description to "an embodiment," "an exemplary embodiment," "an exemplary embodiment," etc., indicate that the described embodiment may include a certain property, structure, or feature; however, not every embodiment necessarily includes that certain property, structure, or feature. Furthermore, such formulations do not necessarily refer to the same embodiment.Furthermore, it should be noted that if a particular property, structure or feature is described in connection with an embodiment, it is within the scope of the skilled person's knowledge to implement such a property, structure or feature in connection with other embodiments, whether this is expressly described or not. Implementations of the systems, devices, and methods disclosed in this document may include or utilize a specialized or general-purpose computer that incorporates computer hardware, such as one or more processors and system memory, as discussed herein. Implementations within the scope of this disclosure may also include physical and other computer-readable media for transporting or storing computer-executable instructions and / or data structures. Such computer-readable media may be any available media accessible by a general-purpose or specialized computer system. Computer-readable media on which computer-executable instructions are stored are computer storage media (computer storage devices). Computer-readable media that transport computer-executable instructions are transmission media.Thus, implementations of the disclosure may, for example, and without limitation, include at least two distinctly different types of computer-readable media: computer storage media (devices) and transmission media. Computer storage media (devices) include RAM, ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM), flash memory, phase-change memory (“PCM”), other types of storage, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code resources in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or specialized computer. An implementation of the devices, systems, and methods disclosed in this document can communicate via a computer network. A "network" is defined as one or more data connections that enable the transport of electronic data between computer systems and / or modules and / or other electronic devices. When information is transmitted to or provided to a computer via a network or other communication link (either wired, wireless, or a combination of wired and wireless), the computer correctly views the link as a transmission medium. Transmission media can include a network and / or data links that can be used to transport desired program code resources in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or specialized computer.Combinations of the above should also be included in the scope of computer-readable media. Computer-executable instructions include, for example, instructions and data that, when executed on a processor, cause a general-purpose computer, a specialized computer, or a specialized processing device to perform a specific function or group of functions. Computer-executable instructions can be, for example, binary files, intermediate format instructions such as assembly language, or source code. Although the subject matter is described in a language specific to structural features and / or methodological actions, it is understood that the subject matter defined in the appended claims is not necessarily limited to the features or actions described therein. Rather, the described features and actions are disclosed as exemplary forms of implementation of the claims. The person skilled in the art will understand that the disclosure can be implemented in network computing environments with many types of computer system configurations, including dashboard vehicle computers, personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, tablets, pagers, routers, switches, various storage devices, and the like. The disclosure can also be implemented in distributed systems environments where both local and remote computer systems connected by a network (either by wired data links, wireless data links, or a combination of both) perform tasks.In a distributed systems environment, program modules can reside in both local and remote storage devices. Furthermore, the functions described in this document may optionally be performed in one or more of the following: hardware, software, firmware, digital components, or analog components. For example, one or more application-specific integrated circuits (ASICs) may be programmed to execute one or more of the systems and procedures described in this document. Certain terms are used throughout the description and in the claims to refer to specific system components. A person skilled in the art will understand that reference may be made to components with different designations. This document does not distinguish between components that differ in name but not in function. It should be noted that the sensor embodiments discussed herein may include computer hardware, software, firmware, or any combination thereof to perform at least some of their functions. For example, a sensor may include computer code configured to run on one or more processors and may include a hardware logic / electrical circuit controlled by the computer code. These exemplary devices are provided in this document for illustrative purposes and are not intended to be limiting. Embodiments of the present disclosure may be implemented in other types of devices, as is known to the person skilled in the art. At least some embodiments of the disclosure are applied to computer program products that include such logic (e.g., in the form of software) stored on any computer-usable medium. When executed in one or more data processing devices, such software causes a device to operate as described in this document. Fig. 1 is a block diagram illustrating an embodiment of a vehicle control system 100 within a vehicle, which includes an occupancy management system 104. An automated driving / assistance system 102 can be used to automate or control the operation of a vehicle or to assist a human driver. For example, the automated driving / assistance system 102 can control one or more of the vehicle's braking, steering, seatbelt tension, acceleration, lights, warnings, driver notifications, radio, vehicle locks, or any other assistance systems. In another example, the automated driving / assistance system 102 may not be able to control all driving functions (e.g.,The vehicle control system 100 does not provide steering, acceleration, or braking information, but can also provide notifications and warnings to assist a human driver in driving safely. The vehicle control system 100 includes the occupancy management system 104, which interacts with various components in the vehicle to identify passengers and their seats. Although the occupancy management system 104 is shown as a separate component in Fig. 1, in alternative embodiments it can be integrated into the automated driving / assistance system 102 or any other vehicle component. The vehicle control system 100 can be used with any type of vehicle, such as cars, trucks, buses, trains, airplanes, boats, and the like. The vehicle control system 100 also includes one or more sensor systems / devices for detecting the presence of nearby objects (or obstacles) or for determining the location of a parent vehicle (e.g., a vehicle that includes the vehicle control system 100). For example, the vehicle control system 100 may include one or more radar (radio detection and ranging) systems 106, one or more lidar (light detection and ranging) systems 108, one or more camera systems 110, a global positioning system (GPS) 112, and / or ultrasonic systems 114. The one or more camera systems 110 may include a rear-facing camera mounted on the vehicle (e.g., a rear section of the vehicle), a forward-facing camera, and a side-facing camera.As discussed herein, the camera systems 110 may also include one or more interior cameras that capture images of passengers and other objects inside the vehicle. Lidar systems 108 may include one or more interior lidar sensors that capture data related to the interior of the vehicle. The vehicle control system 100 may include a data storage device 116 for storing relevant or useful navigation and safety data, such as map data, driving history, or other data. The vehicle control system 100 may also include a transceiver 118 for wireless communication with a mobile or wireless network, other vehicles, infrastructure, or any other communication system. The vehicle control system 100 may include vehicle control actuators 120 to control various aspects of driving the vehicle, such as electric motors, switches, or other actuators to control braking, acceleration, steering, seatbelt tension, door locks, or the like. The vehicle control system 100 may also include one or more displays 122, speakers 124, or other devices to provide notifications to a human driver or passenger. A display 122 may be a front display, a gauge or indicator on the instrument panel, a screen display, or any other visual indicator visible to a driver or passenger. The speakers 124 may be one or more speakers of a vehicle sound system or may be a speaker specifically designed for driver or passenger notification. It is understood that the embodiment shown in Fig. 1 serves only as an example. Other embodiments may include fewer or additional components without deviating from the scope of the disclosure. Furthermore, the illustrated components may be combined or integrated into other components without restriction. In one embodiment, the automated driving / assistance system 102 is configured to control the driving or navigation of a base vehicle. For example, the automated driving / assistance system 102 can control the vehicle control actuators 120 to drive a route on a road, parking lot, driveway, or other location. For example, the automated driving / assistance system 102 can determine a route based on information or perception data provided by any of the components 106-118. A route can also be determined based on a path that the vehicle maneuvers to avoid or mitigate a potential collision with another vehicle or object. The sensor systems / devices 106-110 and 114 can be used to obtain real-time sensor data, enabling the automated driving / assistance system 102 to assist a driver or drive a vehicle in real time. In some embodiments, the vehicle control system 100 also includes one or more passenger input devices, such as microphones, touchscreen displays, buttons, and the like. These passenger input devices enable a passenger in a vehicle to provide input to the vehicle control system 100, such as answering questions, requesting information, requesting vehicle operations, and the like. Fig. 2 is a block diagram illustrating an embodiment of an occupancy management system 104. As shown in Fig. 2, the occupancy management system 104 includes a communication manager 202, a processor 204, and a memory 206. The communication manager 202 enables the occupancy management system 104 to communicate with other systems, such as an automated driving / assistance system 102. The processor 204 executes various instructions to implement the functionality provided by the occupancy management system 104, as discussed here. The memory 206 stores these instructions as well as other data used by the processor 204 and other modules and components included in the occupancy management system 104. Furthermore, the occupancy management system 104 includes an image processing module 208 that receives current image data (e.g., image data depicting the current interior of a vehicle) from one or more camera systems 110. The image processing module can also receive image data depicting the interior of the vehicle without any passengers (e.g., images of an empty vehicle). As discussed herein, the current vehicle images and the images of the empty vehicle are used to identify the passengers and passenger seats in the vehicle. In some embodiments, the image processing module 208 includes an image management algorithm or process that manages one or more images of the empty vehicle from multiple camera systems 110 located inside the vehicle. These same camera systems 110 capture current images of the vehicle interior for comparison with the images of the empty vehicle.In some embodiments, the current vehicle images and the images of the empty vehicle are taken from approximately the same position or perspective for proper comparison of the images. The occupancy management system 104 also includes an image subtraction module 210, which subtracts the images of the empty vehicle from the current vehicle images to identify differences between the images. These differences can represent passengers sitting in specific seats in the vehicle. The image subtraction module 210 can use any of a variety of background subtraction algorithms, such as Frame Difference, Weighted Moving Mean, Adaptive Background Learning, Fuzzy Gaussian, Gaussian Mixture Model, Multi-layer BGS, and the like. A passenger identification module 212 identifies one or more passengers in the vehicle, for example, by subtracting the images of the empty vehicle from the current vehicle images. Based on these results (e.g., differences between the images of the empty vehicle and the current vehicle images), the passenger identification module 212 can identify the faces or bodies of the passengers and associate each identified passenger with a specific seating position in the vehicle. Furthermore, the passenger identification module 212 can identify specific characteristics of each identified passenger, such as clothing, facial features, hair characteristics, and the like. These passenger characteristics can be used to identify the same passenger when they move to a different seat in the vehicle. In some embodiments, a facial recognition module 214 attempts to determine the identity of each passenger in the vehicle based on current vehicle images. The facial recognition module 214 can access a database or other data storage mechanism to correlate the facial features of current passengers with known individuals (e.g., previous passengers of the current vehicle or from other vehicles). The facial recognition module 214 can employ any of a variety of facial recognition algorithms, such as PCA (Principal Component Analysis), LDA (Linear Discriminant Analysis), ICA (Independent Component Analysis), EP (Evolutionary Pursuit), kernel-based algorithms, SVM (Support Vector Machine), CLM (Constrained Local Model), neural networks, and the like.If the facial recognition module 214 can determine the identity of one or more passengers in the vehicle, the vehicle control system 100 can adjust one or more vehicle settings or operating parameters (e.g., radio station preferences, temperature preferences, etc.) based on known passenger preferences. The occupancy management system 104 also includes a seat map generator 216, which generates an up-to-date seat map for the vehicle based on the current number of passengers. The seat map shows which passenger is occupying each seat. The seat map also identifies empty seats in the vehicle. The Seat Map Generator 216 can also associate passenger identification information, passenger characteristics, and facial recognition identity with each passenger in the vehicle. For example, passenger identification information, passenger characteristics, and facial recognition identity can be included as metadata associated with each seat. An estimation module 218 can determine (or estimate) other passenger characteristics, such as passenger height, passenger weight, passenger age, passenger gender, and passenger emotion. In some embodiments, these additional passenger characteristics are useful for determining the correct / safe airbag deployment, identifying the need for a child seat or booster seat, and the like. In certain embodiments, one or more sensors in the vehicle seats can estimate a passenger's weight. A passenger's height can be estimated, for example, based on received images and the position of the top of the passenger's head relative to the known dimensions of the seat surfaces. In some embodiments, passenger age, passenger gender, and passenger emotion can be estimated using machine learning techniques, such as neural networks.An alarm generator 220 generates various alarms and warnings based on activities and situations detected by the occupancy management system 104. For example, if a particular passenger attempts to leave the vehicle at the wrong destination, the alarm generator 220 can generate an audible or visual alarm instructing the passenger to remain in the vehicle. Similarly, if one or more passengers are not wearing seatbelts, the alarm generator 220 can generate an audible or visual alarm instructing the passenger to fasten their seatbelt. Another type of alarm indicates whether an airbag for a specific seat has been activated or deactivated. In some configurations, a specific seat in a vehicle can be reserved for a particular passenger (e.g., a passenger scheduled to be picked up in the future).If another person tries to sit in the reserved seat, an alarm may indicate that the seat is already reserved for another passenger. Fig. 3 illustrates an embodiment of a vehicle 300 with multiple interior cameras that record different aspects of the vehicle's interior. As shown in Fig. 3, the vehicle 300 has four interior cameras 302, 304, 306, and 308. In some embodiments, the cameras 302-308 are positioned and oriented in the vehicle 300 such that all seats are within the field of view of at least one camera 302-308. Other areas of the vehicle 300's interior may also be within the field of view of one or more cameras 302-308. The cameras 302-308 can be of any type, such as an RGB (red, green, and blue) camera, an IR (infrared) camera, a stereo camera, and the like. In some versions, other types of sensors (e.g. lidar systems 108) can be used instead of or in combination with the cameras 302-308 to detect passengers and other objects in the vehicle 300. In the configuration of vehicle 300, cameras 302 and 304 are positioned and oriented to capture images of the seats in the front section of vehicle 300. Cameras 306 and 308 are also positioned and oriented to capture images of the seats in the rear section of vehicle 300. Although four interior cameras 302-308 are shown in Fig. 3, in alternative embodiments, vehicle 300 can have any number of interior cameras positioned at different locations within the vehicle and oriented at different angles. In some embodiments, cameras 302-308 can capture different types of images, such as RGB, RGB-D, IR, thermal, stereoscopic images, and the like. Fig. 4 illustrates an embodiment of a method 400 for identifying a passenger in a vehicle. Initially, an occupancy management system 402 receives one or more current images of a vehicle interior. The occupancy management system also accesses one or more images of an empty vehicle 404 associated with the vehicle interior. These empty vehicle images represent the interior of the vehicle when there are no passengers in the vehicle (i.e., the vehicle is empty). These empty vehicle images can also be referred to as reference images. In some embodiments, the perspective of each empty vehicle image (or the area captured in each empty vehicle image) is essentially the same as the perspective (or the area captured) in each current vehicle interior image.The occupancy management system can access images of the empty vehicle from a storage device inside or outside the vehicle. Method 400 continues when the occupancy management system detects passengers in the vehicle based on the current vehicle images and the images of the empty vehicle 406. In some embodiments, passengers are detected by subtracting the images of the empty vehicle from the current vehicle images to identify differences between the images. In other embodiments, passengers may be detected using neural networks or other techniques. In one particular implementation, the occupancy management system analyzes the differences between the images to determine whether the differences represent a passenger. For example, if the differences represent a person's face or body, the occupancy management system may determine that the difference is a passenger.As discussed in more detail below, the method can identify regions of interest within the images of the empty vehicle where passengers are expected to be located. When analyzing the differences between the images of the empty vehicle and the actual vehicle images, Method 400 can focus on the differences within the identified regions of interest. The occupancy management system (408) then assigns a seat to each passenger in the vehicle. Based on the analysis of current vehicle images and images of the empty vehicle, the occupancy management system determines which seats in the vehicle are currently occupied by a passenger. Based on the identified seats, the occupancy management system (410) generates a seat card associated with the vehicle. The seat card defines all seats in the vehicle and identifies which seats are currently occupied by a passenger. Procedure 400 continues as the occupancy management system attempts to determine the identity of each passenger in the vehicle by performing facial recognition on the current vehicle images. In some embodiments, the facial recognition process can access a database or other data storage mechanism to correlate the facial features of the current passengers with known individuals (e.g., previous passengers of the current vehicle or of other vehicles). In other embodiments, any other type of identification or recognition system can be used to determine the identity of specific passengers. If facial recognition fails to identify a particular passenger, the occupancy management system 414 identifies characteristics of that passenger. These passenger characteristics may include, for example, clothing, facial features, hair characteristics, and the like. In this situation, a passenger can be identified by a unique identifier (e.g., Passenger A, Driver B, or Passenger 4) to distinguish them from other passengers. This identifier can be used during the passenger's current journey in the vehicle and then deleted after the passenger has reached their destination. By providing a unique identifier for each passenger, the systems and procedures described herein are able to track the position of each passenger in the vehicle (e.g., track each passenger's current seat location).In some embodiments, these passenger characteristics (and markings) are used to identify the same passenger when they move to a different seat in the vehicle. Furthermore, the passenger characteristics are useful for distinguishing one passenger from another, even when the passenger's actual identity is unknown. When a passenger's identity is detected using facial recognition, this identity information is associated with the seat card. For example, if the identity of a specific user is determined, this identity information is associated with the respective seat where the passenger is currently located. Similarly, any passenger characteristics are associated with the seat card, so that these characteristics are associated with the respective seat where the passenger is currently located. Procedure 400 continues when the occupancy management system estimates an age, weight, height, gender, and / or emotion of each passenger 418. The occupancy management system 420 adjusts one or more vehicle settings or operating parameters based on the identified passengers. Vehicle settings or operating parameters may include, for example, radio station settings, temperature settings, autonomous driving characteristics (slow / smooth driving or faster driving and faster cornering). In some embodiments, these vehicle settings or operating parameters may be determined based on known passenger preferences identified in a passenger profile or other passenger data settings. In some embodiments, the occupancy management system knows the number of available seats within a given vehicle. After determining the number of current passengers in the vehicle, the occupancy management system can determine the number of available seats. Based on the number of available seats, the occupancy management system can determine whether the vehicle can accommodate additional passengers. In some embodiments, one or more sensors in the vehicle seat surfaces can confirm the presence of a passenger in a particular seat. For example, if a sensor in a seat surface detects a weight consistent with the weight of a passenger, this data can confirm the determination (based on the current image of the vehicle interior) that the corresponding seat is occupied by a passenger. Fig. 5 illustrates an embodiment of a method 500 for managing the occupancy of a vehicle when passengers board or disembark. First, the vehicle stops 502 to allow one or more passengers to board and / or disembark. The occupancy management system 504 identifies the current vehicle passengers (after boarding and / or disembarking has been completed). In some embodiments, 506 the occupancy management system generates an updated seat map based on the current vehicle passengers. For example, passengers who have disembarked are removed from the previous seat map, and passengers who have boarded are added to the seat map. The seat map is also updated with respect to passengers who have changed seats within the vehicle.In some embodiments, new passengers who have boarded the vehicle are identified and analyzed in the manner discussed herein with reference to Fig. 4. Procedure 500 continues when the occupancy management system determines 508 whether the correct passengers have exited the vehicle. In some embodiments, the occupancy management system maintains a list of destinations for each passenger in the vehicle. If the correct passengers have exited the vehicle 510, no action is required. However, if one or more passengers have inadvertently exited the vehicle at the wrong destination 510, 512 the occupancy management system generates an alarm indicating that a passenger has mistakenly exited the vehicle at the wrong location. The alarm may be an audible alarm, a visual alarm, a haptic alarm, or any other type of alarm. In some situations, the vehicle may not be permitted to travel again until the erroneous passenger re-enters the vehicle or indicates that the vehicle can proceed without them.If the vehicle is not permitted to proceed (514), procedure 500 reverts to 504 and waits until the erroneous passenger has re-entered the vehicle. If the vehicle is permitted to proceed (514), procedure 500 (516) adjusts one or more vehicle settings or operating parameters based on the identified current passengers. For example, passenger preferences may change after one or more passengers have exited and / or entered the vehicle. Figures 6A and 6B illustrate exemplary images of the interior of a vehicle with four passenger seats. Figure 6A illustrates an empty vehicle interior 600 with four passenger seats. Figure 6B shows the same vehicle interior 600 with the four passenger seats identified by the boundary frames 602-608. In this context, the boundary frames can also be referred to as regions of interest. In particular, a first boundary frame 602 identifies a first passenger seat, a second boundary frame 602 identifies a second passenger seat, a third boundary frame 606 identifies a third passenger seat, and a fourth boundary frame 608 identifies a fourth passenger seat. In some embodiments, the boundary frames for a particular vehicle are defined by a human operator.In situations where a particular vehicle model has uniform seating arrangements and uniform camera positions, the same boundary frames can be used for all production vehicles of that model or for all vehicles in a fleet of autonomous vehicles. In particular embodiments, an automated process is used to define boundary frames for a specific vehicle by identifying seating areas within the vehicle. Images, such as those shown in Figures 6A and 6B, are examples of images of an empty vehicle, as discussed herein. Figures 7A and 7B illustrate exemplary images of the interior of a vehicle with five passengers. Figure 7A illustrates an actual vehicle interior 700 with five passengers occupying the vehicle. The faces of the five passengers are identified by bounding frames or regions of interest. In particular, a first bounding frame 702 identifies a first passenger, a second bounding frame 704 identifies a second passenger, a third bounding frame 706 identifies a third passenger, a fourth bounding frame 708 identifies a fourth passenger, and a fifth bounding frame 710 identifies a fifth passenger. Figure 7B illustrates the same actual vehicle interior 700 with the five passengers identified by boundary frames 702-710. As shown in Figure 7B, each boundary frame 702-710 is associated with a passenger identity or passenger tag. If the passenger's identity is determined (e.g., by facial recognition), the passenger's name is associated with the corresponding boundary frame. If the passenger's identity is not determined, a tag is generated and associated with the corresponding boundary frame. For example, the first passenger, identified by boundary frame 702, is tagged as "Driver A" and associated with seat #2 in the vehicle. The second passenger, identified by boundary frame 704, is tagged as "Robert" (e.g., facial recognition identified the passenger as Robert) and associated with seat #5 in the vehicle.The third passenger, identified by boundary frame 706, is labeled "Driver C" and associated with seat #4 in the vehicle. The fourth passenger, identified by boundary frame 708, is labeled "Driver B" and associated with seat #3 in the vehicle. The fifth passenger, identified by boundary frame 710, is labeled "James" and associated with seat #1 in the vehicle. As discussed herein, information about occupied vehicle seats, as well as the identity (or label) associated with each passenger, can be linked to a seat map generated for the vehicle. Fig. 8 illustrates an exemplary image of the interior of the vehicle shown in Figs. 7A and 7B after some of the passengers have changed seats. Fig. 8 illustrates an actual vehicle interior 800 with five passengers occupying the vehicle. The actual vehicle interior 800 is the same vehicle as the one shown in actual vehicle 700, but the vehicle has been stopped to allow passengers to board or alight. As shown in actual vehicle interior 800, James and Robert have changed seats, driver C remains in the same seat, driver B has taken a different seat, driver A has alighted from the vehicle, and driver D has boarded the vehicle.As updated herein, the seat map generated for the vehicle is updated to include current information about occupied vehicle seats as well as the identity (or identifier) ​​associated with each passenger. While various embodiments of the present disclosure are described herein, it is understood that these serve only as examples and not as limitations. It is evident to the person skilled in the art that various modifications in form and detail can be made without departing from the spirit and scope of the disclosure. Therefore, the breadth and scope of the present disclosure are not intended to be limited by any of the described embodiments, but are merely defined in accordance with the following claims and their equivalents. The description is set forth herein for illustrative and descriptive purposes. It makes no claim to completeness and is not intended to limit the disclosure to the specific form disclosed. Many modifications and variations are possible in light of the disclosed teachings.Furthermore, it should be noted that any or all of the alternative implementations discussed here can be used in any desired combination to form additional hybrid implementations of the revelation.

Claims

Method comprising: Receiving a current vehicle image representing a current interior of a vehicle (100); Detecting, by means of an occupancy management system (104), of at least one passenger (F) in the vehicle (100) based on the current vehicle image, further comprising: Accessing an image of an empty vehicle (100) representing the interior of the vehicle (100) without passengers (F); and Subtracting the image of the empty vehicle (100) from the current vehicle image using a background subtraction algorithm, wherein the background subtraction algorithm is one of the following: Weighted Moving Mean, Adaptive Background Learning, Fuzzy Gaussian, Gaussian Mixture Model, and Multi-Layer BGS; Determining, by means of the occupancy management system (104), a seat (S) of the passenger (F);Generating, through the occupancy management system (104), a seat card that identifies the seat (S) of the passenger (F) in the vehicle (100); and estimating the age, weight, height, gender, and emotion of the passenger (F). Method according to claim 1, further comprising determining the identity of the passenger (F) by performing facial recognition using the current vehicle image. Method according to claim 2, further comprising adjusting at least one vehicle setting or operating parameter based on the identity of the passenger (F). Method according to claim 1, further comprising identifying features of the passenger (F) based on the current vehicle image. Method according to claim 4, wherein the features of the passenger (F) include passenger clothing, passenger facial features and passenger hair features. Method according to claim 5, wherein the characteristics of the passenger (F) are associated with the seat ticket. The method of claim 1, further comprising: detecting a plurality of passengers (F) in the vehicle (100) based on the current vehicle image; determining, by means of the occupancy management system (104), a seat (S) of each of the plurality of passengers (F); and generating, by means of the occupancy management system (104), a seat card which identifies the seat (S) of each of the plurality of passengers (F) in the vehicle (100). Method according to claim 1, wherein the vehicle (100) is an autonomous vehicle. Method comprising: Receiving a current vehicle image representing a current interior of a vehicle (100); Detecting, by means of an occupancy management system (104), a plurality of passengers (F) in the vehicle (100) based on the current vehicle image, further comprising: Accessing an image of an empty vehicle (100) representing the interior of the vehicle (100) without passengers (F); and Subtracting the image of the empty vehicle (100) from the current vehicle image using a background subtraction algorithm, wherein the background subtraction algorithm is one of the following: Weighted Moving Mean, Adaptive Background Learning, Fuzzy Gaussian, Gaussian Mixture Model, and Multi-Layer BGS; Determining, by means of the occupancy management system (104), a seat (S) from each of the plurality of passengers (F);Determine, through the occupancy management system (104), an identity of each of the multitude of passengers (F) by performing facial recognition using the current vehicle image; generate, through the occupancy management system (104), a seat card containing the identity and seat location (S) of each of the multitude of passengers (F) in the vehicle (100); and estimate the age, weight, height, gender, and emotion of the passenger (F). The method of claim 9, further comprising adjusting at least one vehicle setting or operating parameter based on the identity of at least one of the plurality of passengers (F). Method according to claim 9, further comprising identifying features of at least one of the plurality of passengers (F) based on the current vehicle image. Method according to claim 11, wherein the features of the passenger (F) include passenger clothing, passenger facial features and passenger hair features. The device comprises: a communications manager (202) configured to receive a current vehicle image representing the current interior of a vehicle (100); an image processing module (208) configured to detect a passenger (F) in the vehicle (100) based on the current vehicle image, the image processing module (208) further being configured to determine a seat (S) of the passenger (F) and to estimate the age, weight, height, gender, and emotion of the passenger (F); and accessing an image of an empty vehicle (100) representing the interior of the vehicle (100) without passengers (F).and to subtract the image of the empty vehicle (100) from the current vehicle image using a background subtraction algorithm, wherein the background subtraction algorithm is one of the following: Weighted Moving Mean, Adaptive Background Learning, Fuzzy Gaussian, Gaussian Mixture Model and Multi-Layer BGS; and a seat card generator (216) configured to generate a seat card that identifies the seat (S) of the passenger (F) in the vehicle (100). Device according to claim 13, further comprising a facial recognition module (214) configured to determine the identity of the passenger (F).

Images