System and method for monitoring user leaving vehicle

Abstract

Systems and methods for monitoring a user leaving a vehicle are provided. A vehicle is disclosed that includes a detection unit and a processor. The detection unit may be configured to detect the presence of a user in and near the vehicle. The processor may be configured to obtain an input from the detection unit and determine, based on the input, that a predefined condition may be satisfied. The processor may also be configured to generate a virtual zone around the vehicle in response to determining that the predefined condition may be satisfied. The processor may additionally transmit information associated with the virtual region to another vehicle in response to generating the virtual region.

Description

Technical Field

[0001] The present disclosure relates to systems and methods for monitoring users exiting a vehicle, and more particularly to systems and methods for monitoring children exiting a school bus and notifying nearby vehicles of the child's presence. Background Art

[0002] School buses provide young users (e.g., children) with a convenient means of transportation from their homes to school and back. When transporting children, school bus operators typically implement a number of measures to ensure that children arrive at school or their homes conveniently without any discomfort. For example, when children may be entering or leaving a school bus, the school bus operator may flash the stop lights of the school bus to notify passersby near the school bus of the presence of the child. The school bus operator may also audibly notify passersby of the presence of the child. Summary of the invention

[0003] The present disclosure describes a vehicle configured to monitor the movement of multiple users inside and outside the vehicle and notify other vehicles of the presence of the users. Specifically, the present disclosure describes a school bus that can be configured to monitor the movement of multiple children leaving the school bus and crossing a road or intersection, and notify other vehicles near the school bus of the presence of the children. The vehicle can be configured to obtain an image of the user from an internal camera and an external camera of the vehicle when the user leaves the vehicle, such as when the vehicle stops at a bus stop. In response to the vehicle stopping at a bus stop and / or at least one user leaving the vehicle, the vehicle can generate a virtual zone around or around the vehicle. The size of the virtual zone can be based on the size of the vehicle, the size (e.g., width) of the road where the vehicle may be located, the expected duration required for the user to cross the road (and the expected average user walking speed), etc. In response to generating a virtual zone, the vehicle can transmit information associated with the virtual zone to one or more other vehicles near the vehicle.

[0004] In some aspects, the information associated with the virtual zone may include the size and dimensions of the virtual zone, the associated expiration duration, etc. Other vehicles may receive the information and may display the information at a corresponding human-machine interface (HMI). Drivers associated with other vehicles may then view the information on the HMI and stop the movement of their respective vehicles. The virtual zone size may indicate to the driver that the respective vehicle should not enter the virtual zone until the virtual zone is displayed on the HMI. In some aspects, the virtual zone may disappear from the HMI after the expiration duration has passed. At this point, the other vehicles may resume their respective movements.

[0005] In other aspects, the vehicle can track the movement of each user by using images obtained from the vehicle's internal and external cameras when the user leaves the vehicle, and can cause the HMI to remove the virtual zone when each user may have crossed the road or left the vehicle. In other aspects, the vehicle can track the movement of each user by determining the location of a user device associated with each user (which can be communicatively coupled to the vehicle). The vehicle can also be configured to broadcast the determined location of each user device (and therefore the associated user) to multiple additional vehicles that may be located in the geographic area where the user may be located. The vehicle can broadcast the location of each user device to multiple additional vehicles directly via vehicle-to-vehicle (V2V) communication or indirectly via a server or connected infrastructure (e.g., by using vehicle-to-infrastructure (V2I) or vehicle-to-external (V2X) communication).

[0006] The present disclosure discloses a vehicle or school bus that notifies other vehicles of the presence of children near the school bus, thereby enabling the children to conveniently leave the school bus and / or cross a road or intersection. In addition, the vehicle broadcasts the location of each child to multiple vehicles in a geographic area where the children may be present, so that the drivers of these vehicles can be aware of the presence of the children and drive cautiously accordingly.

[0007] These and other advantages of the present disclosure are provided in detail herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Detailed description is set forth with reference to the accompanying drawings. The use of the same reference numerals may indicate similar or identical items. Various embodiments may utilize elements and / or components other than those shown in the accompanying drawings, and some elements and / or components may not be present in various embodiments. The elements and / or components in the accompanying drawings are not necessarily drawn to scale. Throughout this disclosure, singular and plural terms may be used interchangeably, depending on the context.

[0009] Figure 1 An example environment is depicted in which techniques and structures for providing the systems and methods disclosed herein may be implemented.

[0010] Figure 2 A block diagram of an example system for monitoring multiple users associated with a vehicle according to the present disclosure is depicted.

[0011] Figure 3A and Figure 3B Depicted is an example view of multiple users in proximity to a vehicle in accordance with the present disclosure.

[0012] Figure 4 Depicted is an example view of a plurality of vehicles and a plurality of users located in a geographic area in accordance with the present disclosure.

[0013] Figure 5 A flow chart of an example method for monitoring a user associated with a vehicle according to the present disclosure is depicted. DETAILED DESCRIPTION

[0014] The present disclosure will be described more fully hereinafter with reference to the accompanying drawings, in which example embodiments of the disclosure are shown and which are not intended to be limiting.

[0015] Figure 1 An example environment 100 is depicted in which techniques and structures for providing the systems and methods disclosed herein may be implemented. The environment 100 may include a vehicle 102 (e.g., a first vehicle), which may be, for example, a school bus, a bus, a shuttle, etc. The vehicle 102 may be a manually driven vehicle, and / or may be configured to operate in a partially autonomous or fully autonomous mode. The vehicle 102 may be configured to transport a plurality of users 104a, 104b, 104c, 104d, 104e, 104n (collectively, users 104) from one location to another. The users 104 may be, for example, school children, and the vehicle 102 may transport the users 104 from respective bus stops / homes to school and back. In Figure 1 In the exemplary aspect depicted in FIG. 1 , user 104 is shown exiting vehicle 102 and crossing road 106 .

[0016] In some aspects, the vehicle 102 can implement various measures to ensure the convenience of the user when entering or exiting the vehicle 102 and / or when crossing the road 106. For example, when the vehicle 102 stops at a bus stop and the user 104 begins to exit the vehicle 102, the vehicle 102 can illuminate or flash one or more vehicle exterior lights (e.g., parking lights, tail lights, headlights, etc.) for a predefined duration. The flashing of the vehicle exterior lights can provide a warning to other vehicles in the vicinity of the vehicle 102 (e.g., a second vehicle 108, such as a Figure 1 ) provides instructions to stop and allow the user 104 to conveniently exit the vehicle 102 and / or cross the road 106.

[0017] In another aspect, the vehicle 102 may include a notification system (in Figure 2 104 ), the notification system may implement additional measures to enable the user 104 to conveniently exit the vehicle 102 and / or cross the road 106. When the vehicle 102 stops at a bus stop, the notification system (“system”) may first detect the presence of a vehicle 102 by using a vehicle detection unit (in the Figure 2The system may determine the count of users 104 leaving the vehicle 102 by using input obtained from the detection unit 242 (shown as the detection unit 242 in FIG. 1 ). The detection unit may be, for example, an internal camera and an external camera of the vehicle, and the input obtained from the detection unit may include images captured by the internal camera and the external camera of the vehicle. In addition, when the vehicle 102 stops at a bus stop, the system may generate a virtual zone (in the vicinity of the vehicle 102) around the vehicle 102. Figure 3A and Figure 3B 102). In an exemplary aspect, the system can generate a virtual zone on a digital map of a geographic area where the vehicle 102 may be located. In some aspects, the size of the virtual zone can depend on the size of the vehicle 102, the road 106, etc. In other aspects, the size of the virtual zone can be preset.

[0018] In response to determining the count of users 104 and generating the virtual zone, the system may transmit information associated with the virtual zone to one or more vehicles (e.g., the second vehicle 108) that may be located near the vehicle 102. The system may transmit the information associated with the virtual zone to the second vehicle 108 directly via vehicle-to-vehicle (V2V) communication or indirectly via a server (e.g., by using cellular communication) and / or connected infrastructure (e.g., a connected lamppost or any other connected infrastructure, using vehicle-to-infrastructure (V2I) or vehicle-to-external (V2X) communication). The information associated with the virtual zone may include a virtual zone size, an expiration duration associated with the virtual zone, and / or a digital map (e.g., when the virtual zone may be generated on a digital map).

[0019] In response to the system transmitting information associated with the virtual zone, the second vehicle 108 may receive the information. The second vehicle 108 may then display the virtual zone on a human-machine interface (HMI) associated with the second vehicle 108 (on a digital map or otherwise). The driver associated with the second vehicle 108 may view the virtual zone on the HMI and may receive (from viewing the virtual zone) an indication that the user 104 may be leaving the vehicle 102 and / or crossing the road 106, and therefore the vehicle 108 should stop and not enter the virtual zone. In some aspects, the expiration duration may be associated with the duration that the virtual zone may be visible on the HMI associated with the second vehicle 108. The expiration duration may be the same or different from the above-mentioned predefined duration that the vehicle 102 may flash the vehicle exterior lights. In some aspects, the expiration duration may depend on the expected duration that the user 104 may take to leave the vehicle 102 and cross the road 106, and may depend on the vehicle and road size.

[0020] The present disclosure is not limited to the information associated with the virtual zone having an expiration duration. In some aspects, the HMI associated with the second vehicle 108 can continue to display the virtual zone until the second vehicle 108 continues to receive information associated with the virtual zone from the system. In this case, the system can continue to transmit information to the second vehicle 108 until the system determines that all users 104 may have crossed the road 106 and / or have left the virtual zone based on the input obtained from the detection unit and the determined user 104 count. In response to determining that all users 104 may have crossed the road 106 and / or left the virtual zone, the system can stop transmitting information to the second vehicle 108, at which point the HMI associated with the second vehicle 108 can stop displaying the virtual zone. When the HMI associated with the second vehicle 108 stops displaying the virtual zone, the second vehicle driver can obtain an indication that the driver can now drive the second vehicle 108. In this way, the system enables the user 104 to conveniently leave the vehicle 102 and / or cross the road 106 without encountering any interference from one or more vehicles (e.g., the second vehicle 108) that may be located near the vehicle 102.

[0021] In further aspects, the system may be communicatively coupled with a user device associated with the user 104. The system may "track" the real-time location of each user 104 by determining the real-time location of the corresponding user device relative to the vehicle 102, and may broadcast the real-time location of each user 104 to the second vehicle 108 and one or more additional vehicles that may be located in the geographic area where the vehicle 102 or the user 104 may be located but may not be near the vehicle 102. In some aspects, the vehicle 102 may broadcast the real-time location of each user 104 to the second vehicle 108 and the additional vehicles directly via V2V communication or indirectly via a server and / or connected infrastructure (e.g., via V2I / V2X communication). The system may continue to broadcast the real-time location of each user 104 for a predefined broadcast duration even after the user 104 may have crossed the road 106 or left the virtual zone. The real-time location of each user 104 may provide an indication to the driver associated with the second vehicle 108 and the additional vehicles that the user 104 may be near the geographic area, and the driver may therefore drive the corresponding vehicle with caution.

[0022] In some aspects, the system itself can broadcast the real-time location of each user 104 for a predefined broadcast duration, as described above. In other aspects, the system can transmit the real-time location of each user 104 to a server and / or infrastructure, and the server and / or infrastructure can broadcast the real-time location to the second vehicle 108 and the additional vehicles. In some aspects, one or more of the additional vehicles can be vehicles of one or more users' parents.

[0023] Combined with the following Figure 2 Describe the system in further detail.

[0024] The vehicle 102 and notification system implement and / or perform operations as described herein in the present disclosure in accordance with the vehicle owner's manual and safety guidelines. Additionally, any actions taken by the vehicle operator and / or driver associated with the second vehicle 108 and / or additional vehicles based on recommendations or notifications provided by the vehicle 102 and / or notification system should comply with all rules specific to the location (e.g., federal, state, country, city, etc.) and operation of the respective vehicle. Recommendations or notifications as provided by the vehicle 102 and / or notification system should be considered suggestions and followed only in accordance with any rules specific to the location and operation of the respective vehicle.

[0025] Figure 2 A block diagram of an example system 200 for monitoring a user 104 associated with a vehicle 202 according to the present disclosure is depicted. The vehicle 202 may be the same as the vehicle 102 described above. Figure 2 When Figure 3A , Figure 3B and Figure 4 .

[0026] System 200 may include a vehicle 202, one or more user devices 204, and one or more servers 206 communicatively coupled to each other via one or more networks 208. User device 204 may be associated with user 104. Figure 2 A single user device 204 is depicted, but the vehicle 202 may be communicatively coupled with multiple user devices 204, and each user device 204 may be associated with each user 104. The user device 204 may be, for example, a mobile phone, a laptop, a computer, a tablet, a wearable device, or any other similar device having communication capabilities.

[0027] The server 206 may be part of a cloud-based computing infrastructure and may be associated with and / or include a telematics service delivery network (SDN) that provides services to the vehicle 202 and other vehicles (e.g., the second vehicle 108 and the additional vehicles described above, Figure 2206). In another aspect, the server 206 may be associated with a navigation service company and may be configured to provide the vehicle 202 with a digital map of the geographic area where the vehicle 202 may be located. The server 206 may also be configured to store a mapping of each user device 204 (e.g., an identifier of each user device 204) and information associated with each user 104 (e.g., the user's name, contact information, home address, bus stop location, etc.). The server 206 may be configured to transmit the mapping to the vehicle 202 at a predefined frequency or when the vehicle 202 transmits a request to the server 206 to obtain the mapping. In an additional aspect, the server 206 may be configured to store the expected travel / movement path of each user 104 when the user 104 leaves the vehicle 202, and may transmit information associated with the expected path of each user 104 to the vehicle 202 at a predefined frequency or when the vehicle 202 transmits a request to the server 206 to obtain the information. In some aspects, the server 206 can be configured to predict the expected path of each user 104 based on historical user travel / movement patterns, user input, calendar events, user past use / availability of public transportation or shared mobility solutions, user reservations for public transportation or shared mobility solutions, etc. In other aspects, the vehicle 202 itself can predict the expected path of each user 104 based on the above parameters. In other aspects, each user device 204 can be associated with each user's vehicle. In this case, the vehicle 202 can use the last known location of each user's vehicle to predict the expected path of each user.

[0028] The network 208 illustrates an example communication infrastructure in which the connected devices discussed in various embodiments of the present disclosure may communicate. The network 208 may be and / or include the Internet, a private network, a public network, or other configurations operating using any one or more known communication protocols, such as, for example, Transmission Control Protocol / Internet Protocol (TCP / IP), BLE, Wi-Fi based on the Institute of Electrical and Electronics Engineers (IEEE) standard 802.11, ultra-wideband (UWB), and cellular technologies such as time division multiple access (TDMA), code division multiple access (CDMA), high-speed packet access (HSPDA), long-term evolution (LTE), global system for mobile communications (GSM), and fifth generation (5G), to name a few examples.

[0029] The vehicle 202 may include a plurality of units including, but not limited to, a vehicle computer 210, a vehicle control unit (VCU) 212, and a notification system 214 (or system 214). The VCU 212 may include a plurality of electronic control units (ECUs) 216 arranged to communicate with the vehicle computer 210.

[0030] The user device 204 may be connected to the vehicle computer 210 and / or the system 214 via a network 208, which may communicate via one or more wireless connections, and / or the user device may communicate using a near field communication (NFC) protocol, Protocol, Wi-Fi, ultra-wideband (UWB) and other possible data connection and sharing technologies can be directly connected to the vehicle 202. In some aspects, the vehicle computer 210 and / or system 214 can be connected to the user device 204 via, for example, The vehicle computer 210 and / or the system 214 may determine that the user device 204 is near the vehicle 202 or in the vehicle 202 by measuring a received signal strength indicator (RSSI) value associated with the user device 204 when the BLE or BLE is connected to the vehicle 202. In an exemplary aspect, if the measured RSSI value may be greater than a predefined threshold, the vehicle computer 210 and / or the system 214 may determine that the user device 204 may be near the vehicle 202 or may be in the vehicle 202.

[0031] In some aspects, according to the present disclosure, the vehicle computer 210 and / or the system 214 can be installed anywhere in the vehicle 202. In addition, the vehicle computer 210 can operate as a functional part of the system 214. The vehicle computer 210 can be or include an electronic vehicle controller having one or more processors 218 and a memory 220. In addition, the system 214 can be separate from the vehicle computer 210 (e.g., Figure 2 ), or may be integrated as part of the vehicle computer 210.

[0032] The processor 218 may be configured to communicate with one or more memory devices (e.g., memory 220 and / or Figure 2 The processor 218 may utilize the memory 220 to store programs and / or store data in code form for executing aspects according to the present disclosure. The memory 220 may be a non-temporary computer-readable storage medium or memory storing notification program code. The memory 220 may include any one or combination of volatile memory elements (e.g., dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), etc.), and may include any one or more non-volatile memory elements (e.g., erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), etc.).

[0033] According to some aspects, the VCU 212 can share a power bus with the vehicle computer 210 and can be configured and / or programmed to communicate between vehicle systems, connected servers (e.g., server 206), and other vehicles operating as part of a vehicle fleet (e.g., Figure 2 The VCU 212 may include or communicate with any combination of ECUs 216, such as, for example, a body control module (BCM) 222, an engine control module (ECM) 224, a transmission control module (TCM) 226, a telematics control unit (TCU) 228, a driver assistance technology (DAT) controller 230, etc. The VCU 212 may also include and / or communicate with a vehicle perception system (VPS) 232, which may be connected to and / or control one or more vehicle sensing systems 234. The vehicle sensing system 234 may include one or more vehicle sensors including, but not limited to, radio detection and ranging (RADAR or “radar”) sensors configured to use radio waves to detect and locate objects inside and outside the vehicle 202, seating area latch sensors, seating area sensors, light detection and ranging (“lidar”) sensors, door sensors, proximity sensors, temperature sensors, wheel sensors, one or more ambient weather or temperature sensors, interior and exterior vehicle cameras, steering wheel sensors, inertial measurement units (IMUs), etc. The interior and exterior vehicle cameras may be configured to capture images of the user inside and outside the vehicle 202.

[0034] In some aspects, VCU 212 may control aspects of vehicle operation and implement one or more instruction sets received from server 206 , one or more instruction sets stored in memory 220 , including instructions to operate as part of system 214 .

[0035] The TCU 228 may be configured and / or programmed to provide vehicle connectivity to wireless computing systems on and off the vehicle 202 and may include a navigation (NAV) receiver 236 for receiving and processing GPS signals, module (BLEM) 238, a Wi-Fi transceiver, a UWB transceiver, and / or may be configured to communicate between the vehicle 202 and other systems (e.g., a vehicle key fob ( Figure 2 ), server 206, user device 204, etc.), other wireless transceivers (including cellular communications) for wireless communication between computers and modules Figure 22). The TCU 228 may be configured to communicate with the ECU 216 via a bus. In some aspects, the TCU 228 may be configured to determine real-time vehicle geographic location, for example, via the NAV receiver 236. Additionally, the TCU 228 may enable the user device 204 to be communicatively coupled to the vehicle 202 via the BLEM 238.

[0036] ECU 216 may control various aspects of vehicle operation and communications using input from a human driver, input from vehicle computer 210 , input from system 214 , and / or input via wireless signals received over a wireless connection from other connected devices (such as server 206 , etc.).

[0037] The BCM 222 typically includes an integration of sensors, vehicle performance indicators, and varactors associated with vehicle systems, and may include processor-based power distribution circuits that may control functions associated with the vehicle body such as lights (e.g., parking lights, tail lights, headlights, interior lights, etc.), windows, security devices, cameras, audio systems, speakers, wipers, door locks and entry controls, and various comfort controls, etc. The BCM 222 may also operate as a gateway for bus and network interfaces to communicate with remote ECUs ( Figure 2 (not shown) interaction.

[0038] The DAT controller 230 may provide Level 1 to Level 3 automated driving and driver assistance functions, which may include, for example, active park assist, vehicle back-up assist, adaptive cruise control, and / or lane keeping, among other features. The DAT controller 230 may also provide aspects of user and environmental input that may be used for user authentication.

[0039] In some aspects, the vehicle computer 210 can be connected to an infotainment system 240. The infotainment system 240 can include a touch screen interface portion, and can include voice recognition features, biometric recognition capabilities, and the biometric recognition capabilities can identify users based on facial recognition, voice recognition, fingerprint recognition, or other biometric recognition means. In other aspects, the infotainment system 240 can also be configured to receive user commands via the touch screen interface portion, and / or output or display notifications, navigation maps, travel information, etc. on the touch screen interface portion.

[0040] In some aspects, the vehicle 202 may also include a detection unit 242. The detection unit 242 may be part of the vehicle sensing system 234 or may be separate from the vehicle sensing system 234. In some aspects, the detection unit 242 may include internal and external vehicle cameras and may be configured to detect the presence of a user in or near the vehicle 202 by capturing an image of the user. In other aspects, the detection unit 242 may be a radar sensor (and / or a lidar sensor) configured to detect the presence of a user in or near the vehicle 202 by using radio waves. In further aspects, the detection unit 242 may include an infrared sensor, a sound-based sensor, a user device locator, and the like. When the detection unit 242 includes a user device locator, the detection unit 242 may be configured to determine the location of the user device associated with each user 104 relative to the vehicle 202. In addition, the accuracy of the detection unit 242 may be enhanced by using any combination of the above sensors and artificial intelligence (AI)-based analysis. In additional aspects, the detection unit 242 can obtain input from sensors external to the detection unit 242 (eg, sensors located on road infrastructure).

[0041] The computing system architecture of the vehicle computer 210, VCU 212 and / or system 214 may omit certain computing modules. It should be readily understood that Figure 2 The computing environment depicted in FIG. 1 is an example of possible implementations according to the present disclosure, and thus should not be considered limiting or exclusive.

[0042] According to some aspects, system 214 can be integrated with and / or executed as part of ECU 216. System 214, whether integrated with vehicle computer 210 or ECU 216, or operating as a standalone computing system in vehicle 202, can include transceiver 244, processor 246, and computer readable memory 248.

[0043] The transceiver 244 may be configured to receive information / input from one or more external devices or systems (e.g., user devices 204, servers 206, etc.) via the network 208. For example, the transceiver 244 may be configured to receive from the server 206 a mapping of each user device 204 (e.g., an identifier of each user device 204) and information associated with each user 104, a digital map of the geographic area where the vehicle 202 may be located, an expected path for each user 204, etc. In addition, the transceiver 244 may transmit notifications, requests, signals, etc. to the external device or system. In addition, the transceiver 244 may be configured to receive information / input from a vehicle component (such as a detection unit 242, a vehicle sensing system 234, one or more ECUs 216, etc.). In addition, the transceiver 244 may transmit a signal (e.g., a command signal) or a notification to a vehicle component, such as a BCM 222, etc.

[0044] The processor 246 and the memory 248 may be the same or similar to the processor 218 and the memory 220, respectively. In some aspects, the processor 246 may utilize the memory 248 to store programs and / or store data in code form to perform various aspects according to the present disclosure. The memory 248 may be a non-transitory computer-readable storage medium or memory that stores notification program code. In some aspects, the memory 248 may additionally store instructions / information / data obtained from the server 206, the user device 204, etc.

[0045] In operation, the processor 246 may obtain input from the detection unit 242 at a predefined frequency or when the vehicle 202 stops moving (e.g., when the vehicle 202 stops at a bus stop). The processor 246 may determine that the vehicle 202 may have stopped moving based on the input obtained from the VCU 212. In some aspects, the input obtained from the detection unit 242 may include an image of the user 104 captured by a vehicle internal camera and / or an external camera. In response to obtaining the input obtained from the detection unit 242, the processor 246 may execute one or more image processing algorithms (which may be pre-stored in the memory 248) to determine that a predetermined condition may be satisfied based on the input obtained from the detection unit 242. In some aspects, the predefined condition may be satisfied when the processor 246 determines that at least one of the users 104 may have left the vehicle 202.

[0046] In response to determining that a predefined condition may be satisfied or when the vehicle 202 stops moving, the processor 246 may transmit a command signal to the BCM 222 to activate / flash one or more exterior lights (e.g., parking lights, tail lights, headlights, etc.) for a predefined duration, as described above in conjunction with Figure 1The flashing of the vehicle exterior lights may provide an indication to other vehicles near the vehicle 202 to stop and allow the user 104 to conveniently exit the vehicle 202 and / or cross the road 106. In addition, in response to determining that the predefined condition may be satisfied, the processor 246 may determine a count of the users 104 who may have exited the vehicle 202 based on input (e.g., user images) obtained from the detection unit 242. For example, the processor 246 may determine that six users (e.g., users 104a, 104b, 104c, 104d, 104e, 104n) may have exited the vehicle 202 based on images obtained from the vehicle interior camera and the exterior camera.

[0047] In response to determining that the predefined conditions may be satisfied and the count of users 104 that may have exited the vehicle 202, the processor 246 may generate a virtual zone 302 around or around the vehicle 202, such as Figure 3A and Figure 3B . In some aspects, the virtual zone size may depend on the size of the vehicle 202, the size of the road 106, the count of users 104, etc. For example, when the count of users 104 is likely to be high and / or the road 106 is likely to have a large width, the virtual zone 302 may have a larger size. On the other hand, when the count of users 104 is likely to be low and / or the road 106 is likely to have a small width, the virtual zone 302 may have a relatively small size. In other aspects, the virtual zone size may be preset (and stored in the memory 248).

[0048] Virtual area 302 may have any shape. Figure 3A and Figure 3B In the exemplary aspect depicted in , the virtual area 302 has a rectangular shape; however, in other aspects, the virtual area 302 may be shaped as a circle, an ellipse, etc. The arrangement of the virtual area 302 relative to the vehicle 202 may depend on the road size and the position of the vehicle on the road 106. In an exemplary aspect, the processor 246 may generate and arrange the virtual area 302 around the vehicle 202 such that the virtual area 302 may cover the entire area of ​​the vehicle 202. In addition, the front edge and the rear edge of the virtual area 302 may be set at a first predefined distance "D1" from the front of the vehicle and a second predefined distance "D2" from the rear of the vehicle, respectively. In addition, the left edge and the right edge of the virtual area 302 may be set at the edge of the road 106 or slightly away from the edge of the road, such that the virtual area 302 may cover the entire road width "W", as shown in FIG. Figure 3A Although Figure 3A” D1 ” is depicted as being greater than “ D2 ”, but the present disclosure is not limited to such an arrangement of the virtual zone 302 around the vehicle 202 . In some aspects, the distance “ D1 ” may be equal to or less than the distance “ D2 ”. The distances “ D1 ” and “ D2 ” may depend on a number of factors, such as the count of users 104 , the size of the vehicle, the expected travel / movement path of each user 104 , etc.

[0049] In addition, as above combined Figure 1 As described, the virtual zone 302 may be associated with an expiration duration. The concept of an expiration duration associated with the virtual zone 302 has been described above. In some aspects, the expiration duration may depend on the size of the vehicle 202, the size of the road 106, the count of users 104, the expected duration that one or more of the plurality of users 104 may take to cross the road 106, the expected travel / movement path of each user 104, the expected movement speed of each user 104, etc. For example, when the count of users 104 may be large and / or the width "W" may be large, the expiration duration may be large, and when the count of users 104 may be small and / or the width "W" may be small, the expiration duration may be relatively small. In other aspects, the expiration duration may be preset (and the expiration duration stored in the memory 248).

[0050] As above combined Figure 1 As described, in some aspects, the processor 246 may generate a virtual area 302 on a digital map of a geographic area where the vehicle 202 may be located. In some aspects, the processor 246 may first obtain a digital map of the geographic area from the server 206, and then generate the virtual area 302 on the digital map. In other aspects, the processor 246 may not generate the virtual area 302 on the digital map. In other words, the virtual area generation does not depend on the availability of a digital map, and the processor 246 may also generate the virtual area 302 without a digital map.

[0051] In response to generating the virtual zone 302, the processor 246 may transmit information associated with the virtual zone 302 to one or more other vehicles 304a, 304b (eg, Figure 3A and Figure 3B As shown). The information associated with the virtual area 302 may include, but is not limited to, the virtual area shape and size, the associated expiration duration, the digital map (if the virtual area 302 is generated on a digital map), etc. Figure 1As described, the processor 246 may transmit information to the other vehicles 304a, 304b directly via V2V communication via the transceiver 244, or indirectly via cellular communication, V2V, or V2X communication via the server 206 and / or connected infrastructure. In the case of transmitting information indirectly to the other vehicles 304a, 304b, the processor 246 may first transmit the information to the server 204 / connected infrastructure, which may then transmit the information to the other vehicles 304a, 304b.

[0052] As above combined Figure 1 As described, the HMI associated with the other vehicles 304a, 304b may display the virtual zone 302 (on a digital map or otherwise) in response to receiving information from the processor 246 (directly or via the server 206 / connected infrastructure). When the virtual zone 302 begins to be displayed, the drivers associated with the other vehicles 304a, 304b may view the virtual zone 302 on the corresponding HMI screen. When the virtual zone 302 begins to be displayed on the HMI screen, the drivers may receive instructions that they should stop their vehicles near the beginning of the boundary of the virtual zone 302 and not enter the virtual zone 302 until the virtual zone 302 is being displayed on the HMI screen, or until the processor 246 continues to transmit information associated with the virtual zone 302. If one or both of the vehicles 304a, 304b are autonomous vehicles, the vehicles 304a, 304b may automatically stop near the beginning of the boundary of the virtual zone 302 when the vehicles 304a, 304b receive information associated with the virtual zone 302 from the processor 246. In this case, the vehicles 304a, 304b can automatically determine the boundaries of the virtual area 302 based on the received information, and can autonomously control vehicle movement so that the vehicles 304a, 304b do not enter the virtual area 302 until the processor 246 continues to transmit information associated with the virtual area 302.

[0053] In addition to transmitting information associated with the virtual zone 302 to other vehicles 304a, 304b (directly or indirectly), the processor 246 may also "monitor" the movement of each user 104 based on input obtained from the detection unit 242 / images obtained from the vehicle's external camera. By monitoring the user's movement, the processor 246 may determine whether each user 104 may have conveniently crossed / left the road 106. Specifically, based on the input obtained from the detection unit 242 and the count of users 104, the processor 246 may determine whether each user 104 has traveled a predefined distance away from the vehicle 202, or has crossed or left the road 106, or has left the virtual zone 302. In some aspects, when the user 104 leaves the virtual zone 302, the processor 246 may determine that the user 104 may have traveled a predefined distance away from the left edge or right edge of the virtual zone 302 (which may be parallel to the road edge) away from the vehicle 202.

[0054] In response to determining that each user 104 may not have left the virtual zone 302, the processor 246 may continue to transmit information associated with the virtual zone 302 to the other vehicles 304a, 304b, and the other vehicles 304a, 304b may continue to wait / not move (because their respective HMI screens may still be displaying the virtual zone 302). On the other hand, in response to determining that each user 104 has traveled a predefined distance away from the vehicle 202, or has crossed or left the road 106, or has left the virtual zone 302 (e.g., Figure 3B 304b), the processor 246 can stop transmitting information associated with the virtual zone 302 to other vehicles 304a, 304b, which can cause their corresponding HMI screens to remove or stop displaying the virtual zone 302. When the HMI screens associated with the vehicles 304a, 304b stop displaying the virtual zone 302, the corresponding drivers of these vehicles can obtain an indication that the vehicles 304a, 304b can move. If the vehicles 304a, 304b are autonomous vehicles, when the processor 246 stops transmitting information associated with the virtual zone 302 to the vehicles 304a, 304b, the vehicles 304a, 304b can automatically start vehicle movement. In this way, the processor 246 can ensure that other vehicles 304a, 304b do not move and do not enter the virtual zone 302 until all users 104 may have left the virtual zone 302 or the road 106, thereby enabling the users 104 to conveniently pass through or leave the road 106.

[0055] In another aspect, the processor 246 can monitor the movement of each user 104 based on the input obtained from the detection unit 242 even after the user 104 leaves the virtual area 302. In this case, the processor 246 can determine the position of each user 104 relative to the vehicle 202 based on the image captured by the vehicle's external camera, and can transmit the determined position of each user 104 to other vehicles 304a, 304b (and additional vehicles that may be near the vehicle 202) (via the transceiver 244). Drivers associated with other vehicles 304a, 304b can view the position of each user 104 on the corresponding HMI screen, and can manipulate the corresponding vehicle movement based on the position of each user 104. For example, the driver can slow down the vehicle when one or more users 104 may be near their vehicle (as determined by viewing the position of each user 104 on the HMI screen). If the vehicles 304a, 304b are autonomous vehicles, the vehicles 304a, 304b can automatically control the vehicle movement based on the position of each user 104 received from the transceiver 244.

[0056] Although the above description describes aspects in which the processor 246 determines the location of each user 104 based on images captured by a camera external to the vehicle, in additional aspects, the processor 246 may determine the location of each user 104 based on the real-time location of the user device 204 associated with each user 104 relative to the vehicle 202. In some aspects, the processor 246 may determine the real-time location of the user device 204 associated with each user 104 based on input obtained from a user device locator, which may be part of the detection unit 242.

[0057] In some aspects, when each user 104 boards or enters the vehicle 202 (e.g., at a school), the vehicle 202 can be communicatively coupled to each user device 204 via the BLEM 238. In response to being communicatively coupled to each user device 204, the vehicle 202 / processor 246 can obtain an identifier of each user device from the corresponding user device. The processor 246 can also obtain a mapping of each user device 204 (specifically, an identifier of each user device 204) and information associated with each user 104 from the server 206 to determine the information associated with each user 104. For example, the processor 246 can determine the name, bus stop location, etc. of each user 104 that may be present in the vehicle 202 in response to being communicatively coupled to the user device 204 and obtaining the mapping, as described above.

[0058] When the users 104 leave (or walk out of) the vehicle 202, the processor 246 may determine the real-time location of each user device 204 relative to the vehicle 202 in the geographic area where the vehicle 202 may be located (because each user device 204 may have been communicatively coupled to the vehicle 202, as described above) as each user 104 moves away from the virtual area 302. In some aspects, the processor 246 (or detection unit 242) may determine the real-time location of each user device 204 by calculating the distance and direction of each user device 204 relative to the real-time geographic location of the vehicle (which the processor 246 may obtain from the NAV receiver 236).

[0059] In response to determining the real-time location of each user device 204 (and therefore each associated user 104), the processor 246 may transmit / broadcast the location of each user device 204 / user 104 to the plurality of vehicles 402a, 402b, and 402c (e.g., Figure 4 ), the plurality of vehicles may be located in a geographic area 404 and near the vehicle 202. The geographic area size may be predetermined and may be pre-stored in the memory 248.

[0060] In some aspects, geographic area 404 may be an area where user 104 may be expected to be located or travel / walk when user 104 leaves virtual area 302 (as determined based on the expected path associated with each user 104 that vehicle 202 may obtain from server 206 or predict itself, as described above). For example, geographic area 404 may include an area where each user 104 may travel to reach their respective home from a bus stop as determined based on the expected path associated with each user 104. In addition, multiple vehicles 402a, 402b, and 402c may be those vehicles that may be present in geographic area 404. In some aspects, one or more of vehicles 402a, 402b, and 402c may be located at a distance from vehicle 202 (e.g., at other streets) such that the respective drivers of these vehicles may not be able to directly see / view user 104. In response to receiving the location of each user 104 from the processor 246, the corresponding HMI screens of the plurality of vehicles 402a, 402b, and 402c can display the received user location, thereby enabling the corresponding driver to view the location of each user 104, even if the driver cannot directly see the user 104. In response to viewing the location of each user 104, the driver can manipulate the corresponding vehicle movement (or if the vehicle is an autonomous vehicle, the vehicle can autonomously manipulate the vehicle movement), thereby enabling each user 104 to conveniently move in the geographic area, even after the user 104 leaves the virtual area 302. In some aspects, the corresponding HMI screens of the plurality of vehicles 402a, 402b, and 402c can display the location of each user 104 relative to the vehicle that received the user location, thereby enabling the driver to conveniently view the location of each user 104 relative to his own vehicle. For example, as shown in view 406a, the HMI associated with vehicle 402a displays the user location relative to vehicle 402a. Similarly, as shown in views 406b and 406c, the HMIs associated with vehicles 402b and 402c display the user's location relative to vehicles 402b and 402c, respectively.

[0061] In some aspects, the processor 246 / vehicle 202 can broadcast the location of each user device 204 / user 104 directly to the plurality of vehicles 402a, 402b, and 402c via V2V communications. In other aspects, the processor 246 / vehicle 202 can transmit the location of each user 104 to a server and connected infrastructure via V2I or V2X communications, and the server and connected infrastructure can then broadcast the location of each user 104 to the plurality of vehicles 402a, 402b, 402c.

[0062] In some aspects, by being communicatively coupled with the user devices 204, the processor 246 may also be configured to determine whether all users 104 associated with the bus stop may have left the vehicle 202 when the vehicle 202 arrives at the bus stop. In this case, when the vehicle 202 arrives at the bus stop, the processor 246 may determine the identifiers of the user devices 204 associated with the bus stop based on the mapping of the identifiers of each user device with the information associated with each user 104. The processor 246 may also track the movement of each such user device 204 as the associated user 104 leaves the vehicle 202. In response to determining that one or more such user devices (and thus the associated users) may still be in the vehicle 202 when the vehicle 202 may be about to start moving from the bus stop (e.g., when the associated users may have forgotten the bus stop or may have fallen asleep in the vehicle 202), the processor 246 and / or the vehicle operator may audibly notify / remind the users to leave the vehicle 202. In this manner, the processor 246 may prevent the users 104 from missing their respective bus stops.

[0063] In additional aspects, the processor 246 may use artificial intelligence (AI) / machine learning (ML) to track the movement of each user 104 (or user device 204) inside and outside the vehicle 202 based on a plurality of data, including, but not limited to, the presence or absence of the user at school on a particular day, activity patterns, etc. In addition, the processor 246 may use AI / ML (and / or data obtained from the server 206 or cloud) to dynamically train and update the map, boundaries, and / or size of the virtual zone 302 around the vehicle 202 based on one or more parameters, including, but not limited to, the presence of obstacles or structures (e.g., buildings, blocks, detours, etc.) near the vehicle 202 and / or bus stops, the time of day, the location of the bus stop, etc. For example, the virtual zone size may be different during school hours or class time on a weekday compared to the weekend.

[0064] Furthermore, even though the present disclosure is described using an example aspect in which the vehicle 202 is a school bus and the user 104 is a school child, the present disclosure is not limited to such an aspect. In alternative aspects, without departing from the scope of the present disclosure, the vehicle 202 may be a mobile seat pushed or driven by another user for an elderly person (in which case the user 104 may be an elderly person); a vehicle used in a nursing home, a nursery, a delivery / construction vehicle carrying multiple members who are dropped off at different locations / places; a bus, a company shuttle.

[0065] Figure 5A flow chart of an example method 500 for monitoring each user 104 associated with a vehicle 202 according to the present disclosure is depicted. Figure 5 The following process is exemplary and is not limited to the steps described below. Furthermore, alternative embodiments may include more or fewer steps than shown or described herein, and may include the steps in an order different from that described in the example embodiments below.

[0066] The method 500 begins at step 502. At step 504, the method 500 may include obtaining, by the processor 246, input from the detection unit 242. As described above, the processor 246 may obtain user images captured by the vehicle interior camera and the exterior camera. At step 506, the method 500 may include determining, by the processor 246, that a predefined condition may be satisfied based on the input obtained from the detection unit 242. As described above, the predefined condition may be satisfied when at least one of the users 104 may have left the vehicle 202.

[0067] At step 508, method 500 may include generating, by processor 246, virtual zone 302 around or around vehicle 202 in response to determining that the predefined conditions may be satisfied. At step 510, method 500 may include transmitting, by processor 246, information associated with virtual zone 302 to vehicles 304a and 304b.

[0068] Method 500 may end at step 512 .

[0069] In the above disclosure, reference has been made to the accompanying drawings that form a part of the above disclosure, which illustrate specific embodiments in which the present disclosure may be practiced. It should be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present disclosure. References to "one embodiment," "embodiment," "example embodiment," etc. in this specification indicate that the described embodiment may include specific features, structures, or characteristics, but each embodiment may not necessarily include the specific features, structures, or characteristics. In addition, such phrases do not necessarily refer to the same embodiment. In addition, when features, structures, or characteristics are described in conjunction with an embodiment, whether or not explicitly described, those skilled in the art will recognize such features, structures, or characteristics in conjunction with other embodiments.

[0070] In addition, where appropriate, the functions described herein may be performed in one or more of the following: hardware, software, firmware, digital components, or analog components. For example, one or more application specific integrated circuits (ASICs) may be programmed to perform one or more of the systems and programs described herein. Certain terms are used throughout the specification and claims to refer to specific system components. As will be appreciated by those skilled in the art, components may be referred to by different names. This document is not intended to distinguish between components that have different names but the same function.

[0071] It should also be understood that the word "example" as used herein is intended to be non-exclusive and non-limiting in nature. More specifically, the word "example" used herein indicates one of several examples, and it should be understood that no undue emphasis or preference is placed on the specific example described.

[0072] Computer-readable media (also referred to as processor-readable media) include any non-transitory (e.g., tangible) media that participate in providing data (e.g., instructions) that can be read by a computer (e.g., by a processor of a computer). Such media may take many forms, including, but not limited to, non-volatile media and volatile media. A computing device may include computer-executable instructions, where the instructions may be executable by one or more computing devices (such as those listed above) and stored on a computer-readable medium.

[0073] With respect to the processes, systems, methods, heuristics, etc. described herein, it should be understood that although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes may be practiced by performing the described steps in an order different from that described herein. It should also be understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. In other words, the descriptions of the processes herein are provided for the purpose of illustrating various embodiments and should in no way be construed as limiting the claims.

[0074] Therefore, it should be understood that the above description is intended to be illustrative and not restrictive. Upon reading the above description, many embodiments and applications other than the examples provided will be apparent. The scope should not be determined with reference to the above description, but should be determined with reference to the entire scope of the appended claims and equivalents to which such claims are entitled. It is expected and anticipated that the technology discussed herein will develop in the future, and the disclosed systems and methods will be incorporated into such future embodiments. In short, it should be understood that the present application is capable of modification and change.

[0075] Unless explicitly indicated to the contrary in this article, all terms used in the claims are intended to be given their ordinary meanings as understood by the skilled person described herein. Specifically, unless the claims state an explicit limitation to the contrary, the use of singular articles such as "one", "the", "said" etc. should be interpreted as describing one or more of the indicated elements. Unless otherwise specifically stated or understood in other ways within the context when used, conditional language such as, in particular, "can", "may", "may" or "can" is generally intended to express that certain embodiments may include certain features, elements and / or steps, while other embodiments may not include certain features, elements and / or steps. Therefore, such conditional language is generally not intended to imply that one or more embodiments require each feature, element and / or step in any way.

[0076] In one aspect of the present invention, the predefined condition is met when the user exits the first vehicle.

[0077] In one aspect of the invention, the information associated with the virtual area includes the virtual area shape and size and an expiration duration associated with the virtual area.

[0078] In one aspect of the invention, the method includes determining based on input that a user has traveled a predefined distance away from a first vehicle; and stopping transmitting information associated with the virtual area to a second vehicle in response to determining that the user has traveled a predefined distance away from the first vehicle.

[0079] In one aspect of the invention, determining that the user has traveled a predefined distance away from the first vehicle includes determining that the user has left the virtual zone.

[0080] According to the present invention, a non-transitory computer-readable storage medium is provided, on which instructions are stored, which, when executed by a processor, cause the processor to: obtain input from a detection unit of a first vehicle, wherein the detection unit is configured to detect the presence of a user in and near the first vehicle; determine based on the input that a predefined condition is satisfied; generate a virtual zone around the first vehicle in response to determining that the predefined condition is satisfied; and transmit information associated with the virtual zone to a second vehicle in response to generating the virtual zone.

Claims

1. A first vehicle, comprising: a detection unit configured to detect the presence of a user in and near the first vehicle; as well as a processor communicatively coupled to the detection unit, wherein the processor is configured to: obtaining an input from the detection unit; determining based on the input that a predefined condition is satisfied; generating a virtual zone around the first vehicle in response to determining that the predefined condition is satisfied; and Information associated with the virtual area is transmitted to a second vehicle in response to generating the virtual area. 2 . The first vehicle of claim 1 , wherein the detection unit comprises an interior vehicle camera and an exterior vehicle camera. 3 . The first vehicle of claim 2 , wherein the input comprises an image of the user captured by the interior vehicle camera and the exterior vehicle camera.

4. The first vehicle of claim 1, wherein the predefined condition is satisfied when the user exits the first vehicle. 5 . The first vehicle of claim 1 , wherein the information associated with the virtual zone includes a virtual zone shape and size and an expiration duration associated with the virtual zone.

6. The first vehicle of claim 5, wherein the processor generates the virtual area on a digital map of a geographic area that includes the first vehicle, and wherein the information further includes the digital map.

7. The first vehicle of claim 1, wherein the processor is further configured to: determining, based on the input, that the user has traveled a predefined distance away from the first vehicle; and Transmitting the information associated with the virtual area to the second vehicle is stopped in response to determining that the user has traveled the predefined distance away from the first vehicle.

8. The first vehicle of claim 7, wherein the processor determines that the user has traveled the predefined distance away from the first vehicle by determining that the user has left the virtual zone.

9. The first vehicle of claim 1, wherein the processor is further configured to: determining a user position relative to the first vehicle based on the input when the predefined condition is satisfied; and The user location is transmitted to the second vehicle.

10. The first vehicle of claim 1, wherein the detection unit is further configured to determine a location of a user device associated with the user relative to the first vehicle, and wherein the processor is further configured to: determining a user location in a geographic area including the first vehicle based on the location of the user device associated with the user relative to the vehicle when the predefined condition is satisfied; and The user location is transmitted to a plurality of vehicles located in the geographic area.

11. The first vehicle of claim 10, wherein the processor is further configured to transmit the user location to a server.

12. The first vehicle of claim 1, wherein the processor is further configured to: determining a count of additional users exiting the first vehicle based on the input obtained from the detection unit; determining that each additional user has left the virtual zone based on the input obtained from the detection unit and the count; and Transmitting the information associated with the virtual zone to the second vehicle is stopped in response to determining that each additional user has left the virtual zone.

13. A method for monitoring a user associated with a first vehicle, the method comprising: obtaining, by a processor, an input from a detection unit, wherein the detection unit is configured to detect the presence of the user in and near the first vehicle; determining, by the processor based on the input, that a predefined condition is satisfied; generating, by the processor, a virtual zone around the first vehicle in response to determining that the predefined condition is satisfied; as well as Information associated with the virtual zone is transmitted, by the processor, to a second vehicle in response to generating the virtual zone. The method of claim 13 , wherein the detection unit comprises an interior vehicle camera and an exterior vehicle camera.

15. The method of claim 14, wherein the input comprises an image of the user captured by the interior vehicle camera and the exterior vehicle camera.

Images