Detection framework inside the cabin

Abstract

Providing a detection framework within a vehicle cabin. 【Solution means】Exemplary operations include receiving a scan of the vehicle cabin, where the scan includes the interior of the vehicle and a spatial area outside the vehicle near at least one vehicle door, classifying and detecting the presence of at least one seat inside the vehicle, determining at least one occupant characteristic based on the scan, determining a cabin situation based on the at least one occupant characteristic, and communicating the cabin situation to a mobile device, including one or more of these.

Description

Background Art

[0001] Generally, vehicles or means of transportation, such as passenger cars, motorcycles, trucks, airplanes, trains, etc., provide transportation needs for passengers and / or goods in various ways. Functions related to the means of transportation can be identified and utilized by various computing devices, such as smartphones or computers, located on and / or away from the means of transportation.

Summary of the Invention

[0002] An exemplary embodiment provides a method including one or more of receiving a scan of a vehicle cabin, where the scan includes the interior of the vehicle and a spatial area outside the vehicle near at least one vehicle door, performing classification detection and presence detection related to at least one seat inside the vehicle, determining at least one passenger characteristic based on the scan, determining a cabin situation based on the at least one passenger characteristic, and communicating the cabin situation to a mobile device.

[0003] Another exemplary implementation provides a system including a memory communicatively connected to a processor, where the processor performs one or more of receiving a scan of a vehicle cabin, where the scan includes the interior of the vehicle and a spatial area outside the vehicle near at least one vehicle door, performing classification detection and presence detection related to at least one seat inside the vehicle, determining at least one passenger characteristic based on the scan, determining a cabin situation based on the at least one passenger characteristic, and communicating the cabin situation to a mobile device.

[0004] A further exemplary embodiment provides a computer-readable storage medium comprising instructions that, when read by a processor, cause the processor to receive a scan of a vehicle cabin, the scan including an interior of the vehicle and a spatial region outside the vehicle proximate to at least one vehicle door, perform classification detection and presence detection for at least one seat in the interior of the vehicle, determine at least one occupant characteristic based on the scan, determine a cabin situation based on the at least one occupant characteristic, and communicate the cabin situation to a mobile device, or perform one or more of the foregoing. BRIEF DESCRIPTION OF THE DRAWINGS

[0005]

Figure 1A

Figure 1B

Figure 1C

Figure 2A

Figure 2B

Figure 2C

Figure 2D

Figure 2E

Figure 2F

Figure 2G

Figure 2H

Figure 2I

Figure 2J

Figure 2K

Figure 2L

Figure 2M

Figure 2N

Figure 3A

Figure 3B

Figure 3C

Figure 4

Figure 5A

Figure 5B

Figure 6A

Figure 6B

Figure 6C

Figure 6D

Figure 7

[0006] It will be readily understood that the components of the present configuration generally described herein and shown in the figures can be arranged and designed in a wide variety of different configurations. Accordingly, the following detailed description of at least one embodiment of a method, apparatus, computer-readable storage medium, and system, as represented in the accompanying figures, is not intended to limit the scope of the present application as claimed, but merely represents the selected embodiments. The multiple embodiments described herein are not intended to limit the scope of the solution. The computer-readable storage medium can be a non-transitory computer-readable medium or a non-transitory computer-readable storage medium.

[0007] Communication between a transport means and a particular entity such as a remote server, another transport means, and a local computing device (e.g., a smartphone, a personal computer, a computer incorporated in a transport means, etc.) can be transmitted, received, and processed by one or more "components" that can be hardware, firmware, software, or a combination thereof. The component can be part of any of the entity or computing device or a particular other computing device. In one example, the determination of consensus related to a blockchain transaction can be made by one or more computing devices or components (which can be any element described and / or depicted herein) associated with the transport means, as well as by one or more of the components located external to or remote from the transport means.

[0008] Throughout this specification, the present functions, structures, or features may be combined in any suitable manner in one or more embodiments. For example, the use of phrases such as "exemplary embodiments", "some embodiments" throughout this specification, or other similar terms indicates that the specific functions, structures, or features described in relation to the embodiments may be included in at least one example. Thus, even if phrases such as "exemplary embodiments", "in some embodiments", "in other embodiments" or other similar terms appear throughout this specification, they do not necessarily all refer to the same group of embodiments, and the described functions, structures, or features may be combined in any suitable manner in one or more embodiments. In the figures, any connection between elements may enable one-way and / or two-way communication, even if the depicted connection is a one-way or two-way arrow. In this solution, the vehicle or means of transportation may include one or more of a passenger car, a truck, a walking area battery electric vehicle (BEV), an e-Palette, a fuel cell bus, a motorcycle, a scooter, a bicycle, a boat, a recreational vehicle, an airplane, and any object that can be used to transport people and / or goods from one place to another.

[0009] In addition, although the term "message" may be used in the description of embodiments, other types of network data such as packets, frames, and datagrams may also be used. Furthermore, specific types of messages and signaling may be depicted in preferred embodiments, but they are not limited to specific types of messages and signaling.

[0010] An exemplary embodiment provides a method, system, component, non-transitory computer-readable medium, device, and / or network that provides at least one of a means of transportation (also referred to herein as a vehicle or passenger vehicle), a data collection system, a data monitoring system, a verification system, an approval system, and a vehicle data distribution system. Vehicle status data received in the form of communication messages such as wireless data network communications and / or wired communication messages can be processed to identify the status of the vehicle / means of transportation and provide feedback regarding the status and / or changes of the means of transportation. In one example, a user profile can be applied to a specific means of transportation / vehicle to approve current vehicle events, service stops, subsequent vehicle rental services, and enable vehicle-to-vehicle communication at a service station.

[0011] In a communication infrastructure, a distributed database is a distributed storage system that includes multiple nodes that communicate with each other. A blockchain is an example of a distributed database that includes an append-only and immutable data structure (i.e., a distributed ledger) that can maintain records among untrusted parties. Untrusted parties are referred to herein as peers, nodes, or peer nodes. Each peer maintains a copy of the database records, and no peer can modify the database records without reaching consensus among the distributed peers. For example, peers can execute a consensus protocol to verify blockchain storage entries, group the storage entries into blocks, and construct a hash chain through the blocks. This process forms a ledger by ordering the storage entries as necessary for consistency. In a public or permissionless blockchain, anyone can participate without having specific identifying information. Public blockchains are involved in cryptocurrencies and can use consensus based on various protocols such as proof-of-work (PoW). Conversely, a permissioned blockchain database can guarantee transactions among a group of entities that share a common goal, such as businesses that exchange funds, goods, information, and the like, but do not fully trust or cannot trust each other. This solution can function in permissioned and / or permissionless blockchain settings.

[0012] A smart contract is a trusted distributed application that leverages the tamper-resistant properties of a shared or distributed ledger (which can be in the form of a blockchain) and the underlying agreement between member nodes, referred to as an endorsement or endorsement policy. Generally, blockchain entries are "approved" before being committed to the blockchain, while unapproved entries are ignored. With a typical endorsement policy, smart contract executable code can specify endorsers for entries in the form of a set of peer nodes required for endorsement. When a client sends an entry to a peer specified in the endorsement policy, the entry is executed to verify the entry. After verification, the entry enters an ordering phase, in which the consensus protocol is used to generate an ordered sequence of approved entries grouped into blocks.

[0013] A node is a communication entity in a blockchain system. A "node" can perform a logical function in the sense that multiple different types of nodes can operate on the same physical server. Nodes are grouped within a trust domain and associated with a logical entity that controls the nodes in various ways. Nodes can include different types such as a client or presenting client node that presents an entry call to an endorser (e.g., a peer) and broadcasts the entry proposal to an ordering service (e.g., an ordering node). Another type of node is a peer node, which can receive a client-presented entry, commit the entry, and maintain the state and a copy of the ledger of blockchain entries. A peer can also have the role of an endorser. An ordering service node or orderer is a node that performs a communication service for all nodes and implements delivery guarantees such as broadcasting to each of the peer nodes in the system when committing an entry to modify the world state of the blockchain. The world state can constitute the initial blockchain entries, usually including control and configuration information.

[0014] A ledger is an ordered and tamper-resistant record of all state transitions in a blockchain. State transitions can result from calls (i.e., entries) to smart contract executable code presented by participating parties (such as client nodes, ordering nodes, endorser nodes, peer nodes, etc.). As a result, an entry can produce a set of key-value pairs of assets that are committed to the ledger as one or more operands such as create, update, delete, and the like. The ledger includes a blockchain (also referred to as a chain) that is used to store the immutable and ordered records in blocks. The ledger also includes a state database that maintains the current state of the blockchain. Usually, there is one ledger per channel. Each peer node maintains a copy of the ledger for each channel of which it is a member.

[0015] A chain is an entry log constructed as a hash-linked block, where each block contains a sequence of N entries, where N is greater than or equal to 1. The block header contains the hash of the block's entries and the hash of the header of the previous block. In this way, all the entries in the ledger can be ordered and cryptographically linked together. Therefore, it is impossible to tamper with the ledger data without breaking the hash link. The hash of the latest added blockchain block represents all the entries on the chain that occurred before it, thereby ensuring that all peer nodes can be in a consistent and trusted state. The chain is stored in a peer node file system (i.e., local, attached storage, cloud, etc.) and can efficiently support the append-only nature of the blockchain workload.

[0016] The current state of the immutable ledger represents the latest values for all keys included in the chain's entry log. Since the current state represents the latest known key values on the channel, it may be referred to as the world state. Calls to smart contract executable code execute entries against the current state data of the ledger. To efficiently handle the interactions of the smart contract executable code, the latest values of the keys can be stored in a state database. The state database can simply be an indexed view of the chain's entry log and can therefore be regenerated from the chain at any time. The state database can be automatically restored (or generated if necessary) when the peer node starts up and before entries are accepted.

[0017] The blockchain differs from conventional databases in that it is a distributed, immutable, and secure storage rather than a central storage, where nodes must share changes to records in the storage. Some of the properties inherent in the blockchain and that help in the implementation of the blockchain include, but are not limited to, immutable ledger, smart contract, security, privacy, decentralization, consensus, endorsement, accessibility, and the like.

[0018] Exemplary embodiments provide services for a particular vehicle and / or user profiles applicable to the vehicle. For example, the user can be the owner of the vehicle or an operator of a vehicle owned by another party. The vehicle may require services at specific intervals, and the service request may require approval before permission to receive the service. Also, the service center can provide services to vehicles within a nearby area based on the vehicle's current route plan and the relative level of service requirements (e.g., emergency, critical, moderate, minor, etc.). The vehicle's requests can be monitored via one or more vehicle and / or road sensors or cameras that report the sensed data to a central controller computer device within and / or away from the vehicle. This data is transferred to a management server for consideration and operation. The sensors can be located on one or more of the interior of the transport means, the exterior of the transport means, on a fixed object away from the transport means, and on another transport means close to the transport means. The sensors can be associated with the speed of the transport means, the brakes of the transport means, the acceleration of the transport means, the fuel level, the service request, the gear shift of the transport means, the steering of the transport means, and the like. Sensors as described herein can also be devices such as wireless devices within and / or close to the transport means. Also, sensor information can be used to identify whether the vehicle is operating safely and whether the occupants are involved in any unexpected vehicle conditions, such as during vehicle access and / or utilization periods. Vehicle information collected before, during, and / or after vehicle operation can be identified and stored in a transaction on a shared / distributed ledger, and the transaction can be generated and committed to an immutable ledger as determined by a consortium that grants permissions and thus in a "distributed" manner by a blockchain membership group or the like.

[0019] Each party with an interest (i.e., owner, user, company, agency, etc.) may want to limit the exposure of private information, and thus, the blockchain and its immutability can be used to manage permissions for each specific user vehicle profile. Smart contracts can be used to provide compensation, quantify user profile scores / ratings / considerations, apply permissions for vehicle events, determine when services are needed, identify collision events and / or degradation events, identify events of safety concern, identify the parties to the event, and distribute to registered entities attempting to access the vehicle event data. Also, results can be identified and information can be shared among registered companies and / or individuals based on consensus techniques associated with the blockchain. Such techniques could not be implemented with a conventional centralized database.

[0020] To create maps of terrain and roads that the means of transportation can use for navigation and other purposes, the various driving systems of the present solution can utilize software, sensor arrays, and machine learning capabilities, as well as light detection and ranging (LiDAR) projectors, radars, ultrasonic sensors, and the like. In some embodiments, instead of LiDAR, GPS, maps, cameras, sensors, and the like can also be used in autonomous vehicles.

[0021] In certain embodiments, the solution includes authorizing a vehicle for a service via an automated and rapid authentication scheme. For example, driving to a charging station or fuel pump can be done by the vehicle's operator or an autonomous transportation means, and authorization to receive charge or fuel can be done without any delay if the authorization is received by the service and / or charging station. The vehicle can provide a communication signal that provides the vehicle's identification information, which is linked to a currently active profile that is authorized to receive services that can be later modified by compensation. Additional measures can be used to provide further authentication. For example, another identifier can be wirelessly transmitted from the user's device to the service center to replace or supplement a first authentication operation between the transportation means and the service center using additional authorization operations.

[0022] Shared and received data can be stored in a database, which generally maintains data in a particular location within a single database (e.g., a database server). This location is often a central computer, such as a desktop central processing unit (CPU), a server CPU, or a mainframe computer. Information stored in a centralized database is usually accessible from multiple different points. A centralized database is easy to manage, maintain, and control, and is particularly for security purposes because the centralized database is in a single location. Within a centralized database, the fact that all data is in a single storage location also means that a given data set has only one primary record, so data redundancy is minimized. A blockchain can be used to store data and transactions related to transportation means.

[0023] Any of the operations described in this specification may be performed by one or more processors (e.g., microprocessors, sensors, electronic control units (ECUs), head units, and the like) that may be located onboard or offboard the transport means. The one or more processors may communicate with other processors onboard or offboard in other transport means to utilize the data transmitted by the transport means. The one or more processors and the other processors may transmit data, receive data, and utilize this data to perform one or more of the operations described or depicted in this specification.

[0024] The detection of the presence and occupancy of a living being results in a cabin recognition that detects the presence of an occupant at each seat level, whether the presence is that of an adult or a child, and that can be used by a plurality of subsystems of a vehicle including a safety system.

[0025] Figure 1A shows a system 110 depicting the cabin recognition data flow between a vehicle and an external server 118. A Controller Area Network (CAN) is a bus standard designed to enable microcontrollers and devices to communicate with each other without a host computer via a message-based protocol. For each device connected to the CAN bus, data passes through the bus as frames that are transmitted continuously, and when multiple devices transmit simultaneously, the highest-priority device communication is prioritized while other devices wait. The cabin recognition upload data flow includes at least one of a scan data file and a CAN data file indicating the recognition of occupants. The cabin recognition logic depicted in Figure 1A can be executed fully or partially in one or more of a processor in the vehicle, a processor in a server that may be on-board or off-board in the vehicle, a device associated with the vehicle, or any other processor associated with the vehicle. Similarly, the memory utilized by one or more processors can be located on-board or off-board in the vehicle. The system periodically transmits CAN information connected to various in-vehicle ECUs to a cloud server. An Electronic Control Unit (ECU) is an embedded system that controls one or more of the electrical systems or subsystems within a vehicle. The ECU can include, but is not limited to, the management of the vehicle's engine, braking system, transmission system, door locks, dashboard, airbag system, infotainment system, electronic differential, and active suspension. The ECU is connected to the vehicle's CAN bus. An Electronic Control Module (ECM) is a type of electronic control unit that includes controlling actuators in an internal combustion engine to ensure optimal engine performance. The ECMs are often connected to each other through a central network of the vehicle, which may be referred to as a Controller Area Network (CAN). A Next Unit of Computing (NUC) is a barebone modular PC. The vehicle's processor can be the vehicle's ECM or another processor, such as a processor in an ECU, HU, NUC server, or another processor in the vehicle.Similarly, the memory used by one or more processors can be located on - board or off - board in the vehicle. The cabin recognition upload data can communicate directly or indirectly with the ECM via a bus such as the CAN bus 112. The ECMs are often connected to each other through a vehicle's central network, which can be referred to as CAN. The cabin situation ECU1 113 can collect scan upload data from the scanner 119 and can also be connected to the CAN bus 112, the NUC server 114, and the CAN ECU115. The scanner can be a radar scanner, an optical scanner, an acoustic scanner, or the like. The data communication module (DCM) can also be connected to the CAN bus that receives and communicates data received from the ECM. The DCM is a vehicle - internal communication device. The DCM116 can also be connected to the CAN bus 112 that receives and communicates the cabin recognition upload data received from the ECM. Vehicle - based processors such as ECUs collect the cabin recognition upload data and transmit data signals through the vehicle's CAN central gateway (CGW). In this example, the cabin recognition upload data is decoded inside the vehicle and communicated to the ECU and can be stored locally or on the server. The head unit (HU) is the command center of the vehicle audio system. The head unit can fuse the sensor data received from the ECM and can also be connected to the CAN bus. The cabin recognition upload data can be routed through the vehicle's HU to the network, where the data can be received directly from the ECU on - the - spot or can be collected in temporary storage by the head unit. The cabin recognition upload data is wirelessly transmitted from the vehicle by the CGW to a network 117 such as the cellular network towards the external server 118. The results of the cabin recognition upload and download are transmitted from the vehicle by the CGW to the external server 118 and / or the mobile device towards a network 117 such as the cellular network.

[0026] In one example, the cabin recognition system includes four main hardware components and three main software components. The hardware includes a NUC, a mobile device used to display information to the operator, a scanner such as a radar sensor, and / or the vehicle itself. The software includes a process running on the NUC, an API server running on the NUC, and / or an application running on the mobile device. The cabin recognition logic may be fully or partially executed on one or more of a processor in the vehicle, a processor in a server that may be on-board or off-board the vehicle, a device associated with the vehicle, or any other processor associated with the vehicle. Similarly, the memory utilized by one or more processors may be located on-board or off-board the vehicle. The vehicle's processor may be the vehicle's ECM or another processor, such as a processor in an ECU, HU, NUC server, or another processor in the vehicle. Similarly, the memory utilized by one or more processors may be located on-board or off-board the vehicle.

[0027] Figure 1B shows an example of cabin recognition logic 120. The mobile device 121 is connected to a server 122 (such as an API server). The mobile device receives the algorithm output and situation 130 and outputs a session start / stop command and a scan start and stop command 131 to the API server. The cabin recognition logic 120 of Figure 1B may be fully or partially executed on one or more of a processor in the vehicle, a processor in a server that may be on-board or off-board the vehicle, a device associated with the vehicle, or any other processor associated with the vehicle. Similarly, the memory utilized by one or more processors may be located on-board or off-board the vehicle. The vehicle's processor may be the vehicle's ECM or another processor, such as a processor in an ECU, HU, NUC server, or another processor in the vehicle.

[0028] In one example, the system includes a plurality of processors and a plurality of processes that share a common backend. The processors and processes monitor messages sent by controller 125 and extract or determine relevant information before exposing the relevant information to channels assigned to those algorithms that API server 122 monitors and uses.

[0029] In FIG. 1B, API server 122 is connected to a messaging system 123 (such as, for example, a Neural Autonomic Transport System (NATS)) that enables an application to communicate across multiple types of devices. The API server outputs session messages 132 and receives algorithm outputs 133.

[0030] API server 122 is a gateway through which information proceeds to and from mobile devices and further to and from a NUC (and vehicles as well as radar and other sensors). It listens and communicates via the same messaging system 123 as the processes and exposes algorithm information and other information via HTTP endpoints at which the application is executed. Additionally, it also has HTTP endpoints that the application uses to send commands.

[0031] Messaging system 123 is connected to various algorithm processes 124 in the vehicle to receive output messages, receives from the algorithm outputs 134 of the processes, and outputs session message sensor data and CAN data 135. Messaging system 123 is further connected to controller 125. Messaging system 123 receives sensor data 136 from the controller and sends session messages 137 to the controller.

[0032] The controller 125 is connected to the sensor system 128. The controller receives sensor data such as point clouds and algorithm outputs 140 from the sensor system 128, and transmits sensor start and stop commands 141 to the sensor system.

[0033] The messaging system 123 is connected to a CAN bus listener / receiver 126 that processes CAN signals coming in from the vehicle. In one example, a USB cable is connected to a NUC that the CAN bus listener / receiver reads using a library. When the CAN bus listener / receiver reads the various CAN messages being output, the CAN bus listener / receiver tracks the progression of the current state of the vehicle. Whenever the cabin recognition logic detects a change in the vehicle state (e.g., the seatbelt goes from being buckled to unbuckled), the cabin recognition logic always sends a message indicating that change over a channel to inform other processes. The messaging system 123 receives the decoded CAN data 138 from the CAN bus listener / receiver.

[0034] The messaging system 123 is connected to a CAN bus publisher 127 that processes outgoing CAN messages regarding the vehicle. As a CAN bus listener / receiver, there is an asynchronous processor that listens to the algorithm output channels and converts their outputs into CAN messages that are output on the same USB-to-CAN cable.

[0035] The CAN bus 129 is connected to the CAN bus listener / receiver 126 and transmits CAN messages 142 to the CAN bus listener / receiver. The CAN bus is also connected to the CAN bus publisher 127, and the CAN bus receives CAN messages 143 from the CAN bus publisher.

[0036] In one example, information flows from the vehicle's internal CAN network (which provides information regarding, for example, seat belt status and ignition status) and scan data (such as radar data generated by sensors), and this data is supplied to the NUC. The process receives this data and decodes it before communicating it to the API server. The API server then transmits this information to a mobile device, which displays it in an application.

[0037] In one example, commands can also flow from the mobile device 121. The command is transmitted to the server 122, and in response to the command, the server 122 can communicate the command to the process. The command includes functions such as starting and stopping a ride and starting a scan in case of inaccurate information.

[0038] Information and commands can also flow from the NUC to a sensor, which is, in one example, a radar sensor, and includes things like stopping and starting the detection or provision of information related to the vehicle's situation.

[0039] In one example, there are multiple processes executed in the NUC that handle the reception of sensor data such as radar data and CAN signals. The multiple processes also handle the communication that needs to be sent to sensors such as radar sensors. These processes are executed simultaneously and asynchronously, meaning that the processes are set up to respond independently to various events that can occur. Communication between the processes occurs via a networking function or library, which enables a process to publish a message on various channels and, accordingly, other processes to listen to and respond to that channel.

[0040] The cabin data endpoint updates at a specific interval (e.g., every second) to obtain update information from the algorithm output regarding the state of the cabin. The scan status endpoint returns information describing whether the system is currently in a scan state and is used to determine whether the scan has ended.

[0041] In one example, the ride start command endpoint warns the process to start listening for scan data (radar data), open a new log file, make a determination about it, and start outputting it on various channels. The ride end command endpoint tells the process to stop listening for radar data and tells server 122 to transmit the log to an external SSD drive. The scan start command endpoint enables the process and the radar to enter scan mode and record data. The feedback presentation command endpoint provides feedback to server 122 regarding the accuracy of the information server 122 is outputting. The shutdown command endpoint tells all programs in the NUC to shut down and, similarly, starts the shutdown in the NUC.

[0042] In one example, based on the CAN data input and classification output of the seat belt status, the system can detect the presence of an occupant (empty, paired, occupied) and whether the occupant has properly buckled their seat belt.

[0043] Warnings outside the designated seat determine when an occupant is sitting inappropriately in the seat, such as when multiple occupants are sitting in one seat, the occupant is lying down, facing sideways, or sitting too far forward. The warnings outside the designated seat can be executed fully or partially in one or more of a processor in the vehicle, a processor in a server that may be onboard or offboard the vehicle, a device associated with the vehicle, or any other processor associated with the vehicle. Similarly, the memory utilized by one or more processors can be located onboard or offboard the vehicle. The processor of the vehicle can be the vehicle's ECM or another processor, such as a processor in an ECU, HU, NUC server, or another processor in the vehicle.

[0044] In one example, footwell detection determines the presence of an object within the footwell up to approximately knee height. Footwell detection ensures that the warning indicates that the seat is not properly occupied when the occupant is leaning too far forward or the occupant in the previous row is leaning too far backward. The footwell detection algorithm detects an infant or small child left in the footwell area while the vehicle is parked after becoming unbuckled inside the vehicle. The output of the child presence detection can be in the form of a boolean indicator that fuses information from the vehicle, including CAN message door status, ignition state, and footwell detection, to generate a boolean indicator regarding whether a child is present in the vehicle cabin. The child presence detection and footwell detection algorithms can be executed fully or partially in one or more of a processor in the vehicle, a processor in a server that may be onboard or offboard the vehicle, a device associated with the vehicle, or any other processor associated with the vehicle. Similarly, the memory utilized by one or more processors can be located onboard or offboard the vehicle. The processor of the vehicle can be the vehicle's ECM or another processor, such as a processor in an ECU, HU, NUC server, or another processor in the vehicle.

[0045] In one example, the foot space detection algorithm utilizes a bounding box / cuboid via density-based spatial clustering to recognize a radar signal region that is approximately equivalent to anatomical structures such as the occupant's head and chest regions.

[0046] In another exemplary implementation, for example, the neural network present in the NUC is specifically designed to process, for example, point cloud data. This exemplary implementation generates a centralized neural network, and the centralized neural network can simultaneously predict, within one neural network, the presence of an adult or child and the seat-by-seat prediction regarding the occupancy of the foot space from a single point cloud. The neural network can be fully or partially executed in one or more of a processor in the vehicle, a processor in a server that may be present on-board or off-board in the vehicle, a device associated with the vehicle, or any other processor associated with the vehicle. Similarly, the memory utilized by one or more processors can be located on-board or off-board in the vehicle. The vehicle's processor can be the vehicle's ECM or another processor, such as a processor in an ECU, HU, NUC server, or another processor in the vehicle.

[0047] The detection of the presence of a child detects the presence of a child anywhere within the vehicle cabin. The footwell space detection algorithm can be used to detect infants or small children who are unbuckled within the vehicle and then left in the footwell area while the vehicle is parked. The output of the detection of the presence of a child can be in the form of a Boolean indicator, which fuses information from the vehicle, including CAN message door status, ignition state, and detection of the footwell space, to generate a Boolean indicator regarding whether a child is present within the vehicle cabin. The detection of the presence of a child and the footwell space detection algorithm can be executed fully or partially in one or more of a processor in the vehicle, a processor in a server that may be on-board or off-board the vehicle, a device associated with the vehicle, or any other processor associated with the vehicle. Similarly, the memory utilized by one or more processors can be located on-board or off-board the vehicle. The processor of the vehicle can be the vehicle's ECM or another processor, such as a processor in the ECU, HU, NUC server, or another processor in the vehicle.

[0048] The detection of the presence of a living being in the trunk detects the movement of a living body within the trunk space based on a scan of the trunk area. The detection of the presence of a living being in the trunk can be executed fully or partially in one or more of a processor in the vehicle, a processor in a server that may be on-board or off-board the vehicle, a device associated with the vehicle, or any other processor associated with the vehicle. Similarly, the memory utilized by one or more processors can be located on-board or off-board the vehicle. The processor of the vehicle can be the vehicle's ECM or another processor, such as a processor in the ECU, HU, NUC server, or another processor in the vehicle.

[0049] Intrusion detection detects very close activities around the vehicle, for example, via radar. Intrusion detection can be executed fully or partially in one or more of a processor in the vehicle, a processor in a server that may be present on-board or off-board in the vehicle, a device associated with the vehicle, or any other processor associated with the vehicle. Similarly, the memory utilized by the one or more processors can be located on-board or off-board in the vehicle. The processor of the vehicle can be the vehicle's ECM or another processor, for example, a processor in an ECU, HU, NUC server, or another processor in the vehicle.

[0050] Figure 1C shows an overview 150 of a further exemplary cabin recognition system. The system includes two main parts, namely a server part 151 and a client part 152 which is in one example a mobile device 154. The API 161, which is connected to the NUC server 153 and the CAN bus 156 that supplies CAN bus information 158 to the API 161, is at the center of the server part 151. The sensor 157 (for example, a radar sensor) supplies to the controller 159 and then to the API. The API receives information regarding the position 162, the passenger count 163, and is written in a specific language 164 as merely an example. On the server side, the NATS system 160 processes the messaging between the various components. The interface between the server part 151 and the client part 152 is called Step-1 155. The mobile device 154 that receives information regarding the seat occupancy 155 is on the client side 152. In this part, the display shows each seat, whether the seat is being used (for example, seats 3 162 and 5 163), and whether the footwell is occupied 164. Warnings are shown at part 156, the passenger count is shown at 157, the passengers in the designated seats 158, the scan part 159, and the accuracy checks 160 and 161 are shown. The cabin recognition logic can be executed fully or partially in one or more of a processor in the vehicle, a processor in a server that may be on-board or off-board the vehicle, a device associated with the vehicle, or any other processor associated with the vehicle. Similarly, the memory utilized by one or more processors can be located on-board or off-board the vehicle. The vehicle's processor can be the vehicle's ECM or another processor, for example, a processor in an ECU, HU, a NUC server, or another processor in the vehicle.

[0051] Flow diagrams described herein, such as FIG. 1B, FIG. 2C, FIG. 2D, FIG. 2E, FIG. 3A, FIG. 3B, and FIG. 3C, are separate examples, but can be of the same or different embodiments. Any of the operations in a flow diagram can be adopted in another flow diagram and shared with another flow diagram. The exemplary operations are not intended to limit any embodiment or the subject matter of the corresponding claims.

[0052] All flow diagrams and corresponding processes obtained from FIG. 1B, FIG. 2C, FIG. 2D, FIG. 3A, and FIG. 3B may be part of the same process or may share sub - processes with each other. Thus, it is important to note that while no single specific operation is required, the figures can be combined into a single preferred embodiment that performs specific operations from one exemplary process and one or more additional processes. All exemplary processes are related to the same physical system and can be used separately or interchangeably.

[0053] FIG. 2A shows a transportation means network diagram 200 according to an exemplary embodiment. The network comprises elements including a transportation means 202 including a processor 204 and a transportation means 202' including a processor 204'. The transportation means 202, 202' communicate with each other via the processor 204, 204' and other elements (not shown) including other elements capable of providing transceivers, transmitters, receivers, storage, sensors, and communication. Communication between the transportation means 202, 202' can occur directly, via a private network and / or a public network (not shown), or via other transportation means and elements comprising one or more of processors, memories, and software. Although described as a single transportation means and processor, multiple transportation means and processors can exist. One or more of the applications, functions, steps, solutions, etc. described and / or depicted herein can be utilized and / or provided by this element.

[0054] Figure 2B shows another transportation means network diagram 210 according to an exemplary embodiment. The network comprises elements including a transportation means 202 including a processor 204 and a transportation means 202' including a processor 204'. The transportation means 202 and 202' communicate with each other via the processors 204 and 204', and other elements (not shown) capable of providing transmitters, receivers, storage, sensors, and other elements for communication. The communication between the transportation means 202 and 202' can occur directly, via a private network and / or a public network (not shown), or via other transportation means and elements comprising one or more of a processor, memory, and software. The processors 204 and 204' can further communicate with one or more elements 230 including sensors 212, wired devices 214, wireless devices 216, databases 218, mobile phones 220, transportation means 222, computers 224, I / O devices 226, and voice applications 228. The processors 204 and 204' can further communicate with elements comprising one or more of a processor, memory, and software.

[0055] Although depicted as a single transportation means, processor, and element, multiple transportation means, processors, and elements can exist. Information or communication can occur to and / or from any of the processors 204 and 204' and the elements 230. For example, the mobile phone 220 can provide information to the processor 204 that can initiate an operation in the transportation means 202, can further provide information or additional information to the processor 204' that can initiate an operation in the transportation means 202', and can further provide information or additional information to the mobile phone 220, the transportation means 222, and / or the computer 224. One or more of the applications, functions, steps, solutions, etc. described and / or depicted herein can be utilized and / or provided by this element.

[0056] FIG. 2C shows yet another transportation means network diagram 240 according to an exemplary embodiment. The network comprises elements including a transportation means 202 that includes a processor 204 and a non-transitory computer-readable medium 242C. The processor 204 is communicatively connected to the computer-readable medium 242C and the element 230 (depicted in FIG. 2B). The transportation means 202 can be a transportation means, a server, or any device including a processor and a memory.

[0057] The processor 204 performs one or more of the following: receiving a scan of the vehicle cabin at 244C, where the scan includes the interior of the vehicle and the spatial area outside the vehicle near at least one vehicle door; performing classification detection and presence detection regarding at least one seat in the vehicle interior; determining at least one occupant characteristic based on the scan at 246C; determining a cabin situation based on the at least one occupant characteristic at 248C; and communicating the cabin situation to a mobile device at 250C. The scan of the cabin can be performed by radar, sonar, optical imaging, and the like. The reception of the scan can be performed by a controller, NUC server, API, ECU, and the like. In one example, the determination of the occupant characteristic can be performed by machine learning. The presence detection of a living being detects the presence of an occupant in each seat and outputs a Boolean variable (a value of 8 seats + 1 to capture all areas outside all seats) for each seat where occupancy is possible. For seats where detection is not possible, the Boolean variable remains 0. In one exemplary implementation, the point cloud center is determined for each seat via clustering using density-based spatial clustering of applications with noise (DBSCAN) and a predefined XYZ cuboid boundary to determine spatial occupancy and occupancy by seat. In another exemplary implementation, a neural network specifically designed to process point cloud data is utilized. This exemplary implementation generates a centralized neural network that can simultaneously predict, within a single neural network, predictions for each seat regarding the classification of presence, adult occupancy, child occupancy, and foot space occupancy from a single point cloud. The cabin situation is the classification and occupancy of each seat and space inside the vehicle and near the vehicle doors. The communication of the cabin situation to the mobile device can be wired or wireless, Wi-Fi, Bluetooth®, and the like. The cabin recognition logic can be fully or partially executed in one or more of a processor in the vehicle, a processor in a server that can be present on-board or off-board in the vehicle, a device associated with the vehicle, or any other processor associated with the vehicle.Similarly, the memory used by one or more processors can be located on-board or off-board in the vehicle. The vehicle's processor can be the vehicle's ECM or another processor, such as a processor in an ECU, HU, NUC server, or another processor in the vehicle.

[0058] FIG. 2D shows a further transportation means network diagram 250 according to an exemplary embodiment. The network comprises elements including a transportation means 202 including a processor 204 and a non-transitory computer-readable medium 242D. The processor 204 is communicatively connected to the computer-readable medium 242D and the element 230 (depicted in FIG. 2B). The transportation means 202 can be a transportation means, server, or any device including a processor and a memory.

[0059] In FIG. 2D, the processor 204 combines at least one occupant feature based on a scan with at least another occupant feature based on a set of data received from a controller area network 246D, and communicates logs of the scan and the cabin situation to a server via the controller area network 254D, where the server updates a cabin situation model based on the log of the scan 254D, and performs one or more of the foregoing. The occupant feature includes a spatial region inside the vehicle that is separated from at least one seat inside the vehicle (244D). The scan captures at least one point cloud centroid for at least one of the foot space and the head space of the cabin (248D), and uses a predefined cuboid boundary for at least one seat inside the vehicle to determine occupancy of the space and occupancy by seat (250D). Classification detection includes occupancy by adults, occupancy by children, and occupancy of the foot space (252D). The cabin recognition logic may be executed in whole or in part in one or more of a processor in the vehicle, a processor in a server that may be present on-board or off-board in the vehicle, a device associated with the vehicle, or any other processor associated with the vehicle. Similarly, the memory used by one or more processors may be located on-board or off-board in the vehicle. The processor of the vehicle may be the ECM of the vehicle or another processor, such as a processor in an ECU, HU, NUC server, or another processor in the vehicle.

[0060] In FIG. 2D, the occupant feature includes a spatial region outside the seat area (244D), and in one example, this includes warnings outside the designated seat that determine if an occupant is sitting inappropriately in the seat, such as multiple occupants sitting in one seat, an occupant lying down, facing sideways, or sitting too far forward.

[0061] In FIG. 2D, combining at least one occupant feature based on a scan with at least one other occupant feature 246D is based in part on the vehicle's internal CAN network (such as seat belt status, ignition status, etc.) and the information flow from scan data such as radar data generated by sensors. These two sources are supplied to the NUC. The process receives this data and performs relevant decisions and decoding before transmitting it to the API server. The API server then sends this information to the mobile device, which displays it in an application.

[0062] In FIG. 2D, the capture of at least one point cloud center regarding at least one of the footwell space and headspace in the cabin 248D is, in one exemplary implementation, based on the point cloud center determined through clustering using density-based spatial clustering (DBSCAN) with noise and predefined XYZ cuboid boundaries for each seat to determine space occupancy and occupancy by seat. In another exemplary implementation, a neural network specifically designed to process point cloud data is utilized. This exemplary implementation generates a centralized neural network that can simultaneously predict, within one neural network, predictions for each seat regarding the classification of presence, adult occupancy, child occupancy, and footwell space occupancy from a single point cloud.

[0063] In FIG. 2D, the use of a bounding box / cuboid via density-based spatial clustering 250D enables the recognition of angular anatomical structures (e.g., the head, chest regions of the occupant) to determine space occupancy and occupancy by seat.

[0064] In Figure 2D, the classification detection includes adult occupancy, child occupancy, and foot space occupancy. 252D can be based on an architecture that processes point cloud data in a way that the rotation and order are invariant, for example. From a single point cloud, a centralized neural network can simultaneously predict, within the same neural network, the predictions for each seat regarding the presence, adult occupancy, child occupancy, and foot space occupancy classifications.

[0065] In Figure 2D, the server updates the cabin situation model based on the scan log. 254D is, for example, an update of a machine learning model based on a larger training data set provided by the log.

[0066] Figure 2E shows an additional transportation means network diagram 260 according to an exemplary embodiment. Referring to Figure 2E, the network diagram 260 includes a transportation means 202 connected to other transportation means 202' and an update server node 203 in a blockchain network 206. The transportation means 202 and 202' can represent transportation means / vehicles. The blockchain network 206 can have a ledger 208 that stores software update verification data and a source 207 of verification for future use (e.g., in an audit).

[0067] This example describes only one transportation means 202 in detail, but a plurality of such nodes can be connected to the blockchain 206. It should be understood that the transportation means 202 may include additional components, and some of the components described herein may be removed and / or modified without departing from the scope of the present application. The transportation means 202 may have a computing device or a server computer, or the like, and may include a processor 204, and the processor 204 may be a semiconductor-based microprocessor, a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), and / or another hardware device. Although a single processor 204 is depicted, it should be understood that the transportation means 202 may include a plurality of processors, a plurality of cores, or the like without departing from the scope of the present application. The transportation means 202 may be a transportation means, a server, or any device including a processor and a memory.

[0068] The processor 204 is to receive an event confirmation from one or more of the elements described or depicted herein, the confirmation comprising a blockchain consensus between peers represented by any of the elements, 244E, and to execute a smart contract to record the confirmation in the blockchain based on the blockchain consensus, 246E, one or more of which are to be performed. The consensus is formed between any element 230 and / or one or more of any of the elements described or depicted herein, including, for example, a transportation means, a server, or a wireless device. In another example, the transportation means 202 may be any element 230 and / or one or more of any of the elements described or depicted herein, including, for example, a server or a wireless device.

[0069] The processor and / or the computer-readable medium 242E may be wholly or partially present inside or outside the transport means. The steps or functions stored in the computer-readable medium 242E may be wholly or partially performed in any order by either the processor and / or an element. Further, additions, omissions, combinations, subsequent executions, etc. may be made to one or more steps or functions.

[0070] FIG. 2F shows FIG. 265 depicting the power supply of one or more elements. In one example, the transport means 266 may provide the power stored in its battery to one or more elements including other transport means 268, a charging station 270, and an electrical grid 272. The electrical grid 272 is connected to one or more of the charging stations 270, and the charging stations 270 may be connected to one or more of the transport means 268. This configuration enables the distribution of electricity / power received from the transport means 266. The transport means 266 may also communicate with other transport means 268 via vehicle-to-vehicle (V2V) technology, cellular communication, WiFi, and the like. The transport means 266 may also communicate with other transport means 268, the charging station 270, and / or the electrical grid 272 in a wireless and / or wired manner. In one example, the transport means 266 is routed (or routes itself) to the electrical grid 272, the charging station 270, or other transport means 268 in a safe and efficient manner. Using one or more embodiments of the present solution, the transport means 266 may provide energy to one or more of the elements described herein in various advantageous ways as described and / or depicted herein. Further, it may enhance the safety and efficiency of the transport means and may have a beneficial impact on the environment as described and / or depicted herein.

[0071] The term "energy" can be used to indicate any form of energy received, stored, used, shared, and / or lost by a means of transportation. Energy can be referred to in conjunction with the voltage source and / or current supply of the charge provided from an entity to a means of transportation during a charging / usage operation. Energy can also be in the form of fossil fuels (e.g., for use in hybrid means of transportation), or alternative power sources including lithium-based, nickel-based, hydrogen fuel cells, atomic / nuclear energy, nuclear fusion-based energy sources, and energy generated on-the-fly during an energy sharing and / or usage operation that increases or decreases the energy level of one or more means of transportation at a given time, including but not limited to these.

[0072] In one example, the charging station 270 manages the amount of energy transmitted from the means of transportation 266 such that sufficient charge remains in the means of transportation 266 to reach the destination. In one example, a wireless connection is used to wirelessly direct the amount of energy transmission between the means of transportation 268, where both means of transportation may be moving. In one embodiment, wireless charging can occur via a fixed charger and the battery of the means of transportation that are aligned with each other (such as a charging mat in a garage or parking space). In one example, an unutilized vehicle such as the vehicle 266 (which may be autonomous) is directed to provide a certain amount of energy to the charging station 270 and return to its original location (e.g., its original location or a different destination). In one example, a mobile energy storage unit (not shown) collects surplus energy from at least one other means of transportation 268 and is used to transmit the stored surplus energy at the charging station 270. In one example, factors such as distance, time, and traffic conditions, road conditions, environmental / weather conditions, the condition of the vehicle (such as weight), the schedule of the passengers during vehicle use, and the expected schedule of the passengers waiting for the vehicle determine the amount of energy transmitted to the charging station 270. In one example, the means of transportation 268, the charging station 270, and / or the electrical grid 272 can provide energy to the means of transportation 266.

[0073] In one embodiment, a location such as a building, a residence, or the like (not depicted) is communicatively connected to one or more of the electrical grid 272, the transportation means 266, and / or the charging station 270. The rate at which electricity flows through one or more of the location, the transportation means 266, and other transportation means 268 is modified according to external conditions such as weather. For example, when the external temperature is extremely high or extremely low, increasing the likelihood of a power outage, the flow of electricity to the connected vehicles 266 / 268 is slowed to help minimize the likelihood of a power outage.

[0074] In one example, the solutions described and depicted herein can be used to determine the impact of the load on the transportation means and / or system, provide energy to the transportation means and / or system based on future demand and / or priorities, provide information between the device including the module and the vehicle, and enable the processor of the device to communicate wirelessly with the vehicle regarding the amount of energy stored in the vehicle's battery. In one example, the solution can also be used to provide a charge from the transportation means to the location based on factors such as the temperature of the location, the cost of energy, and the power level of the location. In one example, the solution can also be used to manage the amount of energy remaining in the transportation means after a portion of the charge has been transmitted to the charging station. In one example, the solution can also be used to notify the vehicle to provide the amount of energy of the battery in the transportation means, and the amount of energy transmitted is based on the distance of the transportation means to the module receiving the energy.

[0075] In one example, the solution may also be utilized to use a mobile energy storage unit, which moves to a means of transportation that has excess energy and stores the energy deposited into the electrical grid using a determined route. In one example, the solution may also be utilized to determine the priority of the decision of the means of transportation regarding the demand for providing energy to the grid, and the priority of the current demand regarding the means of transportation, for example, the priority of passengers or future passengers or current cargo or future cargo. In one example, the solution may also be utilized to determine that when the vehicle is not in use, the vehicle is steered to a location to discharge excess energy into the energy grid and then return to the previous location. In one example, the solution may also be utilized to determine the amount of energy required by a means of transportation based on one or more conditions such as weather, traffic, road conditions, the condition of a passenger vehicle, and passengers and / or articles within another means of transportation, and route the means of transportation to another means of transportation to provide energy by instructing the means of transportation to provide energy. In one example, the solution may also be utilized to transmit energy from one moving vehicle to another moving vehicle. In one example, the solution may also be utilized to extract energy by a means of transportation based on the energy consumed by the means of transportation to reach a location to meet another means of transportation and provide services, and the estimated energy consumption to return to the original location. In one example, the solution may also be utilized to provide the remaining distance required to reach a charging station, and the charging station determines the amount of energy extracted from the means of transportation, and the remaining charge amount is based on the remaining distance. In one example, the solution may also be utilized to manage a means of transportation that is simultaneously charged by both a charging station with a wired connection and another means of transportation with a wireless connection, etc., at more than one point simultaneously.In one example, the solution may also be utilized to apply priorities to the distribution of energy to the transportation means, where the priorities are given to the transportation means that provide a portion of the stored charge of the transportation means to another entity such as the electrical grid, a residence, and the like.

[0076] In one embodiment, the transportation means 266 and 268 can be utilized as bi-directional transportation means. The bi-directional transportation means can function as a mobile microgrid that can assist in supplying power to the grid 272 and / or reduce power consumption when the grid is stressed. In addition to receiving charge for the transportation means, the bi-directional transportation means incorporates bi-directional charging, and the transportation means can take energy from the transportation means and "push" the energy back to the grid 272, which is otherwise referred to as "V2G" in other cases. In bi-directional charging, electricity flows in both the direction to the transportation means and the direction from the transportation means. When the transportation means is being charged, alternating current (AC) electricity from the grid 272 is converted to direct current (DC). This can be done by one or more of the converters in the converter of the transportation means itself or the charger 270. The energy stored in the battery of the transportation means can be sent back to the grid in the opposite direction. The energy is converted from DC to AC through a converter, typically located in the charger 270, which is otherwise referred to as a bi-directional charger in other cases. Further, the solution as described and depicted with respect to FIG. 2F can be utilized in this network and / or system, as well as other networks and / or systems.

[0077] Figure 2G is Figure 275 showing the interconnections between different elements. This solution can be stored and / or executed in whole or in part on one or more computing devices 278’, 279’, 281’, 282’, 283’, 284’, 276’, 285’, 287’, and 277’ associated with various entities and all communicably connected to communicate with network 286, and / or by said one or more computing devices. Database 287 is communicably connected to the network and enables the storage and retrieval of data. In one example, the database is an immutable ledger. One or more of the various entities can be transportation means 276, one or more service providers 279, one or more public buildings 281, one or more transportation infrastructures 282, one or more residential houses 283, an electric grid / charging station 284, a microphone 285, and / or another transportation means 277. Other entities and / or devices such as one or more private users using a smartphone 278, a laptop 280, an augmented reality (AR) device, a virtual reality (VR) device, and / or any wearable device can also cooperate with this solution. The smartphone 278, the laptop 280, the microphone 285, and other devices can be connected to one or more of the connected computing devices 278’, 279’, 281’, 282’, 283’, 284’, 276’, 285’, 287’, and 277’. One or more public buildings 281 can include various institutions. One or more public buildings 281 can utilize computing device 281’. One or more service providers 279 can include sales agencies, towing truck services, collision centers, or other repair shops. One or more service providers 279 can utilize computing device 279’. These various computer devices can be connected to each other directly and / or communicably via a wired network, a wireless network, a blockchain network, and the like. In one example, the microphone 285 can be utilized as a virtual assistant.In one example, the one or more transportation infrastructures 282 may include one or more traffic signals, one or more sensors including one or more cameras, vehicle speed sensors or traffic sensors, and / or other transportation infrastructures. The one or more transportation infrastructures 282 may utilize a computing device 282’.

[0078] In one example, the transportation means 277 / 276 can transport people, objects, permanently or temporarily attached devices, and the like. In one example, the transportation means 277 can communicate with the transportation means 276 via V2V communication through the computers 276’ and 277’ associated with each transportation means, and can be referred to as transportation means, passenger cars, vehicles, automobiles, and the like. The transportation means 276 / 277 can be a self-propelled wheeled vehicle such as a passenger car, a sports utility vehicle, a truck, a bus, a wagon, or other motor or battery-driven, or fuel cell-driven transportation means. For example, the transportation means 276 / 277 can be an electric vehicle, a hybrid vehicle, a hydrogen fuel cell vehicle, a plug-in hybrid vehicle, or any other type of vehicle having a fuel cell stack, a motor, and / or a generator. Other examples of vehicles include bicycles, scooters, trains, airplanes, or boats, and any other form of vehicle capable of transportation. The transportation means 276 / 277 can be semi-autonomous or autonomous. For example, the transportation means 276 / 277 can be self-piloted and operated without human input. An autonomous vehicle can have one or more sensors and / or a navigation unit and use them to drive autonomously.

[0079] In one example, the solution described and depicted herein can be utilized to determine access to a means of transportation via blockchain consensus. In one example, the solution can also be utilized to perform profile verification before allowing a passenger to use the means of transportation. In one example, the solution can also be utilized to indicate (visually but also in another example verbally etc.) on or from the means of transportation to the means of transportation actions that a user needs to perform and that need to be confirmed as being correct actions (which can be pre-recorded). In one example, the solution can also be utilized to split data and provide the means of transportation with the ability to determine a method of distributing a portion of the split data with a lower risk level in a safe driving environment to the passenger and, after the passenger has left the means of transportation, distributing the remaining portion of the split data with a higher risk level to the passenger later, based on the risk level associated with the data and the driving environment. In one example, the solution can also be utilized to use blockchain and / or smart contracts to process the movement of vehicles across (country / state / etc.) boundaries and apply the rules of the new area to the vehicles.

[0080] In one example, the solution can also be utilized to enable the transport means to continue operating outside the boundary when a consensus is reached by the transport means based on the operation of the transport means and the characteristics of the occupants of the transport means. In one example, the solution can also analyze the available data upload / download speed of the transport means, the size of the file, and the speed / direction in which the transport means is moving to determine the distance required to complete the data upload / download and can be utilized to assign a secure area boundary for the data upload / download to be performed. In one example, the solution can also perform normally dangerous maneuvers in a safe manner and instruct the transport means of interest and other nearby transport means to enable the transport means of interest to exit in a safe manner, such as when the system determines that an exit is approaching or when the transport means appears not to be ready to exit (e.g., is in the wrong lane or is moving at a speed not suitable for exiting in the future). In one example, the solution can also be utilized to verify the diagnosis of the other transport means using one or more vehicles while both the one or more vehicles and the other transport means are moving.

[0081] In one example, the solution can also be used to detect the use of lanes at a certain location and time, and inform the passengers of the means of transportation or give instructions to the means of transportation to recommend or not recommend a lane change. In one example, the solution can also be used to eliminate the need to send information via email and the need for the driver / passenger to respond by making payments via email or directly. In one example, the solution can also be used to provide services to the passengers of the means of transportation, the provided services are based on subscriptions, and permissions are obtained from other means of transportation connected to the passenger's profile. In one example, the solution can also be used to record changes in the state of a lent object. In one example, the solution can also be used to request a blockchain consensus from other means of transportation near a damaged means of transportation. In one example, the solution can also be used to receive media from a server such as an insurance entity server or a computer of a means of transportation that may be related to an accident. The server accesses one or more media files, accesses the damage to the means of transportation, and stores the damage assessment on the blockchain. In one example, the solution can also be used to obtain a consensus and determine the severity of an event from a number of devices at various times prior to the event related to the means of transportation.

[0082] In one example, the solution can also be used to solve the problem of lack of video evidence regarding an accident related to the means of transportation. This solution details an inquiry by the means of transportation involved in the accident regarding media related to the accident from other means of transportation that may have been near the accident. In one example, the solution can also be used to record specific parts of a damaged means of transportation using the means of transportation and other devices (e.g., a pedestrian's mobile phone, a streetlight camera, etc.).

[0083] In one example, the solution can also be used to warn a passenger when the transport means is being steered towards a dangerous area and / or event, and to notify the passenger or a central controller of the transport means about a possible dangerous area that is on or near the current path of the transport means. In one example, the solution can also be used to detect when at least one other transport means is being used to assist in decelerating the transport means so that the impact on traffic is minimized when the transport means is moving at high speed. In one example, the solution can also be used to identify a dangerous driving situation, where media is captured by a vehicle involved in the dangerous driving situation. A geofence is established based on the distance of the dangerous driving situation, and further media is captured by at least one other vehicle within the established geofence. In one example, the solution can also be used to send a notification to one or more passengers of the transport means that the transport means is approaching a traffic control sign on the road, and then to receive an indication of bad driving from other nearby transport means if the transport means goes beyond the sign. In one example, the solution can also be used to partially disable the transport means by (in certain embodiments) limiting the speed, limiting the ability to approach another vehicle, limiting the speed to a maximum value, and allowing only a given number of miles (about 1.609 km) per period.

[0084] In one example, the solution can also be utilized to overcome the need for dependence on software updates and to correct problems associated with the transport means when the transport means is not operating correctly. Through the observation of other transport means on the route, the server receives data from a plurality of other transport means that may have observed dangerous or incorrect operation of the transport means. Through analysis, the observation can result in a notification to the transport means when the data suggests dangerous or incorrect operation. In one example, the solution can also be utilized to provide a notification between the transport means and a dangerous situation that may involve persons unrelated to the transport means. In one example, the solution can also be utilized to transmit data to the server by either a device associated with an accident of the transport means or a device near the accident. Based on the severity of the accident or near the accident, the server notifies the sender of the data. In one example, the solution can also be utilized to provide recommendations about the operation of the transport means to either the driver or passenger of the transport means based on the analysis of the data. In one example, the solution can also be utilized to establish a geopreference associated with a physical structure to determine the liability for payment for the transport means. In one example, the solution can also be utilized to adjust whether a vehicle can be disembarked at a location using both the current state and the proposed future state of the location and the navigation destinations of other vehicles. In one example, the solution can also be utilized to adjust the ability to automatically prepare for the disembarkation of a vehicle at a location such as a transport means rental entity.

[0085] In one example, the solution may also be utilized to move a transportation means to another location based on user events. More specifically, the system tracks the user's device and modifies the transportation means to move closer to the user based on the result of the original or modified event. In one example, the solution may also be utilized to enable verification of available locations within an area through the transportation means present within the area. An approximate time when a location may become available is also determined based on verification from the transportation means present. In one example, the solution may also be utilized to move the transportation means to a closer parking space if a certain parking space becomes available and the elapsed time from the first parking is less than the average time of the event. Further, when the event is completed or depending on the location of a device associated with at least one passenger of the transportation means, the transportation means is moved to a final parking space. In one example, the solution may also be utilized to plan for parking prior to approaching congestion. The system interacts with the transportation means to provide some service below the regular rate and / or guide the transportation means to an alternative parking location based on the priority of the transportation means, thereby improving the optimization of the parking situation prior to arrival.

[0086] In one example, the solution can also be used to sell fractional ownership of transportation means or to determine prices and availability for rideshare use. In one example, the solution can also be used to provide an accurate and timely report of the sales activities of a sales agency that is far superior to what is currently available. In one example, the solution can also be used to enable a sales agency to claim assets on a blockchain. By using a blockchain, consensus is obtained before any asset is transferred. Further, the process is automated and payments can be initiated on the blockchain. In one example, the solution can also be used to prepare an agreement made with multiple entities (such as a service center), consensus is obtained, and an operation (such as a diagnosis) is performed. In one example, the solution can also be used to associate digital keys with multiple users. The first user can be an operator of the transportation means, and the second user can be a party responsible for the transportation means. The key is approved by a server, where the proximity of the key is verified against the location of the service provider. In one example, the solution can also be used to determine the services required at the destination of the transportation means. The location of one or more service sites that can provide the required services, which are within the area on the route to the destination and where the execution of the services is available, is located. The navigation of the transportation means is updated at the location of the determined service site. A smart contract containing a compensation value for the service is identified, and the blockchain transaction is stored in the distributed ledger for the transaction.

[0087] In one example, the solution can also be used to associate the service provider's means of transportation with the profile of the occupants of the means of transportation to determine services and items that may be of interest to the occupants within the means of transportation. The services and items are determined by the occupants' history and / or preferences. The means of transportation then receives an offer from the service provider's means of transportation and, in another example, meets with the means of transportation providing the service / item. In one example, the solution can also be used to detect a range of means of transportation and send an offer of a service (such as an offer of maintenance, an offer of a product, or the like) to the means of transportation. An agreement is made between the system and the means of transportation, and the service provider is selected by the system to provide the agreement. In one example, the solution can also be used to assign one or more means of transportation as road administrators, and the road administrators assist in traffic regulation. The road administrators can generate road displays (such as traffic lights, displays, sounds, etc.) to assist in the flow of traffic. In one example, the solution can also be used to warn the driver of the means of transportation by a device, which can be a traffic light or can be near an intersection. The warning is sent in the event of an event such as when the traffic light turns green and the means of transportation in front of the list of means of transportation does not move.

[0088] Figure 2H is another block diagram 290 showing the interconnections between different elements in an example. Transportation means 276 is presented, including ECUs 295, 296 and a head unit (otherwise known as an infotainment system in other cases) 297. An electronic control unit (ECU) is a system incorporated into automotive electronics that controls one or more of the electrical systems or subsystems within the transportation means. The ECU can include, but is not limited to, management of the transportation means' engine, braking system, transmission system, door locks, dashboard, airbag system, infotainment system, electronic differential, and active suspension. The ECU is connected to the transportation means' controller area network (CAN) bus 294. The ECU can also communicate with the transportation means' computer 298 via the CAN bus 294. The transportation means' processor / sensor 298 (such as the transportation means' computer) can communicate with external elements such as server 293 via a network 292 (such as the Internet). Each ECU 295, 296 and head unit 297 can include its own security policy. The security policy defines the allowed processes that are executable in an appropriate context. In one example, the security policy can be provided, in part or in whole, in the transportation means' computer 298.

[0089] ECUs 295, 296, and the head unit 297 may each include a custom security feature element 299 that defines an approved process and the context in which the operation of that process is permitted. By context-based approval of whether a process is executable, the ECU can maintain secure operation and prevent unauthorized access from elements such as the vehicle's controller area network (CAN bus). If the ECU encounters an unauthorized process, the ECU may block the process from operating. Automotive ECUs may use various contexts such as the proximity context of nearby objects, the distance to approaching objects, speed, the trajectory relative to other moving objects, indication of whether the vehicle is moving or parked, the current speed of the vehicle, the operating context such as the transmission state, devices connected to the vehicle via wireless protocols, the use of infotainment, cruise control, parking assistance, driver assistance, etc., the location-based context, and / or other contexts to determine whether a process is operating within its permitted boundaries.

[0090] In one example, the solutions described and depicted herein can be utilized to partially disable a means of transportation by, in certain embodiments, restricting speed, restricting the ability to approach another vehicle, restricting speed to a maximum value, and allowing only a given number of miles (about 1.609 km) per period. In one example, the solution can also be utilized to facilitate the exchange of ownership of a vehicle using a blockchain, and data is sent to a server by either a device associated with an accident involving the means of transportation or a device near the accident. Based on the severity of the accident or near the accident, the server notifies the sender of the data. In one example, the solution can also be utilized to assist a means of transportation in avoiding an accident, such as when the means of transportation is involved in an accident, by a server that queries other means of transportation near the accident. The server attempts to obtain data from the other means of transportation, enabling the server to gain an understanding of the nature of the accident from multiple perspectives. In one example, the solution can also be utilized to determine that the sound from a means of transportation is abnormal and send data related to the sound and the location of the possible source to a server, and the server can determine the possible cause and avoid a potentially dangerous situation. In one example, the solution can also be utilized to establish a boundary of a location via a system when a means of transportation is involved in an accident. This boundary is based on the decibels associated with the accident. Multimedia content for devices within the boundary is obtained to assist in further understanding the unfolding of the accident. In one example, the solution can also be utilized to associate a vehicle with an accident and then capture media obtained by a device near the location of the accident. The captured media is stored as a media segment. The media segment is sent to another computing device that constructs a sound profile of the accident. This sound profile will assist in understanding further details surrounding the accident.

[0091] In one example, the solution may also utilize sensors to record audio, video, movement, etc. when a transport means contacts or may contact another transport means (while in motion or parked), in order to record the area where a potential event has occurred. The system may capture data from one or more of the transport means and / or sensors that may be present on fixed or movable objects. In one example, the solution may also be used to determine if a transport means is damaged by identifying a new state of the transport means during an event of the transport means using sensor data and comparing the state to the state profile of the transport means, thereby enabling the secure capture of important data from transport means that may be involved in a harmful event.

[0092] In one example, the solution may also be used to warn the occupants of a transport means when the transport means determines via one or more sensors that it is approaching or proceeding in the wrong direction on a one-way road. The transport means has sensors / cameras / maps that communicate with the system of the solution. The system recognizes the geographical location of the one-way road. The system may inform the occupants, for example, audibly, that they are "approaching a one-way road". In one example, the solution may also be used to enable a transport means to earn a reward, to enable the owner of an autonomous vehicle to monetize the data collected and stored by their vehicle's sensors, to create an incentive for the vehicle owner to share their data and provide additional data to an entity to improve the performance of future vehicles, and to provide services to the vehicle owner.

[0093] In one example, the solution can also be used to increase or decrease the functionality of a vehicle in response to the vehicle's operation over a period of time. In one example, the solution can also be used to assign fractional ownership to a means of transportation. Sensor data associated with one or more means of transportation and devices close to the means of transportation is used to determine the state of the means of transportation. The fractional ownership of the means of transportation is determined based on the state, and new responsibilities for the means of transportation are defined. In one example, the solution can also be used to provide data to replacement / upfitting parts, and the data allows the parts to use the approved functionality of the replacement / upfitting parts in response to an attempt to break the approved functionality of the replacement / upfitting parts and the approved functionality not being broken.

[0094] In one example, the solution can also be used to enable a passenger to be assured that a passenger is inside a means of transportation and should reach a specific destination. Further, the system ensures that a driver (in the case of a non-autonomous means of transportation) and / or other passengers are authorized to interact with the passenger. Pickup, drop-off, and location are also mentioned. All of the above are stored in the blockchain in an immutable manner. In one example, the solution can also be used to determine a driver's characteristics through analysis of driving style and other factors, and to take measures in the event that the driver is not driving normally, such as when the driver has driven in a specific state before, e.g., during the day, at night, in the rain, in the snow, etc. Further, the attributes of the means of transportation are also taken into account. The attributes consist of, for example, weather, whether headlights are on, whether navigation is being used, whether a HUD is being used, whether a certain volume of media is being played, etc. In one example, the solution can also be used to notify a passenger inside a means of transportation of a dangerous situation when an item inside the means of transportation indicates that the passenger may not be aware of the dangerous situation.

[0095] In one example, the solution can also be used to attach a calibration device to equipment fixed to the vehicle, and various sensors on the means of transportation can automatically self-adjust based on what should be detected by the calibration device as compared to what is actually detected. In one example, the solution can also be used to enable a remote diagnosis function by requiring consensus from multiple service centers using a blockchain when a means of transportation that requires service sends malfunction information, and the consensus is required from other service centers regarding what the severity threshold for the data is. Once the consensus is received, the service center can send the malfunction security level to the stored blockchain. In one example, the solution can also be used to determine the difference between sensor data outside the means of transportation and the sensor data of the means of transportation itself. The means of transportation requests software from the server to correct the problem. In one example, the solution can also be used to enable messaging of means of transportation that are near or within an area when an event (e.g., a collision) occurs.

[0096] Referring to FIG. 2I, an operating environment 290A of a connected means of transportation according to some embodiments is shown. As depicted, the means of transportation 276 includes a controller area network (CAN) bus 291A that connects elements 292A - 299A of the means of transportation. Other elements can be connected to the CAN bus but are not depicted herein. The depicted elements connected to the CAN bus include a sensor set 292A, an electronic control unit 293A, an autonomous function or advanced driver assistance system (ADAS) 294A, and a navigation system 295A. In some embodiments, the means of transportation 276 includes a processor 296A, a memory 297A, a communication unit 298A, and an electronic display 299A.

[0097] Processor 296A includes an arithmetic logic unit, a microprocessor, a general-purpose controller, and / or a similar processor array, and performs calculations to provide an electronic display signal to display unit 299A. Processor 296A may include various computing architectures, including processing data signals and implementing a complex instruction set computer (CISC) architecture, a reduced instruction set computer (RISC) architecture, or an architecture implementing a combination of instruction sets. Transport means 276 may include one or more processors 296A. Other processors, operating systems, sensors, displays, and physical configurations (not depicted) that are communicatively connected to each other may be used in this solution.

[0098] Memory 297A is a non-transitory memory that stores instructions or data that can be accessed and executed by processor 296A. The instructions and / or data may include code for performing the techniques described herein. Memory 297A may be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory, or some other memory device. In some embodiments, memory 297A may also include a hard disk drive, a floppy disk drive, a CD-ROM device, a DVD-ROM device, a DVD-RAM device, a DVD-RW device, a flash memory device, or some other mass storage device for permanently storing information, and may include non-volatile memory or similar permanent storage devices and media. A portion of memory 297A may be reserved for use as a buffer or virtual random access memory (virtual RAM). Transport means 276 may include one or more memories 297A without departing from the solution.

[0099] The memory 297A of the transport means 276 can store one or more of the following types of data, namely, the navigation route data 295A and the autonomous function data 294A. In some embodiments, the memory 297A stores data that may be necessary for the navigation application 295A to provide its functions.

[0100] The navigation system 295A can represent at least one navigation route including a starting point and an end point. In some embodiments, the navigation system 295A of the transport means 276 receives a request from the user regarding a navigation route, and the request includes a starting point and an end point. The navigation system 295A can query a real-time data server 293, such as a server that provides a driving direction, about the navigation route data corresponding to the navigation route including the starting point and the end point (via the network 292). The real-time data server 293 transmits the navigation route data to the transport means 276 via the wireless network 292, and the communication system 298A stores the navigation data 295A in the memory 297A of the transport means 276.

[0101] The ECU 293A controls the operation of a number of systems of the transport means 276 including the ADAS system 294A. The ECU 293A can disable any dangerous and / or unselected autonomous functions during the period of the journey controlled by the ADAS system 294A in response to an instruction received from the navigation system 295A. In this way, the navigation system 295A can control whether to operate or enable the ADAS system 294A so that the ADAS system 294A can operate on a given navigation route.

[0102] Sensor set 292A may include any sensors that generate sensor data in the transportation means 276. For example, sensor set 292A may include a short-range sensor and a long-range sensor. In some embodiments, the sensor set 292A of the transportation means 276 includes the following vehicle sensors, namely, cameras, LIDAR sensors, ultrasonic sensors, automotive engine sensors, radar sensors, laser altimeters, manifold absolute pressure sensors, infrared detectors, motion detectors, thermostats, voice detectors, carbon monoxide sensors, carbon dioxide sensors, oxygen sensors, mass air flow sensors, engine coolant temperature sensors, throttle position sensors, crankshaft position sensors, valve timers, air-fuel ratio meters, blind spot meters, curve feelers, defect detectors, Hall effect sensors, parking sensors, speed guns, speedometers, speed sensors, tire pressure monitoring sensors, torque sensors, transmission fluid temperature sensors, turbine speed sensors (TSS), variable reluctance sensors, vehicle speed sensors (VSS), moisture sensors, wheel speed sensors, GPS sensors, mapping functions, and one or more of any other types of automotive sensors. The navigation system 295A may store the sensor data in the memory 297A.

[0103] The communication unit 298A transmits and receives data to and from the network 292 or another communication channel. In some embodiments, the communication unit 298A may include a DSRC transceiver, a DSRC receiver, and other hardware or software necessary to make the transportation means 276 a DSRC-equipped device.

[0104] The transportation means 276 may communicate with other transportation means 277 via V2V technology. In one example, V2V communication includes detecting radar information corresponding to the relative distance to an external object, receiving the GPS information of the transportation means, setting an area as the area where the other transportation means 277 is located based on the detected radar information, calculating the probability that the GPS information of the target vehicle is located in the set area, and identifying the transportation means and / or object corresponding to the radar information and GPS information of the target vehicle based on the calculated probability.

[0105] In one example, the solutions described and depicted herein can be utilized to manage the emergency deployment and the functions of a transportation means when it is determined that the transportation means has entered an area without network access. In one example, the solution can also be utilized to manage and provide functions (such as voice, video, navigation, etc.) in a transportation means without network connection. In one example, the solution can also be utilized to determine when the profile of a person near the transportation means matches the profile attributes of the profile of at least one passenger within the transportation means. A notification is sent from the transportation means to establish communication.

[0106] In one example, the solution can also be utilized to analyze the availability of passengers within each transportation means where voice communication is available, based on the amount of time remaining within the transportation means and the context of the communication being conducted. In one example, the solution can also be utilized to determine two threat levels regarding road obstacles, receive gestures that may indicate that the obstacles do not reach a warning threshold, and be used by the transportation means to proceed along the road. In one example, the solution can also be utilized to delete confidential data from the transportation means when the transportation means has received damage such that it is rendered inoperable.

[0107] In one example, the solution can also be used to confirm that the customer data to be removed is truly removed from all the necessary locations within an enterprise that has explicit GDPR compliance. In one example, the solution can also be used to provide consideration from one means of transportation to another in exchange for data related to safety and important notifications, etc., to enhance the autonomous capabilities of a lower level of autonomous vehicles. In one example, the solution can also be used to provide the ability for a means of transportation to receive data based on a first biometric associated with a passenger. Then, the means of transportation decrypts the encrypted data based on the verification of a second biometric, and the second biometric is a continuum of the first biometric. The means of transportation provides the decrypted data to the passenger only when the passenger can receive it, deletes the confidential portion of the decrypted data when a confidential portion is provided, and deletes the non-confidential portion after the period associated with the biometric has elapsed. In one example, the solution can also be used to provide the ability for a means of transportation to verify an individual based on the weight and gripping pressure applied to the steering wheel of the means of transportation. In one example, the solution can also be used to provide a function that exists but is not currently enabled in a passenger vehicle to present functions reflecting the characteristics of the passengers to the passengers of the vehicle.

[0108] In one example, the solution can also be used to enable the reflection of modifications regarding the means of transportation, in particular inside and outside the means of transportation, and to assist at least one passenger in one example. In another example, the reproduction of the passenger's work environment and / or home environment is disclosed. The system can attempt to "reproduce" the user's work environment / home environment while the user is inside the means of transportation if the means of transportation determines that the user is in "work mode" or "home mode". All data related to the inside and outside of the means of transportation and all passengers using the means of transportation are stored in the blockchain and executed via smart contracts. In one example, the solution can also be used to detect the passenger's gestures and assist in communication with nearby means of transportation, and the means of transportation can be maneuvered accordingly. In one example, the solution can also be used to use a gesture definition data store to provide the means of transportation with the ability to detect the intended gestures. In one example, the solution can also be used to provide the means of transportation with the ability to take various measures based on the user's pace and gestures. In one example, the solution can also be used to ensure that the driver of the means of transportation, who is currently involved in various operations (such as driving while talking to navigation), does not exceed the number of dangerous operations before obtaining permission for gestures.

[0109] In one example, the solution can also be used to assign a situation to each occupant within a means of transportation and verify gestures from the occupants based on the occupant's situation. In one example, the solution can also be used to collect details of sounds related to a collision (such as where, in which direction, whether it is getting louder or quieter, from which device, data associated with the device, e.g., type, manufacturer, owner, as well as the number of sounds occurring simultaneously and the time the sound was emitted) and provide them to the system when the analysis of the data aids in determining details regarding the collision. In one example, the solution can also be used to provide a determination that the operation of the means of transportation is dangerous. The means of transportation includes a plurality of components that interact to control the means of transportation, and each component is associated with a key of a separate component. An encryption key is sent to the means of transportation to reduce the functionality of the means of transportation. In response to receiving the encryption key, the means of transportation disables one or more of the component keys. The disabling of one or more component keys results in one or more of the following: restricting the means of transportation from moving faster than a given speed, restricting the means of transportation from approaching another means of transportation by more than a certain distance, and restricting the means of transportation from moving farther than a threshold distance.

[0110] In one example, the solution can also be used to provide a display from a particular means of transportation (trying to vacate a location) to another particular means of transportation (trying to occupy the location), and the blockchain is used to perform authentication and adjustment. In one example, the solution can also be used to determine partial responsibility for a means of transportation. This is used by the system to update co-ownership when multiple people own a single means of transportation and the use of the means of transportation can be changed over a period of time. Other embodiments include uses that involve not the use of the means of transportation but the availability of the means of transportation and minimal ownership of the means of transportation based on the decisions of the operator of the means of transportation and other factors.

[0111] In one example, the solution can also be used in a transportation means for the user to permit their subscription regarding people in a closed group such as family or friends. For example, a user may want to share membership, and in that case, the associated transaction is stored in a blockchain or a conventional database. When the material of a regular subscription is requested by a user who is not the main subscriber, the blockchain node (i.e., the transportation means) can confirm that the person requesting the service is an approved person with whom the subscriber has shared the profile. In one example, the solution can also be used to enable a person to arrive at the intended destination using an auxiliary transportation means. Functional relationship values (e.g., values indicating various parameters and their importance when determining which type of alternative transportation means should be used) are used when determining the auxiliary transportation means. In one example, the solution can also be used to enable passengers in an accident to access other transportation means and continue to their original destination.

[0112] In one example, the solution may also be used to communicate software / firmware uploads to a first subset of transport means. This first set of transport means tests the update, and when the test is successful, the update is communicated to a further set of transport means. In one example, the solution may also be used to communicate software / firmware updates from a master transport means to a vehicle, where the update is communicated through the vehicle's network from a first subset, then a larger subset, and so on. A portion of the update may be sent first, and then the remaining portion may be sent from the same vehicle or a different vehicle. In one example, the solution may also be used to provide updates for the computers of the transport means to the transport means and the devices of the operator / occupants of the transport means. The update may be approved by all drivers and / or all occupants. The software update is provided to the vehicle and the devices. The user need do nothing other than go near the vehicle, and the function occurs automatically. A notification indicating that the software update is complete is sent to the device. In one example, the solution may also be used to verify that an OTA software update is being performed by an authorized technician, and that the situation regarding the origin of the verification code, the procedure for receiving the software update wirelessly, the information included in the software update, and the result of the verification is being generated by components of one or more transport means.

[0113] In one example, the solution can also be utilized to provide the ability for a software update located within a first component to be parsed by a second component. Then, a first portion of the critical update and a second portion of the non-critical update are identified, and in the transportation means, the identified first portion is assigned to a certain process and the identified first portion is operated in that process for a certain period. In response to a positive result based on that period, after that period, the identified first portion is operated in another process. In one example, the solution can also be utilized to provide a selection of services to a passenger, and the services are based on the profile of the passenger of the transportation means and a shared profile shared with the passenger's profile. In one example, the solution can also be utilized to store user profile data in a blockchain and intelligently present offers and recommendations to the user based on the automatically collected purchase history of the user and the preferences obtained from the user profile on the blockchain.

[0114] To make the transportation means sufficiently secure, the transportation means needs to be protected from unauthorized physical access and unauthorized remote access (e.g., cyber threats). In one example, to prevent unauthorized physical access, the transportation means is equipped with a secure access system such as keyless entry. On the other hand, in one example, a security protocol is added to the computer and computer network of the transportation means to facilitate secure remote communication with the transportation means.

[0115] An electronic control unit (ECU) is a node within a means of transportation that controls tasks ranging from tasks such as windshield wiper operation to tasks such as an anti-lock braking system. ECUs are often interconnected with each other through a central network of the means of transportation, which can be referred to as a controller area network (CAN). State-of-the-art functions such as autonomous driving strongly depend on new and complex ECU implementation modes such as advanced driver assistance systems (ADAS), sensors, and the like. These new technologies have helped improve the safety and driving experience of the means of transportation, but these new technologies also increase the number of external communication units within the means of transportation and make the external communication units more vulnerable to attacks. The following are some examples of protecting the means of transportation from physical and remote intrusions.

[0116] FIG. 2J shows a keyless entry system 290B for preventing unauthorized physical access to a means of transportation 291B according to an exemplary embodiment. Referring to FIG. 2J, in one example, a key fob 292B transmits commands to the means of transportation 291B using a radio frequency signal. In this example, the key fob 292B includes a transmitter 2921B having an antenna capable of transmitting short-range radio signals. The means of transportation 291B includes a receiver 2911B having an antenna capable of receiving short-range radio signals transmitted from the transmitter 2921B. The key fob 292B and the means of transportation 291B each also include CPUs 2922B and 2913B that control their respective devices. Here, there is memory of the CPUs 2922B and 2913B (or accessible to the CPU). In one example, each of the key fob 292B and the means of transportation 291B includes a power supply unit 2924B and 2915B that powers their respective devices.

[0117] When the user presses the button 293B of the key fob 292B (or in other cases, when the fob is activated, etc.), the CPU 2922B is activated within the key fob 292B and transmits a data stream output via the antenna to the transmitter 2921B. In other embodiments, the user's intention is recognized in the key fob 292B via other means such as a microphone that receives voice, a camera that captures images and / or video, or other sensors commonly used in the art to detect intentions from the user including gestures, movements, eye movements, and the like. The data stream can be a long signal from 64 bits to 128 bits in length including one or more of a preamble, a command code, and a rolling code. The signal can be transmitted at a speed between 2KHz and 20KHz, but the embodiments are not limited to this. Accordingly, the receiver 2911B of the transport means 291B captures the signal from the transmitter 2921B, demodulates the signal, and transmits the data stream to the CPU 2913B, and the CPU 2913B decodes the signal and transmits a command (e.g., locking or unlocking the door, etc.) to the command module 2912B.

[0118] If the key fob 292B and the transport means 291B use a fixed code between them, a replay attack can be carried out. In this case, if an attacker can capture / find out the fixed code during short-range communication, the attacker can replay this code to achieve access to the transport means 291B. To improve security, the key fob and the transport means 291B can use a rolling code that changes after each use. Here, the key fob 292B and the transport means 291B are synchronized with an initial seed 2923B (for example, a random number or a pseudo-random number, etc.). This is called pairing. The key fob 292B and the transport means 291B also include a shared algorithm that modifies the initial seed 2914B each time the button 293B is pressed. The next key press takes the result of the previous key press as input and converts it into the next number in the sequence. In some cases, the transport means 291B can store a plurality of next codes (for example, 255 next codes) if the key presses of the key fob 292B are not detected by the transport means 291B. Therefore, a large number of key presses of the key fob 292B that are not recognized by the transport means 291B do not prevent the transport means from becoming asynchronous.

[0119] In addition to the rolling code, the key fob 292B and the transport means 291B can adopt other methods to make the attack even more difficult. For example, various frequencies can be used to transmit the rolling code. As another example, two-way communication between the transmitter 2921B and the receiver 2911B can be used to establish a secure session. As another example, the code can have a limited expiration date or timeout. Further, the solution as described and depicted with respect to FIG. 2J can be utilized in this network and / or system, including what is described and depicted herein, as well as in other networks and / or systems.

[0120] Figure 2K shows a Controller Area Network (CAN) 290C within a transportation means according to an exemplary embodiment. Referring to Figure 2K, CAN 290C includes a CAN bus 297C having high and low terminals, and a plurality of electronic control units (ECUs) 291C, 292C, 293C, etc. connected to the CAN bus 297C via a wired connection. The CAN bus 297C is designed to enable microcontrollers and devices to communicate with each other in an application without using a host computer. The CAN bus 297C implements a message-based protocol (i.e., ISO 11898 standard) that enables ECUs 291C to 293C to send commands to each other at the root level. On the other hand, ECUs 291C to 293C represent controllers that control electrical systems or subsystems within the transportation means. Examples of electrical systems include power steering, antilock brakes, air conditioning, tire pressure monitoring, cruise control, and numerous other functions.

[0121] In this example, ECU 291C includes a transceiver 2911C and a microcontroller 2912C. The transceiver can be used to transmit and receive messages to and from the CAN bus 297C. For example, the transceiver 2911C can convert data from the microcontroller 2912C into the format of the CAN bus 297C, and also convert data from the CAN bus 297C into a format for the microcontroller 2912C. On the other hand, in one example, the microcontroller 2912C interprets messages and determines which messages to send using the ECU software installed in the microcontroller 2912C.

[0122] To protect the CAN290C from cyber threats, various security protocols can be implemented. For example, a subnetwork (such as subnetwork A and B, etc.) can be used to divide the CAN290C into smaller sub-CANs to limit the ability of an attacker who remotely accesses the transportation means. In the example of FIG. 2K, the ECUs 291C and 292C can be part of the same subnetwork, while the ECU 293C is part of an independent subnetwork. Further, a firewall 294C (or a gateway, etc.) can be added to prevent messages from traversing the CAN bus 297C across the subnetwork. If an attacker achieves access to a certain subnetwork, the attacker does not have access to the entire network. In one example, to make the subnetwork even more secure, the most important ECUs are not placed in the same subnetwork.

[0123] Although not shown in FIG. 2K, other examples of security control within the CAN include an intrusion detection system (IDS), which can be added to each subnetwork to read all passing data and detect malicious messages. If a malicious message is detected, the IDS can notify the vehicle user. Other possible security protocols can include encryption / security keys that can be used to obfuscate messages. As another example, in one example, an authentication protocol is implemented that allows messages to authenticate themselves.

[0124] In addition to protecting the internal network of the transport means, the transport means can also be protected when communicating with an external network such as the Internet. One advantage of having a transport means connected to a data source such as the Internet is that information from the transport means can be sent through the network to a remote location for analysis. Examples of transport means information include GPS, on-board diagnostics, tire pressure, and the like. Since these communication systems include a combination of telecommunications and informatics, such communication systems are often referred to as telematics. Further, the solution as described and depicted with respect to FIG. 2K can be utilized in this network and / or system, including what is described and depicted herein, as well as other networks and / or systems.

[0125] FIG. 2L shows a secure end-to-end transport means communication channel according to an exemplary embodiment. Referring to FIG. 2L, a telematics network 290D includes a transport means 291D and a host server 295D that is located at a remote location (such as a web server, a cloud platform, a database, etc.) and is connected to the transport means 291D via a network such as the Internet. In this example, a device 296D associated with the host server 295D can be installed inside the transport means 291D within the network. Further, although not shown, the device 296D can be connected to other elements of the transport means 291D, such as a CAN bus, an on-board diagnostics (ODBII) port, a GPS system, a SIM card, a modem, and the like. The device 296D can collect data from any of these systems and transmit the data to the server 295D via the network.

[0126] Secure management of the data begins with the transport means 291D. In some embodiments, the device 296D may collect information before, during, and after movement. The data may include GPS data, movement data, passenger information, diagnostic data, fuel data, speed data, and the like. However, the device 296D may simply communicate and return the collected information to the host server 295D in response to the ignition and completion of movement of the transport means. Further, the communication may be initiated only by the device 296D and not by the host server 295D. Thus, in one example, the device 296D does not receive communications initiated by an external source.

[0127] To perform the communication, the device 296D may establish a secure private network between the device 296D and the host server 295D. Here, the device 296D may include an anti-tampering SIM card that provides secure access to the carrier network 294D via the radio tower 292D. When preparing to send data to the host server 295D, the device 296D may establish a one-way secure connection with the host server 295D. The carrier network 294D may communicate with the host server 295D using one or more security protocols. As a non-limiting example, the carrier network 294D may communicate with the host server 295D via a VPN tunnel that enables access through the firewall 293D of the host server 295D. As another example, the carrier network 294D may use data encryption (e.g., AES encryption, etc.) when sending data to the host server 295D. In some cases, the system may use multiple security measures such as both VPN and encryption to make the data more secure.

[0128] In addition to communicating with external servers, the transport means can also communicate with each other. In particular, a vehicle-to-vehicle (V2V) communication system enables the transport means to communicate with each other, with roadside infrastructure (such as traffic lights, signs, cameras, parking meters, etc.), and with others of the same kind through a wireless network. The wireless network can include one or more of a Wi-Fi network, a cellular network, a dedicated short-range communication (DSRC) network, and the like. The transport means can use V2V communication to provide information regarding, among other things, the speed, acceleration, brakes, and direction of the transport means to other transport means. Thus, the transport means can receive insights into the state ahead before that state becomes visible, and thus can greatly reduce collisions. Further, the solution as described and depicted with respect to Figure 2L can be utilized in this network and / or system, including those described and depicted herein, as well as in other networks and / or systems.

[0129] Figure 2M shows an example 290E of transport means 293E and 292E that perform secure V2V communication using security certificates, according to an exemplary embodiment. Referring to Figure 2M, the transport means 293E and 292E can communicate with each other via V2V communication through a short-range network, a cellular network, or the like. Before transmitting a message, the transport means 293E and 292E can sign the message using their respective public key certificates. For example, the transport means 293E can sign a V2V message using the public key certificate 294E. Similarly, the transport means 292E can sign a V2V message using the public key certificate 295E. In one example, the public key certificates 294E and 295E are respectively associated with the transport means 293E and 292E.

[0130] When receiving communications from each other, the transport means can confirm the signature with the certification authority 291E or the like. For example, the transport means 292E can confirm with the certification authority 291E that the public key certificate 294E used by the transport means 293E for signing V2V communications is a certified one. When the transport means 292E successfully confirms the public key certificate 294E, the transport means recognizes that the data is from a legitimate source. Similarly, the transport means 293E can confirm with the certification authority 291E that the public key certificate 295E used by the transport means 292E for signing V2V communications is a certified one. Further, the solution as described and depicted with respect to FIG. 2M can be utilized in this network and / or system, including what is described and depicted herein, as well as in other networks and / or systems.

[0131] FIG. 2N shows a further additional FIG. 290F depicting an example of a transport means that interacts with a security processor and a wireless device according to an exemplary embodiment. In some embodiments, the computer 224 shown in FIG. 2B can include a security processor 292F as shown in the process 290F of the example of FIG. 2N. In particular, the security processor 292F can perform approval, authentication, encryption (e.g., encryption), and the like for data transmissions sent between the ECU and other devices on the vehicle's CAN bus, and also for data messages sent between different vehicles.

[0132] In the example of FIG. 2N, the security processor 292F may include an approval module 293F, an authentication module 294F, and an encryption module 295F. The security processor 292F may be implemented within the computer of the transport means and may communicate with other elements of the transport means, such as the ECU / CAN network 296F, wired and wireless devices 298F, such as a wireless network interface, an input port, and the like. The security processor 292F may ensure that data frames (such as CAN frames, etc.) transmitted internally within the transport means (e.g., via the ECU / CAN network 296F) are secure. Similarly, the security processor 292F may ensure that messages transmitted between different transport means and to devices attached or connected to the computer of the transport means via a wire are also secure.

[0133] For example, the approval module 293F may store passwords, usernames, PIN codes, biometric scans, and the like for various users of the transport means. The approval module 293F may determine whether a user (or technician) has the permission to access certain settings of the transport means, such as the computer of the transport means. In some embodiments, the approval module may communicate with a network interface to download any necessary approval information from an external server. When a user requests to make changes to the settings of the transport means or modify the technical details of the transport means via a console or GUI within the transport means or via an attached / connected device, the approval module 293F may request the user to identify itself in some way before the settings are changed. For example, the approval module 293F may require a username, password, PIN code, biometric scan, a predefined line drawing or gesture, and the like. Accordingly, the approval module 293F may determine whether the user has the required permission (such as access, etc.) requested.

[0134] The authentication module 294F can be used to authenticate the internal communication between ECUs in the vehicle's CAN network. As an example, the authentication module 294F can provide information for authenticating the communication between ECUs. As an example, the authentication module 294F can send a bit signature algorithm to the ECUs of the CAN network. The ECU can use the bit signature algorithm to insert authentication bits into the CAN fields of the CAN frame. All ECUs on the CAN network usually receive each CAN frame. Each time a new CAN frame is generated by one of the ECUs, the bit signature algorithm can dynamically change the position and amount of the authentication bits, etc. The authentication module 294F can also provide a list of ECUs that are exempted (are in the safe list) and do not need to use authentication bits. The authentication module 294F can communicate with a remote server to retrieve updates and the like for the bit signature algorithm.

[0135] The encryption module 295F can store an asymmetric key pair used by the transport means to communicate with other external user devices and transport means. For example, the encryption module 295F can provide the private key used by the transport means to encrypt / decrypt communication, while the corresponding public key can be provided to other user devices and the transport means so that other devices can decrypt / encrypt the communication. The encryption module 295F can communicate with a remote server to receive new keys, updates to keys, keys for new transport means or users, and the like. The encryption module 295F can also send any updates to the local private / public key pair to the remote server.

[0136] Figure 3A shows a flowchart 300 according to an exemplary embodiment. Referring to Figure 3A, the flow includes receiving a scan of the vehicle cabin 302, where the scan includes the interior of the vehicle and a spatial area outside the vehicle near at least one vehicle door; performing classification detection and presence detection regarding at least one seat in the vehicle interior, and determining at least one occupant feature based on the scan 304; determining a cabin situation based on the at least one occupant feature 306; and communicating the cabin situation to a mobile device 308. The scan of the cabin can be performed by radar, sonar, optical imaging, and the like. The reception of the scan can be performed by a controller, NUC server, API, ECU, and the like. In one example, the determination of the occupant feature can be performed by machine learning. For the detection of the presence and occupancy of a living being, in one implementation, the system detects the presence of an occupant at each seat level and whether the presence is an adult or a child. It generates a boolean variable (a value of 8 seats + 1 to capture all areas outside all seats) for each seat that is potentially available. For seats that cannot be detected, the boolean variable remains 0. In one exemplary implementation, the point cloud center is determined via clustering for each seat using density-based spatial clustering of applications with noise (DBSCAN) and a predefined XYZ rectangular cuboid boundary to determine the occupancy of the space and the occupancy by seat. In another exemplary implementation, a neural network specifically designed to process point cloud data is utilized. This exemplary implementation generates a centralized neural network that can simultaneously predict, within one neural network, predictions for each seat regarding the classification of presence, adult occupancy, child occupancy, and occupancy of the foot space from a single point cloud. The cabin situation is the classification and occupancy of each seat and space inside the vehicle and near the vehicle doors. The communication of the cabin situation to the mobile device can be wired or wireless, Wi-Fi, Bluetooth®, and the like.The cabin recognition logic may be executed in whole or in part on one or more of a processor in a vehicle, a processor in a server that may be on-board or off-board in the vehicle, a device associated with the vehicle, or any other processor associated with the vehicle. Similarly, the memory utilized by the one or more processors may be located on-board or off-board in the vehicle. The vehicle's processor may be the vehicle's ECM or another processor, such as a processor in an ECU, HU, NUC server, or another processor in the vehicle.

[0137] Figure 3B shows another flow diagram 320 according to an exemplary embodiment. Referring to Figure 3B, the flow includes combining at least one occupant feature based on a scan with at least another occupant feature based on a set of data received from a controller area network 324, and communicating a log of the scan and the cabin situation to a server via the controller area network 332, where the server updates a cabin situation model based on the log of the scan 332. The occupant feature includes a spatial area inside the vehicle that is separated from at least one seat inside the vehicle (322). The scan captures at least one point cloud center for at least one of the foot space and head space of the cabin (326), and uses a predefined rectangular parallelepiped boundary for at least one seat inside the vehicle to determine space occupancy and occupancy by seat (328). The classification detection includes adult occupancy, child occupancy, and foot space occupancy (330).

[0138] In Figure 3B, the occupant feature includes a spatial area outside the seat area (322), and in one example, this includes warnings outside the designated seat to determine that an occupant is sitting inappropriately in the seat, such as multiple occupants sitting in one seat, an occupant lying down, facing sideways, or sitting too far forward.

[0139] In FIG. 3B, combining at least one occupant feature based on a scan with at least one other occupant feature 324 is based, in part, on information flow from the vehicle's internal CAN network (such as seat belt status and ignition status) and scan data such as radar data generated by sensors. These two sources are supplied to the NUC. The process receives this data and performs relevant determinations and decoding before transmitting it to the API server. The API server then sends this information to the mobile device, which displays it in an app.

[0140] In FIG. 3B, the capture 326 of at least one point cloud center regarding at least one of the foot and head spaces in the cabin, in one exemplary implementation, is based on the point cloud center determined via clustering for each seat using density-based spatial clustering (DBSCAN) of noisy applications and a predefined XYZ cuboid boundary to determine space occupancy and occupancy by seat. In another exemplary implementation, a neural network specifically designed to process point cloud data is utilized. This exemplary implementation generates a centralized neural network that can simultaneously predict, within one neural network, predictions for each seat regarding the classification of presence, adult occupancy, child occupancy, and foot space occupancy from a single point cloud.

[0141] In FIG. 3B, the use 328 of a bounding box / cuboid via density-based spatial clustering enables the recognition of angular anatomical structures (e.g., the occupant's head, chest regions) to determine space occupancy and occupancy by seat.

[0142] In FIG. 3B, the fact 330 that classification detection includes adult occupancy, child occupancy, and foot space occupancy can, in one example, be based on an architecture that processes point cloud data in a way that rotation and order are invariant. A centralized neural network that can simultaneously predict, within the same neural network, seat-by-seat predictions regarding the presence, adult occupancy, child occupancy, and foot space occupancy classifications from a single point cloud.

[0143] In FIG. 3B, the fact 332 that the server updates the cabin situation model based on the scan log is, in one example, an update of a machine learning model based on a larger training data set provided by the log.

[0144] FIG. 3C shows yet another flow diagram 340 according to an exemplary embodiment. Referring to FIG. 3C, the flow diagram involves receiving an event confirmation from one or more of the elements described or depicted herein, where the confirmation comprises a blockchain consensus between peers represented by any of the elements 342, and executing a smart contract to record the confirmation in the blockchain based on the blockchain consensus 344, including one or more of these.

[0145] FIG. 4 shows a machine learning transportation means network diagram 400 according to an exemplary embodiment. Network 400 includes transportation means 402 coupled to a machine learning subsystem 406. The transportation means includes one or more sensors 404.

[0146] The machine learning subsystem 406 includes a learning model 408, which is a mathematical artifact created by a machine learning training system 410 that generates predictions by finding patterns within one or more training data sets. In some embodiments, the machine learning subsystem 406 is present within the transportation means 402. In other embodiments, the machine learning subsystem 406 is external to the transportation means 402.

[0147] The transport means 402 transmits data from one or more sensors 404 to the machine learning subsystem 406. The machine learning subsystem 406 provides the data of one or more sensors 404 to the learning model 408, and the learning model 408 returns one or more predictions. The machine learning subsystem 406 transmits one or more instructions to the transport means 402 based on the predictions from the learning model 408.

[0148] In a further embodiment, the transport means 402 may transmit the data of one or more sensors 404 to the machine learning training system 410. In yet another example, the machine learning subsystem 406 may transmit the data of the sensors 404 to the machine learning subsystem 410. One or more of the applications, functions, steps, solutions, etc. described and / or depicted herein may utilize the machine learning network 400 as described herein.

[0149] FIG. 5A shows an exemplary vehicle configuration 500 that manages database transactions associated with a vehicle, according to an exemplary embodiment. Referring to FIG. 5A, when a particular transportation means / vehicle 525 is involved in a transaction (e.g., vehicle service, dealership transaction, delivery / pickup, transportation service, etc.), the vehicle can receive (510) and / or give / transfer (512) assets according to the transaction. The transportation means processor 526 exists within the vehicle 525, and there is communication between the transportation means processor 526, the database 530, the transportation means processor 526, and the transaction module 520. The transaction module 520 can record information such as assets, parties, credits, service descriptions, dates, times, locations, results, notifications, unexpected events, etc. The transaction in the transaction module 520 can be replicated in the database 530. The database 530 can be one of an SQL database, an RDBMS, a relational database, a non-relational database, a blockchain, a distributed ledger, can be on-board in the transportation means, can be off-board in the transportation means, can be directly and / or accessible through a network, or can be accessible to the transportation means.

[0150] Figure 5B shows an exemplary vehicle configuration 550 that manages database transactions performed between various vehicles according to an exemplary embodiment. When a vehicle reaches a situation where a service needs to be shared with another vehicle, vehicle 525 can engage with another vehicle 508 to perform various operations such as sharing, transmitting, and obtaining service requests. For example, vehicle 508 may be scheduled for battery charging and / or may have a problem with its tires and may be within the route to pick up the goods for delivery. The transportation means processor 528 exists within vehicle 508, and there is communication between the transportation means processor 528, the database 554, and the transaction module 552. Vehicle 508 can notify another vehicle 525 that is within its network and operating on its blockchain member service. The transportation means processor 526 exists within vehicle 525, and there is communication between the transportation means processor 526, the database 530, the transportation means processor 526, and the transaction module 520. Then, vehicle 525 can receive information via a wireless communication request and pick up the goods from vehicle 508 and / or a server (not shown). The transaction is logged in the transaction modules 552 and 520 of both vehicles. Credits are transmitted from vehicle 508 to vehicle 525, and the record of the transmitted service is logged in the database 530 / 554 assuming that the blockchains are different from each other, or logged in the same blockchain used by all members. The database 554 can be one of an SQL database, an RDBMS, a relational database, a non-relational database, a blockchain, a distributed ledger, can be on-board in the transportation means, can be off-board in the transportation means, and can be accessible directly and / or through a network.

[0151] FIG. 6A shows a blockchain architecture configuration 600 according to an exemplary embodiment. Referring to FIG. 6A, the blockchain architecture 600 may include a group of blockchain member nodes 602-606 as part of a particular blockchain element, e.g., a blockchain group 610. In an exemplary embodiment, in a permissioned blockchain, only members who have permission to access blockchain data, rather than all parties, are accessible. Blockchain nodes are involved in a number of activities such as the addition and verification process (consensus) of blockchain entries. One or more of the blockchain nodes may approve an entry based on an endorsement policy and may provide an ordering service to all blockchain nodes. A blockchain node may initiate a blockchain operation (such as authentication), attempt to write to a blockchain ledger stored on the blockchain, and a copy thereof may also be stored on the underlying physical infrastructure.

[0152] Once a blockchain transaction 620 is received and approved by a consensus model determined by a member node, it is stored in the computer's memory. The approved transaction 626 is stored in the current block of the blockchain and committed to the blockchain via a commit procedure, which includes hashing the data content of the transactions within the current block and referencing the previous hash of the previous block. Within the blockchain, there may be one or more smart contracts 630 that define the conditions for the agreement and operation of transactions contained within the smart contract executable application code 632, such as registered recipients, vehicle functions, requirements, permissions, sensor thresholds, etc. The code can be configured to identify whether the requesting entity is registered to receive vehicle services, what service functions the entity is eligible / required to receive considering the entity's profile status, and whether to monitor the entity's operation in subsequent events. For example, when a service event occurs and the user is in the vehicle, the monitoring of sensor data can be triggered, and specific parameters such as the vehicle's charge level can be identified as exceeding / falling below a specific threshold over a specific period, and as a result, the current situation can be changed, which may require sending a warning to the management parties (i.e., vehicle owner, vehicle operator, server, etc.), so the service can be identified and stored for reference. The vehicle sensor data collected can be based on the type of sensor data used to collect information about the vehicle's situation. The sensor data can also be the basis for vehicle event data 634 such as the location traveled, average speed, maximum speed, acceleration, whether there was any collision, whether the expected route was taken, where the next destination is, whether safety measures are being implemented, whether the vehicle has sufficient charge / fuel, etc. All such information can be the basis for the smart contract conditions 630, which are then stored in the blockchain.For example, the sensor thresholds stored in the smart contract can be used as a basis for determining whether a detected service is required and when and where the service should be performed.

[0153] FIG. 6B shows the configuration of a shared ledger according to an exemplary embodiment. Referring to FIG. 6B, an example 640 of blockchain logic includes a blockchain application interface 642 as an API or a plug-in application that couples to a computing device and an execution platform for a particular transaction. The blockchain configuration 640 may include one or more applications coupled to an application programming interface (API) to access and execute stored program / application code (e.g., smart contract executable code, smart contracts, etc.), and the program / application code may be created according to a customized configuration required by participants, maintain its own state, control its own assets, and receive external information. This can be deployed and installed as an entry by appending to a distributed ledger on all blockchain nodes.

[0154] The smart contract application code 644 provides a basis for blockchain transactions by establishing application code that enables transaction conditions and states when executed. The smart contract 630, when executed, generates a particular approved transaction 626, which is then transferred to the blockchain platform 652. The platform includes security / approval 658, a computing device 656 that executes transaction management, and a storage unit 654 as a memory that stores transactions and smart contracts in the blockchain.

[0155] A blockchain platform can include blockchain data of various layers, services (such as encryption trust services, virtual execution environments, etc.), and underlying physical computer infrastructure that can be used to provide access to auditors who are attempting to receive and store new entries and access data entries. The blockchain can expose an interface that processes program code and provides access to the virtual execution environment necessary to participate in the physical infrastructure. Encryption trust services can be used to verify entries such as asset exchange entries and keep information private.

[0156] The blockchain architecture configurations of FIGS. 6A and 6B can process and execute program / application code through one or more interfaces exposed and services provided by the blockchain platform. As a non-limiting example, smart contracts can be created to execute reminders, updates, and / or other notifications that are the subject of changes or updates, etc. The smart contracts themselves can be used to identify approval and access requirements as well as rules associated with the use of ledgers. For example, the information can include new entries that can be processed by one or more processing entities (such as processors, virtual machines, etc.) included in the blockchain layer. The result can include a decision to reject or approve new entries based on criteria defined by smart contracts and / or peer consensus. The physical infrastructure can be utilized to retrieve any of the data or information described herein.

[0157] Within the smart contract executable code, the smart contract is created via a high-level application and programming language and can then be written to the blocks within the blockchain. The smart contract can include executable code that is registered, stored, and / or replicated using a blockchain (e.g., a distributed network of blockchain peers). The entry is the execution of the smart contract code, which can occur in response to the conditions associated with the smart contract being met. The execution of the smart contract can trigger a reliable modification to the state of the digital blockchain ledger. Modifications to the blockchain ledger resulting from the execution of the smart contract can be automatically replicated across the distributed network of blockchain peers by one or more consensus protocols.

[0158] The smart contract can write data to the blockchain in the format of key-value pairs. Further, the smart contract code can read the values stored in the blockchain and use those values during application operation. The smart contract code can write the outputs of various logical operations into the blockchain. The code can be used to create temporary data structures within a virtual machine or other computing platform. The data written to the blockchain can be made public and / or encrypted to remain private. Temporary data used / generated by the smart contract is held in memory by the provided execution environment and then deleted once the data required by the blockchain is identified.

[0159] Smart contract executable code may include the code interpretation of a smart contract with additional functionality. As described herein, smart contract executable code may be program code deployed on a computing network, where the program code is executed and verified by chain validators together during a consensus process. Smart contract executable code receives a hash and extracts from the blockchain a hash associated with a data template created by using a previously stored function extractor. When the hash of the hash identifier matches the hash created from the stored identifier template data, the smart contract executable code then sends an approval key to the requested service. Smart contract executable code may write data associated with encryption details to the blockchain.

[0160] FIG. 6C shows a blockchain configuration for storing blockchain transaction data according to an exemplary embodiment. Referring to FIG. 6C, an exemplary configuration 660 provides a vehicle 662, a user device 664, and a server 666 that share information with a distributed ledger (i.e., a blockchain) 668. In an event where a known established user profile is attempting to rent a vehicle using an established rating profile, the server may represent a service provider entity that queries a vehicle service provider to share user profile rating information. The server 666 may receive and process data related to the service requirements of the vehicle. When a service event occurs, such as vehicle sensor data indicating a need for fuel / charge or maintenance services, smart contracts may be used to call rules, thresholds, collection of sensor information, etc. that can be used to trigger a vehicle service event. Blockchain transaction data 670 is stored for each transaction, such as access events, subsequent updates to the service status of the vehicle, and event updates. The transaction may include the parties involved, requirements (e.g., 18 years old, eligible candidates for services, valid driver's license, etc.), compensation levels, distance traveled between events, registered recipients permitted access to the event and provision of vehicle services, rights / permissions, sensor data retrieved during vehicle event operations to log details of the next service event and identify the status of the vehicle, and thresholds used to determine whether the service event has been completed and whether the status of the vehicle has changed.

[0161] Figure 6D shows the content of blockchain block 680 that can be added to a distributed ledger and block structures 682A - 682n according to an exemplary embodiment. Referring to Figure 6D, a client (not shown) can present an entry to a blockchain node to perform activities on the blockchain. As an example, the client can be an application that functions on behalf of a requester such as a device, a person, or an entity to propose an entry to the blockchain. Multiple blockchain peers (e.g., blockchain nodes) can maintain the state of the blockchain network and a copy of the distributed ledger. Various types of blockchain nodes / peers can exist within a blockchain network, including an approval peer that simulates and approves an entry proposed by a client, and a commit peer that verifies an endorsement, validates the entry, and commits the entry to the distributed ledger. In this example, a blockchain node can perform the role of an endorser node, a committer node, or both.

[0162] This system includes a blockchain that stores immutable and ordered records in blocks, and a state database (current world state) that maintains the current state of the blockchain. One distributed ledger may exist per channel, and each peer maintains its own copy of the distributed ledger for each channel of which it is a member. This blockchain is an entry log constructed as hash-linked blocks, where each block contains a sequence of N entries. A block may contain various components such as those shown in FIG. 6D. The linkage of blocks can be generated by adding a hash regarding the header of the previous block into the block header of the current block. In this way, all entries in the blockchain are ordered and cryptographically linked to prevent the tampering of blockchain data without breaking the hash link. Further, because they are linked, the latest block in the blockchain represents all the entries that have occurred before it. This blockchain can be stored on a peer file system (local or attached storage) that supports the workload of an append-only blockchain.

[0163] The current state of the blockchain and the distributed ledger can be stored in the state database. Here, the current state data represents the latest values for all keys included in the chain entry log of the blockchain. The invocation of smart contract executable code executes an entry against the current state in the state database. To make the interaction of the smart contract executable code extremely efficient, the latest values of all keys are stored in the state database. The state database may include an indexed view of the entry log of the blockchain, and thus, it can be regenerated from the chain at any time. The state database can be automatically restored (or generated if necessary) at the startup of the peer before an entry is received.

[0164] The approval node receives an entry from the client and approves the entry based on the simulated result. The approval node holds a smart contract that simulates the entry proposal. When the approval node approves an entry, the approval node creates an entry endorsement, and the entry endorsement is a signed response from the approval node to the client application, indicating the endorsement of the simulated entry. The method of approving an entry depends on the endorsement policy that can be specified within the smart contract executable code. An example of an endorsement policy is that "the majority of the approving peers must approve the entry". Different channels may have different endorsement policies. The approved entry is transferred by the client application to the ordering service.

[0165] The ordering service receives the approved entry, orders the entry within a block, and distributes the block to the commit peers. For example, the ordering service may start a new block when the threshold of the entry is reached, when the timer times out, or under another condition. In this example, the blockchain node is the commit peer that has received the data block 682A to be stored on the blockchain. The ordering service can be composed of an orderer cluster. The ordering service does not process the entry or the smart contract, nor does it maintain a shared ledger. Rather, the ordering service can receive the approved entry and specify the order in which the entry is committed to the distributed ledger. The architecture of the blockchain network can be designed such that a specific implementation of "ordering" (e.g., Solo, Kafka, BFT, etc.) is a pluggable component.

[0166] Entries are written to the distributed ledger in a consistent order. The order of the entries is established to ensure that updates to the state database are valid when the entry is committed to the network. Different from cryptocurrency blockchain systems (e.g., Bitcoin, etc.) where ordering occurs by solving an encryption puzzle or by mining, in this example, the parties to the distributed ledger can select the ordering mechanism that best suits the network.

[0167] Referring to FIG. 6D, a block 682A (also referred to as a data block) stored on the blockchain and / or distributed ledger may include a plurality of data segments such as block headers 684A to 684n, transaction-specific data 686A to 686n, and block metadata 688A to 688n. It should be understood that the various blocks and their contents described, such as block 682A and its contents, are for illustrative purposes only and do not mean to limit the scope of the exemplary embodiments. In some cases, both block header 684A and block metadata 688A may be smaller than transaction-specific data 686A that stores entry data, but this is not a requirement. Block 682A may store transaction information for N (e.g., 100, 500, 1000, 2000, 3000, etc.) entries within block data 690A to 690n. Block 682A may also include a link to a previous block (e.g., on the blockchain) within block header 684A. In particular, block header 684A may include a hash of the header of the previous block. Block header 684A may also include a unique block number, a hash of block data 690A of the current block 682A, and the like. The block number of block 682A is unique and can be assigned in an increasing / continuous order starting from zero. The first block in the blockchain may be referred to as a genesis block that includes information about the blockchain, its members, and the data stored therein.

[0168] The block data 690A can store the entry information of each entry recorded in the block. For example, the entry data includes the type, version, timestamp, channel ID of the distributed ledger, entry ID, epoch, visibility of the payload, path of the smart contract executable code (deploy and send), name of the smart contract executable code, version of the smart contract executable code, input (smart contract executable code and functions), client (creator) identification information such as public key and certificate, client signature, endorser identification information, endorser signature, proposal hash, event of the smart contract executable code, response status, namespace, read set (list of keys and versions read by the entry, etc.), write set (list of keys and values, etc.), start key, end key, list of keys, query summary of the Merkle tree, and one or more of the same kind. The entry data can be stored for each of the N entries.

[0169] In some embodiments, the block data 690A may also store transaction-specific data 686A that adds additional information to the hash link chain of blocks in the blockchain. Thus, the data 686A may be stored in the immutable log of blocks in the distributed ledger. Some of the advantages of storing the data 686A are reflected in the various embodiments disclosed and depicted herein. The block metadata 688A may store a plurality of fields of metadata (e.g., as a byte array, etc.). The metadata fields may include signatures in block creation, references to the last configuration block, entry filters that identify valid and invalid entries within the block, the last offset of the ordering service that ordered the block, and the like. The signature, the last configuration block, and the metadata of the orderer may be added by the ordering service. On the other hand, a committer of a block (such as a blockchain node) may add valid / invalid information based on an endorsement policy, verification of read / write sets, and the like. The entry filter may include a byte array of a size equal to the number of entries in the block data 610A and a verification code that identifies whether the entry was valid / invalid.

[0170] The other blocks 682B - 682n in the blockchain also have a header, a file, and a value. However, unlike the first block 682A, each of the headers 684A - 684n in the other blocks includes the hash value of the previous block. The hash value of the previous block may simply be the hash of the header of the previous block or the hash value of the entire previous block. By including the hash value of the previous block in each of the remaining blocks, tracing can be performed block by block back from the Nth block to the genesis block (and the associated original file) as indicated by the arrow 692, establishing an auditable and immutable chain of custody.

[0171] The above-described embodiments can be implemented by hardware, a computer program executed by a processor, firmware, or a combination of the above. The computer program can be embodied on a computer-readable medium such as a storage medium. For example, the computer program can be present in a random access memory ("RAM"), flash memory, read-only memory ("ROM"), erasable programmable read-only memory ("EPROM"), electrically erasable programmable read-only memory ("EEPROM"), register, hard disk, removable disk, compact disk read-only memory ("CD-ROM"), or any other form of storage medium known in the art.

[0172] A preferred storage medium can be connected to the processor such that the processor can read information from and write information to the storage medium. Alternatively, the storage medium can be integrated with the processor. The processor and the storage medium can be present within an application-specific integrated circuit ("ASIC"). Alternatively, the processor and the storage medium can be present as separate components. For example, FIG. 7 shows an exemplary computer system architecture 700 that can represent any of the above-described components or can be integrated with any of the above-described components.

[0173] FIG. 7 is not intended to imply any limitation with respect to the use or functionality scope of the embodiments of the present application described herein. Nevertheless, the computing node 700 is implementable and / or capable of performing any of the above-described functions herein.

[0174] Within the computing node 700, there exists a computer system / server 702 that can operate in the environment or configuration of many other general-purpose or special-purpose computing systems. Examples of well-known computing systems, environments, and / or configurations that may be suitable for use with the computer system / server 702 include personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable household appliances, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems or devices, and the like, but are not limited thereto.

[0175] The computer system / server 702 may be described in the general context of computer system-executable instructions, such as program modules, executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system / server 702 may be executed in a distributed cloud computing environment where tasks are performed by remote processing devices coupled through a communications network. In a distributed cloud computing environment, program modules may be located in both local computer system storage media including memory storage devices and remote computer system storage media.

[0176] As shown in FIG. 7, the computer system / server 702 within the cloud computing node 700 is shown in the form of a general-purpose computing device. The components of the computer system / server 702 may include, but are not limited to, one or more processors or processing units 704, a system memory 706, and a bus that couples various system components including the system memory 706 to the processor 704.

[0177] The bus represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of various bus architectures. By way of example and without limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Extended ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0178] The computer system / server 702 typically includes various computer system-readable media. Such media can be any available media accessible by the computer system / server 702, and it includes both volatile and non-volatile media, as well as removable and non-removable media. In one example, the system memory 706 implements the flow diagrams of other figures. The system memory 706 can include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 708 and / or cache memory 710. The computer system / server 702 can further include other removable / non-removable volatile / non-volatile computer system storage media. By way of mere example, the memory 706 can be provided for reading from and writing to a non-removable non-volatile magnetic medium (commonly referred to as a "hard drive" and not shown). Although not shown, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), as well as an optical disk drive for reading from or writing to a removable non-volatile optical disk, such as a CD-ROM, a DVD-ROM, or other optical media, can be provided. In such cases, each can be connected to the bus by one or more data media interfaces. As further depicted and described below, the memory 706 can include at least one program product having a set of program modules (e.g., at least one) configured to execute the functions of various embodiments of the present application.

[0179] A program / util utility having a set (at least one) of program modules can be stored, by way of example and not limitation, in memory 706, as well as in an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data, or any combination thereof, can include an implementation manner of a networking environment. The program modules generally execute the functions and / or methods of the various embodiments of the present application described herein.

[0180] As will be understood by those skilled in the art, aspects of the present application can be embodied as a system, a method, or a computer program product. Accordingly, aspects of the present application can take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software aspects and hardware aspects that can generally all be referred to herein as a "circuit", "module", or "system". Furthermore, aspects of the present application can take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied therein.

[0181] The computer system / server 702 can also include I / O devices 712 (such as I / O adapters) that may include a keyboard, a pointing device, a display, a voice recognition module, etc., one or more devices that enable a user to interact with the computer system / server 702, and / or any device (such as a network card or a modem) that enables the computer system / server 702 to communicate with one or more other computing devices, and can communicate with one or more external devices via the I / O interface of the device 712. Furthermore, the computer system / server 702 can communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network (such as the Internet), via a network adapter. As depicted, the device 712 communicates with other components of the computer system / server 702 via a bus. Although not shown, it should be understood that other hardware and / or software components can be used with the computer system / server 702. Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data storage systems.

[0182] Preferred embodiments of at least one of a system, a method, and a non-transitory computer-readable medium are shown in the accompanying drawings and described in the foregoing detailed description, but the present application is not limited to the disclosed embodiments, and it will be understood that many rearrangements, modifications, and substitutions are possible as defined by the following claims. For example, the functions of the systems of the various figures can be performed by one or more of the modules or components described herein, or in a distributed architecture, and can include a transmitter, a receiver, or a pair of both. For example, all or part of the functions performed by individual modules can be performed by one or more of these modules. Further, the functions described herein can be performed at various times in relation to various events internal or external to the modules or components. Also, the information transmitted between the various modules can be transmitted between the modules via at least one of a data network, the Internet, a voice network, an Internet protocol network, a wireless device, a wired device, and / or a plurality of protocols. Also, messages transmitted or received by any of the modules can be transmitted or received directly and / or via one or more of the other modules.

[0183] One of ordinary skill in the art will understand that "system" can be embodied as a personal computer, a server, a console, a personal digital assistant (PDA), a mobile phone, a tablet computing device, a smartphone, or any other suitable computing device, or a combination of devices. Presenting the functions described above as being performed by a "system" is not intended to limit the scope of the present application in any way, but rather to provide an example of one of many embodiments. In fact, the methods, systems, and apparatuses disclosed herein can be implemented in a local and distributed form that is consistent with computing technology.

[0184] Note that some of the system functions described in this specification are presented as modules to more specifically emphasize the independence of their implementation modes. For example, a module can be implemented as a hardware circuit comprising custom very large scale integration (VLSI) circuits or gate arrays, or off-the-shelf semiconductors such as logic chips, transistors, or other individual components. A module can also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, graphics processing units, or the like.

[0185] A module can also be implemented at least partially in software for execution by various types of processors. For example, an identified unit of executable code can comprise one or more physical or logical blocks of computer instructions that can be organized, for example, as objects, procedures, or functions. Nevertheless, the executable files of the identified modules need not be physically located together and can comprise different instructions stored in different locations that, when logically combined, comprise the module and achieve the specified purpose for the module. Further, a module can be stored on a computer-readable medium, which can be, for example, a hard disk drive, a flash device, random access memory (RAM), a tape, or any other such medium used for storing data.

[0186] In fact, the modules of executable code can be single instructions or multiple instructions, and further, they can be distributed across several different memory devices among different programs in several different code segments. Similarly, the arithmetic data can be identified and shown within a module herein, embodied in any suitable form, and organized within any suitable type of data structure. The arithmetic data can be collected as a single data set or distributed in different locations including different storage devices and can exist at least partially as mere electronic signals on a system or network.

[0187] It will be readily understood that the components of the present application schematically described and shown in the figures herein can be arranged and designed in a wide variety of different configurations. Accordingly, the detailed description of the embodiments is not intended to limit the scope of the present application as claimed and represents only selected embodiments of the present application.

[0188] Those skilled in the art will readily understand that the above can be performed in different orders of steps and / or with hardware elements of different configurations than those disclosed. Accordingly, although the present application is described based on these preferred embodiments, it will be apparent to those skilled in the art that certain modifications, variations, and alternative structures are obvious.

[0189] Although preferred embodiments of the present application are described, the described embodiments are merely exemplary, and it should be understood that the scope of the present application should be determined only by the appended claims when considering all equivalents and modifications (such as protocols, hardware devices, software platforms, etc.) of the scope of the appended claims.

Claims

1. Scanning the cabin of a vehicle via a scanning device that generates a scan including image data of the interior of the vehicle and the exterior of the vehicle near at least one vehicle door, The scan device transmits the scan to the vehicle's hardware system via the vehicle's Controller Area Network (CAN) bus. The system executes a neural network on the scan via the in-vehicle hardware system, and the execution of the neural network simultaneously determines the prediction of the presence of a living organism for each seat and the prediction of whether the living organism for each seat is a child, and outputs a Boolean indicator for the prediction of the presence of a living organism for each seat and another Boolean indicator for the prediction of whether a child is present for each seat. The vehicle's communication module is transmitted via the CAN bus to the vehicle's communication module, which provides information regarding the prediction of the presence of life in each seat and the prediction of whether the organism in each seat is a child. The communication module wirelessly transmits information to a mobile device regarding the prediction of the presence of a living organism in each seat and the prediction of whether the living organism in each seat is a child. Methods that include...

2. The method according to claim 1, wherein the simultaneous determination includes simultaneously determining the presence prediction of each seat in the footwell of the vehicle.

3. The method according to claim 1, wherein the scan captures at least one point cloud center relating to at least one of the foot space and head space of the cabin.

4. The method according to claim 1, wherein the scan uses a predetermined rectangular boundary for the at least one seat inside the vehicle to determine the occupancy of space and the occupancy of each seat.

5. The method according to claim 1, further comprising communicating the log of the scan to a server via the CAN bus, wherein the server updates a cabin condition model based on the log of the scan.

6. A system, Includes a processor, said processor, The vehicle's cabin is scanned via a scanning device that generates a scan including image data of the interior of the vehicle and the spatial area outside the vehicle near at least one vehicle door. The scan device transmits the scan to the vehicle's hardware system via the vehicle's Controller Area Network (CAN) bus. A neural network is executed on the scan via the in-vehicle hardware system, and the execution of the neural network simultaneously determines the prediction of the presence of a living organism for each seat and the prediction of whether the living organism for each seat is a child, and outputs a Boolean indicator for the prediction of the presence of a living organism for each seat and another Boolean indicator for the prediction of whether a child is present for each seat. Information regarding the prediction of the presence of life in each seat, and the prediction of whether the organism in each seat is a child, is transmitted to the vehicle's communication module via the CAN bus. The communication module wirelessly transmits information to a mobile device regarding the prediction of the presence of a living organism at each seat and the prediction of whether the organism at each seat is a child. A system configured in such a way.

7. The system according to claim 6, wherein the processor is configured to determine the presence prediction of each seat in the footwell of the vehicle.

8. The system according to claim 6, wherein the scan captures at least one point cloud center relating to at least one of the foot space and head space of the cabin.

9. The system according to claim 6, wherein the scan determines the occupancy of space and occupancy of each seat by utilizing a predetermined rectangular boundary for at least one seat inside the vehicle.

10. The system according to claim 6, wherein the processor is configured to communicate the log of the scan to a server via the CAN bus, and the server updates the cabin status model based on the log of the scan.

11. A non-temporary computer-readable medium comprising instructions, wherein, when read by a processor, the instructions provide to the processor: Scanning the vehicle cabin via a scanning device that generates a scan including image data of the interior of the vehicle and the spatial area outside the vehicle near at least one vehicle door, The scan device transmits the scan to the vehicle's hardware system via the vehicle's Controller Area Network (CAN) bus. The system executes a neural network on the scan via the in-vehicle hardware system, and the execution of the neural network simultaneously determines the prediction of the presence of a living organism for each seat and the prediction of whether the living organism for each seat is a child, and outputs a Boolean indicator for the prediction of the presence of a living organism for each seat and another Boolean indicator for the prediction of whether a child is present for each seat. The vehicle's communication module is transmitted via the CAN bus to the vehicle's communication module, which provides information regarding the prediction of the presence of life in each seat and the prediction of whether the organism in each seat is a child. The communication module wirelessly transmits information to a mobile device regarding the prediction of the presence of a living organism in each seat and the prediction of whether the living organism in each seat is a child. A non-temporary computer-readable medium that enables the operation of [the process].

12. The non-temporary computer-readable medium according to claim 11, wherein the simultaneous determination includes simultaneously determining the presence prediction of each seat in the footwell of the vehicle.

13. The non-temporary computer-readable medium according to claim 11, wherein the scan captures at least one point cloud center relating to at least one of the foot space and head space of the cabin.

14. The non-temporary computer-readable medium according to claim 11, wherein the scan utilizes a predetermined rectangular boundary for at least one seat inside the vehicle to determine the occupancy of space and the occupancy of each seat.

Images