Emergency callback system for vehicle initiated emergency calls for emergency probability determination

Abstract

A background security call device includes: a memory, a factor table containing factors related to the host vehicle and the user, and a call event file generated when an emergency call is initiated; an interactive virtual machine assistant (IVMA) module that calls back the user or host vehicle that initiated the emergency call and, while attempting to establish a connection to communicate with the user or during a current call with the user, determines whether the emergency call is related to an actual emergency, a non-emergency situation, or is undetermined; and an unconfirmed emergency module that, in response to an undetermined emergency call, determines the probability that the emergency call is related to an actual emergency situation based on multiple factors, and, based on the probability, sends the current call and / or call event file to a non-emergency advisor device or an emergency advisor device.

Description

Technical Field

[0001] The information provided in this section is intended to provide a general overview of the background to this disclosure. The work of the inventors currently listed in this section, and descriptions that may not conform to prior art at the time of application, are not expressly or implied to be considered prior art in conflict with this disclosure.

[0002] This disclosure relates to vehicle-mounted emergency communication systems. Background Technology

[0003] The vehicle may be equipped with an emergency communication system configured to communicate with a backend provider regarding the services offered. In an emergency, the vehicle user (e.g., the driver) can press the emergency button to contact a trained advisor. The trained advisor can: gather detailed information about the emergency, vehicle status, occupant status, and environmental conditions; provide guidance to the user and / or occupants; and, if necessary, contact emergency services for directions to the vehicle's location. Services provided by the trained advisor include security, emergency, navigation, and diagnostic services. Summary of the Invention

[0004] This application discloses a background security call device, comprising: a memory configured to store: i) at least one factor table including multiple factors related to a host vehicle and a user of the host vehicle; and ii) a call event file, wherein the call event file is generated when a user initiates an emergency call through the host vehicle's emergency interface or when the host vehicle initiates an emergency call; an interactive virtual machine assistant (IVMA) module configured to call back at least one of the user who initiated the emergency call and the host vehicle, and, during an attempt to establish a connection to communicate with the user or during a current call with the user, determine whether the emergency call is i) related to an actual emergency, ii) related to a non-emergency, or iii) undetermined; and an unconfirmed emergency module configured to: i) in response to an undetermined emergency call, determine the probability that the emergency call is related to an actual emergency based on multiple factors; and ii) based on the probability, send one of the current call and the call event file to a non-emergency advisor device or an emergency advisor device.

[0005] Among other features, the unconfirmed emergency module is configured to: send at least one of the current call and call event files to the emergency advisor device in response to a probability greater than a predetermined threshold; and send at least one of the current call and call event files to the non-emergency advisor device in response to a probability less than or equal to the predetermined threshold.

[0006] Among other features, the unconfirmed emergency module is configured to: i) weight multiple factors, and ii) determine the probability based on the weighted multiple factors.

[0007] Among the other features, several factors correspond to categories, including built-in or aftermarket paired device categories, telematics connectivity data factor categories, vehicle usage data factor categories, and context button usage factor categories.

[0008] Among other features, the unconfirmed emergency module is configured to: i) add several plus signs related to factors observed in each of the multiple categories to obtain multiple totals; ii) weight the multiple totals using their respective weights; iii) sum the weighted multiple totals to provide a result total; iv) compare the result total with a predetermined threshold; and v) based on the comparison, send at least one of the current call and call event files to a non-emergency advisor device or an emergency advisor device.

[0009] Among other features, the IVMA module is configured to determine, based on communication with the user and multiple factors, whether to: i) terminate the emergency call, ii) forward the emergency call to the Emergency Advisor device, or iii) set a flag to determine the probability.

[0010] Among other features, the background security call device also includes a control module configured to collect sensor data from the host vehicle and set the values ​​of at least some of a plurality of factors based on the sensor data.

[0011] Among other features, the Interactive Virtual Machine Assistant (IVMA) module implements an artificial intelligence neural network for speech recognition when calling back the user and confirming whether the emergency call is related to an actual emergency or a non-emergency situation.

[0012] Among other features, the Unconfirmed Emergency module is configured to implement a point-based algorithm to determine the probability based on a confidence threshold and multiple context-based inputs when the IVMA module cannot confirm whether an actual emergency or non-emergency situation exists.

[0013] Among other features, the unconfirmed emergency module is configured to determine probabilities based on the outputs of at least one camera, at least one microphone, at least one biometric sensor, radar sensor, at least one occupant sensor, at least one door sensor, and at least one mobile device.

[0014] Among other features, the unconfirmed emergency module is configured to determine the probability based on diagnostic fault codes, current driving data, previous call or voice recognition attempts, driving mode, navigation route, active safety events, and hazard indicator status.

[0015] Among other features, the unconfirmed emergency module is configured to determine probabilities based on network condition data, local emergency or crisis event data, weather, traffic information, geographic environment data, backend availability, call outcome data using other connection platforms, sourced 911 center data, and crowdsourced or third-party partner emergency data.

[0016] Among other features, the unconfirmed emergency module is configured to determine the probability based on the use of the emergency button, the use of the end call button, the duration of button presses, and the time between pressing the emergency button and pressing the end call button.

[0017] Among other features, an emergency callback system is disclosed, which includes: a background security call device; an emergency advisor device; and a non-emergency advisor device.

[0018] Among other features, an emergency callback method is disclosed, comprising: storing: i) at least one factor table including multiple factors related to the host vehicle and the user of the host vehicle; and ii) a call event file, wherein the call event file is generated when the user initiates an emergency call through the host vehicle's emergency interface or when the host vehicle initiates an emergency call; calling back at least one of the user who initiated the emergency call and the host vehicle via an interactive virtual machine assistant (IVMA) module, and determining whether the emergency call is i) related to an actual emergency, ii) related to a non-emergency, or iii) undetermined during an attempt to establish a connection to communicate with the user or during a current call with the user; and in response to the emergency call being undetermined, determining the probability of whether the emergency call is related to an actual emergency based on multiple factors; and sending one of the current call and the call event file to a non-emergency advisor device or an emergency advisor device based on the probability.

[0019] Among other features, the emergency callback method further includes: sending at least one of the current call and the call event file to an emergency advisor device in response to a probability greater than a predetermined threshold; and sending at least one of the current call and the call event file to a non-emergency advisor device in response to a probability less than or equal to a predetermined threshold.

[0020] Among other features, the emergency call-back method includes: weighting multiple factors; and determining the probability based on these weighted factors. These factors correspond to the following categories: built-in or aftermarket pairing device category, telematics connection data factor category, vehicle usage data factor category, and context button usage factor category.

[0021] Among other features, the emergency callback method also includes: adding several plus signs associated with factors observed in each of the multiple categories to obtain multiple totals; weighting the multiple totals using their respective weights; summing the weighted multiple totals to provide a result total; comparing the result total with a predetermined threshold; and based on the comparison, sending at least one of the current call and the call event file to a non-emergency advisor device or an emergency advisor device.

[0022] Among other features, the emergency callback method also includes: when the IVMA module cannot confirm whether an emergency or non-emergency situation exists, implementing a point-based algorithm to determine the probability based on a confidence threshold and multiple context-based inputs.

[0023] Among other features, an in-vehicle emergency communication system is disclosed, comprising: a telematics module configured to communicate with a background safety call device; a sensing module configured to collect vehicle status information and environmental condition information; an emergency interface including an emergency button; and an emergency module configured to receive input signals from the emergency interface, initiate an emergency call through the telematics module, terminate the emergency call before the background safety call device confirms whether it is an emergency or a non-emergency situation, and receive a callback from an interactive virtual machine assistant of the background safety call device to confirm whether it is an emergency or a non-emergency situation.

[0024] The further scope of this disclosure will become apparent from the detailed description, claims, and drawings. The detailed description and specific examples are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0025] This disclosure will be more fully understood through the following detailed description and accompanying drawings, in which: Figure 1 This is a functional block diagram of an example emergency callback system including a background security call device, based on this disclosure; Figure 2 This is a functional block diagram of a vehicle including an example in-vehicle emergency communication system according to this disclosure; Figure 3 This is a functional block diagram of an example emergency interface and emergency module for a vehicle, based on this disclosure; Figure 4 It shows Figure 1 An example of an emergency callback method implemented by a background security call device; Figure 5 It shows Figure 2 An example emergency call initiation method implemented in a vehicle-mounted emergency communication system; and Figure 6 It shows Figure 2An example callback response method implemented in a vehicle-mounted emergency communication system.

[0026] In the accompanying drawings, reference numerals may be reused to identify similar and / or identical elements.

[0027] Specific implementation method Vehicle users can press the emergency button on the in-vehicle emergency communication system to call an emergency advisor. Users may accidentally press the emergency button or change their mind about calling the advisor, so they can quickly press the end-of-call button. Due to the high communication speeds of 4G and 5G systems, the call is usually received by the backend and / or Public Safety Answering Point (PSAP) before the user ends the call. Typically, users cannot end an accidentally triggered emergency call before the backend receives call notification. The call arrives faster than the user realizes, so the backend and / or PSAP will call the user back to confirm if it was an actual emergency call. Sometimes emergency calls may be unexpectedly dropped (or terminated). Therefore, a response protocol can be established to call back each emergency call that has not yet been handled by an emergency advisor. Emergency calls that have ended but not yet been handled by an emergency advisor can be referred to as “lost emergency calls.” The emergency events corresponding to emergency calls that have ended but not yet been handled by an emergency advisor can be referred to as “tracked emergency call events” and / or “pending emergency call events.”

[0028] Emergency button presses are often caused by accidental button presses and are frequently unrelated to an actual emergency. Up to half of emergency calls end before the system receives the call notification, the user is connected to a live agent, and an emergency advisor handles the call. However, every call for help needs to be treated as an emergency first to ensure proper handling of the actual situation. This can put pressure on when and where to focus resources on dispatching highly trained emergency advisors. Emergency advisors are highly trained professionals whose time is extremely valuable. Therefore, to minimize the time emergency advisors spend calling back users / vehicles in lost emergency calls, non-emergency advisors can call back the user / vehicle first and then transfer the call to an emergency advisor if an actual emergency occurs. However, sometimes when a non-emergency advisor calls back, the user does not respond. This could be because: the user is nervous and does not want to respond; the user has left the vehicle; or the user is unable to respond. Therefore, a non-emergency advisor may need to: call back the original emergency call number multiple times; dial one or more other phone numbers registered with the vehicle user and / or owner; and so on. Resolving lost emergency calls can take a considerable amount of time.

[0029] Another approach is to lock the in-vehicle emergency communication system button (e.g., the end-of-call button) after the emergency button is pressed, preventing the user from ending the call before the emergency advisor handles it. This significantly reduces the number of follow-up calls. However, this results in emergency advisors receiving and handling many calls that are not actually emergencies, wasting a considerable amount of their time.

[0030] The examples cited in this article include a vehicle emergency callback system that determines the probability that a lost emergency call corresponds to an actual emergency. The vehicle emergency callback system calls back the user / vehicle (i.e., the original phone number that initiated the original emergency call) and / or other archived phone numbers associated with the original phone number. This is done via a virtual machine assistant (i.e., not a human assistant). The virtual machine assistant then determines whether to terminate the corresponding call and track the emergency call event, or to transfer the call to an emergency advisor, or to determine the probability that the lost emergency call corresponds to an actual emergency and transfer the call to a non-emergency advisor. Therefore, the number of lost calls unrelated to actual emergencies and ultimately handled by an emergency advisor is significantly reduced and / or eliminated.

[0031] For example, in the event of a lost emergency call, an AI-powered automated voice calling system can be used to call the user back. AI is used for voice recognition. The vehicle emergency call system can confirm an emergency, confirm the absence of an emergency, or determine the probability of an emergency based on the context and react accordingly. Upon confirmation of an emergency, the call is transferred to a professional emergency advisor. Upon confirmation of the absence of an emergency, the call ends or is transferred to a non-emergency advisor. If the callback attempt fails, and / or the emergency status cannot be clearly determined during the callback process, the vehicle emergency callback system uses a weighted algorithm and one or more defined probability factors highlighted in an emergency probability factor table to assess whether the call is more likely to be an emergency or less likely to be a non-emergency. If the assessment deems it more likely to be an emergency, the current call is sent to a professional emergency advisor. If the system cannot connect the call, the attempt to contact the user and handle the emergency is recorded in a call event file and sent to an emergency advisor. If the assessment deems it less likely to be a non-emergency, the current call and / or call event file (referred to as a “case file”) is sent to a non-emergency advisor who does not possess emergency service skills. If the non-emergency advisor believes the event constitutes an actual emergency, he / she can send the call and / or case file to an experienced emergency advisor.

[0032] Figure 1An emergency callback system 100 is shown, which includes a background security call device 102, a public safety response device (also known as a PSAP) 104, a non-emergency advisor device 106, an emergency advisor device 108, and a vehicle 110. The background security call device 102 can communicate with devices 104, 106, 108, and vehicle 110, and may include a control module 112, a memory 114, and a transceiver 116. The control module 112 can implement emergency callback methods, an example of which is shown below. Figure 4 As shown. Control module 112 may include interactive virtual machine assistant (IVMA) module 120 and unconfirmed emergency module 122.

[0033] IVMA module 120 calls back the phone number associated with the lost call and / or other archived phone numbers associated with the lost call's phone number. The IVMA module may include and / or implement an artificial intelligence (AI) neural network for speech recognition. Upon callback, IVMA module 120 asks the vehicle occupant whether the original emergency call was an actual emergency, among other questions. Based on previous answers, IVMA module 120 determines whether the call was an actual emergency. Based on the occupant's feedback, IVMA module 120: terminates the call; determines whether the call is related to an emergency; determines that the call is not related to an emergency; and / or determines the emergency status as undetermined. When IVMA module 120 cannot reach someone or cannot understand what the vehicle occupant (or the person answering the phone) is saying, IVMA module 120 may determine the emergency status of the call as undetermined. When the call status is undetermined and / or not considered an emergency, unconfirmed emergency module 122 may determine the probability that the call was an emergency (or the probability that the call was not an emergency). When the probability exceeds a predetermined (or confidence) threshold, the call will be considered an emergency and sent to one of the emergency advisor devices 108. When the probability does not exceed the threshold, the call will be considered a non-emergency and terminated or sent to one of the non-emergency advisor devices 106.

[0034] In the event of an emergency, the emergency advisor device 108 can contact the public safety response device 104 to request police, fire, and / or medical emergency services. The public safety response device 104 can be implemented and / or integrated with the back-end safety call device 102, or it can be a standalone device located away from the back-end safety call device 102. The public safety response device 104 can be installed in a call center or dispatch center that handles emergency calls and coordinates emergency responses.

[0035] The memory 114 can store table 130, customer file 132, and call event file 133. An example of a table is provided below and referred to as Table 1-10.

[0036]

[0037] Table 1 - Emergency Confirmation, Connection, and Action Table Table 1 provides the tracking status of the emergency call, including whether the end-of-call button was pressed, the probability that the call was related to an actual emergency, and the actions taken. These actions can be executed by the control module 112.

[0038]

[0039] Table 2 - Emergency Probability Factors Table 2 shows examples of factors that can be considered when determining the probability that a call is related to an actual emergency. In one embodiment, the more '+' signs a factor has, the higher its weight in determining the probability. The more '-' signs a factor has, the lower its weight in determining the probability. In one embodiment, each factor is weighted.

[0040] Table 3-6 below provides an example of a lost emergency call, containing factors indicating a high probability of an emergency occurring. These factors can be categorized into four classes, such as: Built-in or Aftermarket Pairing Device (BAPD) category, Telematics Connection Data (TCD) category, Vehicle Usage Data (VUD) category, and Context Button Use (CBU) category. In one embodiment, each category is weighted. Table 3-6 lists the weights for each category as an example. Table 3-6 includes an "Observed" column indicating whether a factor has been observed. An "X" in this column indicates that the factor has been observed. When a factor is "deterministic," it can be determined by onboard sensors. Table 3-6 also includes a total value and a weighted total value. The total value is equal to the number of '+' signs in the observed factors. The weighted total value is the product of the category weight and the total value. In one embodiment, the weighted totals are summed to obtain the final total. If the resulting total is greater than a predetermined threshold (e.g., 10), the call is considered relevant to an actual emergency and sent to one of the Emergency Advisor devices 108.

[0041]

[0042] Table 3 - BAPD Table

[0043] Table 4 - TCD Table

[0044] Table 5 - VUD Table

[0045] Table 6 - CBU Table For example, Expression 1 can be used to determine and compare the total result value with a predetermined threshold (e.g., 10). If the total result value is greater than the predetermined threshold, the probability that the call is related to an emergency is high. If the total result value is less than or equal to the predetermined threshold, the probability that the call is related to an emergency is low. Expression 2 includes example values ​​for BAPD, TCD, VUD, and CBU based on Tables 3-6 above.

[0046] (1) (2) In the examples in Table 3-6 above, the probability of an emergency is high. Therefore, the case is sent to the Emergency Advisor device.

[0047] Table 7-10 below are examples of lost emergency calls, which include factors indicating a low probability of an emergency.

[0048]

[0049] Table 7 - BAPD Table

[0050] Table 8 - TCD Table

[0051] Table 9 - VUD Table

[0052] Table 10 - CBU Table Expression 3 includes the weights of the categories and example values ​​for BAPD, TCD, VUD, and CBU based on Tables 7-10 above.

[0053] (3) In the example shown in Table 7-10, the probability of an emergency is low. Therefore, the case was sent to a non-emergency consultant.

[0054] The aforementioned predetermined thresholds are calibrable. The weights (or weighting coefficients) are also calibrable. In one embodiment, weighting coefficients are not used. This is actually achieved by setting each weight to 1. The examples in Tables 3-6 above illustrate a scenario where a user might have a heart attack. This is based on the in-vehicle device detecting biometrics and indicating that the call may have been interrupted based on button usage and network conditions. The examples in Tables 7-10 above are examples of a user accidentally pressing the emergency button in the vehicle, and are determined based on contextual button usage.

[0055] Vehicle 110 may include an emergency module 140, for example, when the emergency button in vehicle 110 is pressed, the emergency module 140 communicates with the control module 112. Figure 3 An example of an emergency button is shown. Figure 5-6 Example methods executed by each emergency module 140 are shown.

[0056] Figure 2 The main vehicle 200 is shown, which includes an example onboard emergency communication system 201 for making emergency calls to the background, as described herein. Figure 1 Each vehicle (110) can be configured similarly to the main vehicle (200). The following is a combination of... Figure 5-6 The method describes some example operations of the vehicle-mounted emergency communication system 201, which can be implemented by the emergency module 202 of the vehicle control module 203. The vehicle control module 203 includes a driving module 204, which implements the perception module 205 and the emergency module 202.

[0057] The driving module 204 and / or the perception module 205 perform the following operations: perception (or situation) to determine action; target detection, recognition, classification, and graphic and visual recognition operations; data retrieval, collection, and organization operations; interactive timing operations; assisted driving operations; image overlay operations; dialogue operations, including providing voice, text, and / or tactile messages; and so on. The perception module 205 can determine the status of the main vehicle 200, environmental conditions, weather, and the status and position of other nearby vehicles and objects. The vehicle control module 203 can perform various operations based on the judgments of modules 202, 204, and 205 and the interaction with vehicle occupants such as the driver and / or passengers of the main vehicle 200.

[0058] The main vehicle 200 also includes one or more power supplies 209, a telematics module 206, an infotainment module 207, other control modules 208, and a propulsion system 210. The vehicle control module 203 can control the operation of the main vehicle 200, including the propulsion system 210 and other systems described below. The power supply 209 may include one or more battery packs, a generator, a converter, control circuitry, high and low voltage load terminals, etc., and one or more battery sensors 212 for detecting the status of the power supply 209, including voltage, current levels, and charging status.

[0059] The telematics module 206 provides wireless communication services within the main vehicle 200 and communicates wirelessly with service providers, network devices (e.g., cloud-based network devices, central office devices, and / or back-end devices), other vehicles, mobile devices, infrastructure devices, and other devices external to and / or internal to the main vehicle 200. The telematics module 206 may support Wi-Fi®, Bluetooth®, Bluetooth Low Energy (BLE), Ultra Wideband (UWB), Near Field Communication (NFC), cellular networks, Legacy Transmission Control Protocol (TCP), Long Term Evolution (LTE), and / or other wireless communications, and / or operate according to Wi-Fi®, Bluetooth®, BLE, UWB, NFC, cellular networks, and / or other wireless communication protocols. The telematics module 206 may include one or more transceivers 213 and a navigation module 214 having a Global Positioning System (GPS) and GNSS (or Global Navigation Satellite System) receiver 216. The navigation module 214 may include an inertial measurement unit (IMU) 217 ​​and an odometer / wheel sensor 219. Transceiver 213 communicates wirelessly with network devices inside and outside the main vehicle 200, including cloud-based network devices, central stations, back-end systems, and portable network devices. Transceiver 213 can perform pattern recognition, channel addressing, channel access control, and filtering operations.

[0060] Navigation module 214 executes a navigation application to provide navigation services. The navigation services may include location identification services to determine the location of the main vehicle 200. The navigation services may also include guiding the driver and / or instructing the main vehicle 200 to a selected location. Navigation module 214 may communicate with a central station to collect map information indicating traffic conditions, traffic object identification and location (e.g., sign location and type), route information, weather information, etc. For example, if the main vehicle 200 is a driver-assisted and / or autonomous vehicle, navigation module 214 may instruct vehicle control module 211 to proceed along a selected route to a selected destination. GPS and GNSS receiver 216 can provide: location information; speed and / or direction (or heading) of the main vehicle 200, other vehicles and objects (e.g., pedestrians and cyclists); and / or global clock timing information.

[0061] Emergency interface 221 is included and may include an emergency call button, an end call button, and one or more other buttons. Figure 3 An example of emergency interface 221 is shown. The emergency interface can be implemented using a rearview mirror, steering wheel, instrument panel, or center console interface, or other types of interfaces. Emergency module 202 can issue an emergency call based on the input signal generated by emergency interface 221.

[0062] The infotainment module 207 may include an audio system 222 and / or a video system, including one or more displays 220, or may be connected to the audio system 222 and / or video system. The displays 220 and audio system 222 may be part of a human-machine interface. The displays 220 may include an instrument panel and / or center console display, a head-up display, etc. In addition to the displays and audio system 222, haptic devices (e.g., steering wheel and / or seat vibration devices) may be used to interact with vehicle occupants (e.g., driver or passenger). This interaction will be described in more detail below. Information may be displayed, played, and / or indicated via the displays 220, audio system 222, haptic devices, and / or one or more other output devices.

[0063] The infotainment module 207 can provide various information, warnings, and proactive messages, including: status information; routing information, rerouting information, inquiring whether the master vehicle 200 should be rerouted from the current path to another path; whether driving operations are subject to autonomous control, restriction, and / or obstruction; gear lever status; upcoming and ongoing operations (e.g., braking, acceleration, turning); object (or obstacle) detection; vehicle status information; diagnostic information; prognostic information; entertainment functions and information; and so on. The infotainment module 207 can be used to guide the vehicle operator to a location, indicate trip estimates (e.g., distance to a selected destination), and other information.

[0064] The propulsion system 210 may include one or more torque sources, such as one or more electric motors and / or one or more engines (e.g., internal combustion engines). Figure 2 In the example shown, the host unit 200 includes an engine 230 and one or more electric motors 232. The torque sources are independently controlled. The propulsion system 210 includes a motor control system 234, which includes one or more electric motors 232 and a motor control module 236 that can control the operation of one or more electric motors 232 based on signals from the vehicle control module 211. The propulsion system 210 may include a gear selector and / or shifter 235 for setting the gear of the transmission 237 and / or setting the drive state of one or more torque sources. The gear selector and / or shifter 235 may be an electronic selector (e.g., one or more electric switches), a mechanical and / or electronic shifter, or other selectors and / or shifters. The propulsion system 210 can be controlled based on the position of the accelerator 239 (e.g., accelerator pedal, throttle, etc.).

[0065] Modules 202-208 can communicate with each other directly or indirectly via one or more buses 240, such as a Controller Area Network (CAN) bus and / or other suitable interfaces. Vehicle control module 203 can control the operation of vehicle modules, devices, and systems based on feedback from sensors 250 and information and / or instructions received from cloud network devices.

[0066] Sensor 250 may include external sensor 252, internal sensor 254, and other sensors 256. External sensor 252 may include radar and / or lidar sensor 258 and imaging and audio equipment (e.g., visible spectrum camera, long-wave infrared camera, short-wave infrared camera, ambient light sensor, and microphone or microphone array) 260. External sensor 252 can be used to detect objects outside the main vehicle 200 and / or objects in the driving path of the main vehicle 200.

[0067] Internal sensor 254 may include one or more internal imaging sensors (e.g., cameras) 263 and microphones or microphone arrays 264. Internal sensor 254 may be part of a driver monitoring system (DMS). Camera 263 may be used to detect, track, and / or monitor vehicle occupants, including detecting the occupant's position within the vehicle, anatomical features of the occupant (e.g., face, arms, hands, legs, etc.), head position, and / or eye position. Internal sensor 254 may include door sensors, seatbelt sensors, seat sensors, etc. Door sensors may indicate whether a door is open or closed. Seatbelt sensors may indicate whether a seatbelt is fastened. Seat sensors may include, for example, load sensors, strain gauges, and / or piezoresistive or piezoelectric sensors for detecting whether an occupant is in a particular seat and the occupant's weight.

[0068] Other sensors 256 may include a gear selector and / or shifter sensor 267, a vehicle speed sensor 268, an acceleration sensor (e.g., longitudinal and lateral acceleration sensors) 268, and a fuel level sensor 270 (as shown), as well as other sensors such as an inclinometer, an engine temperature sensor, and an engine oil pressure sensor. Other sensors may also be included, such as brake system sensors (brake sensor 279 shown) and steering system sensors (steering angle sensor 281 shown). The gear selector and / or shifter sensor 267 generates a signal indicating the state of the gear selector and / or shifter 235.

[0069] The driving module 204 can use machine learning for facial recognition, determining occupant limb positions, anatomical feature recognition, object classification (including identifying and / or classifying pedestrians, cyclists, and vehicles (e.g., oncoming vehicles)), and determining the possible trajectory of each detected, identified, and / or classified object. The driving module 204 can determine the position of objects based on feedback from the sensor 250.

[0070] The vehicle control module 203 may also include a mode selection module 272 and a parameter adjustment module 274. The parameter adjustment module 274 can be used to adjust the parameters of the main vehicle 200. The vehicle control module 203 can perform autonomous operation based on interaction with the vehicle occupants. This may be based on... Figure 1 The instructions received by the control module 112 of the background security call device 102, and / or from Figure 1 The vehicle control module 203 may control the propulsion system 210, braking system 276, and steering system 278, or via instructions from one of the devices 104, 106, and 108. For example, the vehicle control module 203 may operate fully or partially autonomously and may control the propulsion system 210, braking system 276, and steering system 278. In one embodiment, the vehicle control module 203 may control the operation of system 210, 276, and 278 based on or not based on interaction with vehicle occupants. The vehicle control module 203 may i) perform autonomous operations such as steering, braking, and acceleration, and / or ii) display and / or play sound messages, perform tactile operations via tactile devices, and / or output messages and / or corresponding signals via other output devices.

[0071] In one embodiment, the driving module 204 uses computer vision, machine learning, and cloud computing to identify, communicate, and assess scenarios in which the moving vehicle should yield to pedestrians, obstructed roads, and / or oncoming (right-of-way) traffic. The driving module 204 can visualize and consider pedestrians, road obstacles, and oncoming vehicles in real time and perform actions to enhance the situational awareness of the vehicle occupants.

[0072] The driving module 204 is configured to perceive the road ahead and the surrounding area based on the output of sensors (e.g., cameras, radar sensors and / or lidar sensors) and vehicle-to-everything (V2X) communications (including vehicle-to-vehicle communications, vehicle-to-mobile device communications, vehicle-to-infrastructure communications and other communications (e.g., vehicle-to-distributed network communications)).

[0073] The host 200 may also include a memory 280. The memory 280 may store sensor data 282, parameters 284, application programs 286, algorithms 288, historical data 290, on-board inputs 291, off-board inputs 292 from other devices outside the host vehicle 200, and other data 293. Sensor data and parameters may include occupant position, occupant weight, occupant heart rate, vehicle position, vehicle speed, vehicle acceleration, battery charge status, fuel level, etc. The application program 286 may include applications executed by modules 202-208.

[0074] Although the memory 280 and the vehicle control module 203 are shown as separate devices, they can be implemented as a single device. The memory 280 can also store historical data 290 and other data 293, such as driver driving mode, driver refueling mode, driver parking mode, driver passenger pick-up mode, other driver modes, data collected and / or generated by at least one module 202-206, traffic data, navigation data, map data, GPS data, route data, speed data, and acceleration data, etc.

[0075] The vehicle control module 203 can control the operation of the propulsion system 210, the video system including the display 220, the audio system 222, the tactile device, the braking system 276, the steering system 278, the seat system 396, and / or other devices and systems according to the parameters set by modules 202-208 and 274. The seat system 296 may include a seat sensor 297 for detecting the presence and / or changes in the occupant, such as changes in the driver's position. The vehicle control module 203 can set at least several parameters based on signals received from the sensor 250.

[0076] The vehicle control module 203 receives power from the power supply 209, which provides power to the propulsion system 210, braking system 276, steering system 278, seat system 296, etc. The vehicle control module 203 can control the power supplied to the haptic device, motor 232, braking system 276, steering system 278, seat system 296, and / or their actuators, for example, to adjust: motor speed, torque, and / or acceleration; braking pressure; steering wheel angle; pedal position; haptic device status; etc. This control can be based on the outputs of sensors 250, navigation module 214, GPS and GNSS receiver 216, data and information received from external devices, and data and information stored in memory 280.

[0077] The vehicle control module 203 can determine various parameters, including vehicle speed, motor speed, gear status, accelerator position, brake pedal position, regenerative (charging) power, automatic start / stop discharge power, and / or other information. The vehicle control module 203 can control the operation of systems 210, 276, and 278 according to these parameters. The driving module 204 can display vehicle status information according to these parameters.

[0078] The host vehicle 200 may include various systems for assisting the driver, performing autonomous operations, and / or instructing vehicle occupants on information about the host vehicle's environment. For example, the host system may include a navigation system that provides map information, indicating lane boundaries, street locations, speed limits, the geographic location of the selected destination, etc. The host system may provide the driver with driving instructions to the selected destination and / or perform autonomous operations, such as braking, steering, and acceleration, to drive the vehicle to the destination based on map information.

[0079] For example, the host vehicle 200 may include an object detection and collision warning system for detecting an impending object and executing countermeasures and / or taking evasive action to prevent a collision. The vehicle control module 203 determines the position of the object relative to the host vehicle 200 and the trajectory of the object and the host vehicle 200. If it is determined that the host vehicle 200 may collide with one of the objects, one or more warning signals may be generated to indicate the potential collision to the driver and / or the relevant object. These warnings may be provided in addition to the digital gateway and other information described herein. The vehicle control module 203 may also, or optionally, execute one or more other countermeasures (e.g., applying brakes to slow the host vehicle, changing the steering angle of the host vehicle, etc.) to prevent a collision.

[0080] Figure 3 Showing Figure 2 Emergency interface 300 and emergency module 202. Emergency interface 300 is... Figure 2 An example of emergency interface 221. Emergency interface 300 is an example of a rearview mirror assembly, including a rearview mirror 302, a call button 304, an end-of-call button 306, and an emergency call button 308. When a user makes a non-emergency call, they can press the call button 304. In the event of an emergency, the user can press the emergency call button 308. Pressing the end-of-call button 306 ends both non-emergency and emergency calls. Emergency module 202 performs operations based on the input signal generated by emergency interface 300 due to the pressing of one of buttons 304, 306, or 308.

[0081] Figure 4 Showing Figure 1 The emergency callback method implemented by the background security call device shown can iteratively perform the following operations.

[0082] At point 400, control module 112 receives an emergency call from the main vehicle and creates a call event file to track information such as whether the emergency call has been confirmed to be related to an emergency, whether the call has been dropped, and whether the emergency call has been handled by an emergency advisor when necessary.

[0083] At point 402, control module 112 determines whether a voice connection has been established with the user in the main vehicle. If yes, operation 404 is executed; otherwise, operation 408 is executed.

[0084] At 404, control module 112 determines whether the end call button has been pressed. If yes, operation 406 is executed; otherwise, operation 408 is executed.

[0085] At 406, control module 112 determines whether the call has been completed. If so, the method can end; otherwise, operation 408 can be performed.

[0086] At point 408, IVMA module 120 activates the interactive voice assistant (i.e., performs IVMA operation) to call the vehicle owner and / or a recorded phone number associated with the phone number used to make the emergency call. IVMA operation involves communicating with the vehicle's occupants and / or passengers to gather information and confirm whether the call is a non-emergency or emergency call. The recorded phone number may be another phone number of the user initiating the emergency call, the vehicle owner's or lessee's phone number, the vehicle's registered user's phone number, a vehicle occupant's phone number, etc.

[0087] At 410, IVMA module 120 determines whether an emergency has been confirmed. If so, operation 412 can be performed; otherwise, operation 414 can be performed.

[0088] At 412, IVMA module 120 can transmit call and / or call event files, along with corresponding collected information, to the emergency advisor's device. This may include sending a signal to the emergency advisor's device to allow the emergency advisor to handle the call. This method may terminate after operation 412.

[0089] At 414, IVMA module 120 and / or unconfirmed emergency module 122 determine whether a non-emergency situation has been confirmed. If not, operation 416 can be performed; otherwise, operation 420 can be performed. IVMA module 120 may set a flag to determine the probability of an emergency (or non-emergency situation).

[0090] At point 416, the unconfirmed emergency module 122 can determine the probability of an unconfirmed emergency (or the probability of a confirmed emergency) based on the original call, the result of the current call, collected sensor data, and other collected information, as described above. The probability is determined based on various factors and weights, as described in Tables 1-10 above.

[0091] At point 418, the unconfirmed emergency module 122 determines whether the probability is greater than a predetermined threshold. If so, operation 412 can be performed; otherwise, operation 420 can be performed.

[0092] At point 420, the unconfirmed emergency module 122 transmits the call, call event file, and related information to the non-emergency advisor's device. The non-emergency advisor can then reconfirm that it is not an emergency and resolve the user's concerns. If the non-emergency advisor determines that the call is indeed an emergency call, the non-emergency advisor can send the call, call event file, and related information to the emergency advisor's device via the non-emergency advisor's device.

[0093] After completing operations 406, 412, and 420, you can close the call event file, store it for future reference, or discard it.

[0094] Figure 5 Showing Figure 2 The emergency call initiation method implemented by the vehicle-mounted emergency communication system 201 can iteratively perform the following operations.

[0095] At point 500, the emergency button on emergency interface 221 is pressed, generating the input signal as described above, or emergency module 202 initiates a call request. An emergency call can be initiated by a user via emergency interface 221 of the main vehicle 200, or it can be initiated by the main vehicle 200 itself (referred to as an automatic emergency call). An emergency call initiated by the main vehicle 200 may be initiated by emergency module 202, for example, due to detection of the main vehicle's status, detection of a collision involving the main vehicle, environmental conditions, or detection of the status of the main vehicle's occupants.

[0096] At point 502, the emergency module 202 initiates a call to the backend via the remote information processing module 206 and begins establishing a voice connection with the backend.

[0097] At 504, emergency module 202 determines whether the end call button on the emergency interface has been pressed. If yes, the call ends and the method terminates. If no, operation 506 can be performed.

[0098] At point 506, emergency module 202 determines whether a voice connection has been established. If not, the method can end; otherwise, operation 508 can be performed.

[0099] At point 508, emergency module 202 continues to call the emergency advisor. Emergency module 202 communicates with the emergency advisor device via telematics module 206.

[0100] At 510, emergency module 202 determines whether the voice connection is lost and / or terminated. If so, operation 512 can be performed; otherwise, operation 508 can continue.

[0101] At 512, emergency module 202 determines whether the end call button has been pressed. If so, the method can end; otherwise, operation 514 can be performed.

[0102] At point 514, emergency module 202 may attempt to re-establish a voice connection with the emergency advisor. Operation 506 can be performed after operation 514.

[0103] Figure 6 Showing Figure 2 The call-back response method implemented by the vehicle-mounted emergency communication system 201 can be performed iteratively. Although the following method is mainly described for the case of a call-back to the telematics module 206 of the main vehicle 200, the method can be modified to call back to other devices (e.g., the user's mobile device, such as a mobile phone, tablet, wearable device, etc.).

[0104] At point 600, emergency module 202 receives a message from remote information processing module 206. Figure 1 The call (referred to as the current call) is made by the IVMA module 120 of the background security call device 102 shown.

[0105] At point 602, emergency module 202 determines whether the current call is to be answered by a vehicle occupant. The current call can be answered via emergency interface 221. For example, it can be answered by pressing [button / button]. Figure 3 Use button 304, 308, or emergency interface 221, or another button (not shown) on another device, to answer the current call. If the current call is answered, operation 604 can be performed; otherwise, the method can end.

[0106] At 604, the emergency module 202 continues to execute the current call to i) confirm whether the original emergency call is related to a non-emergency situation, or ii) determine that the original emergency call is actually related to an emergency situation.

[0107] At 606, emergency module 202 determines whether the original emergency call is related to an emergency. If so, operation 608 can be performed; otherwise, operation 610 can be performed.

[0108] At point 608, emergency module 202 continues the current call, allowing vehicle occupants to communicate with the emergency advisor. At point 610, emergency module 202 continues the current call, allowing vehicle occupants to communicate with the non-emergency advisor. This process may terminate after executing steps 608 and 610.

[0109] The examples provided in this article demonstrate how to effectively handle terminated or interrupted emergency calls without wasting the resources of skilled emergency counselors, while ensuring that all actual emergencies are properly handled by emergency counselors.

[0110] For example, the system uses automatic speech recognition and artificial intelligence to call back customers when a potential emergency call is interrupted or terminated.

[0111] For example, the system can automatically detect whether a lost phone belongs to a non-emergency situation, has a low probability of being an emergency, has a high probability of being an emergency, or is a confirmed emergency.

[0112] For example, the system will react accordingly after an emergency is identified, such as ending the call, sending the call and / or case to a non-emergency consultant, or sending the call and / or case directly to a skilled emergency consultant.

[0113] For example, the system uses voice recognition to confirm whether there is a genuine emergency or not through direct interaction with the customer.

[0114] For example, when the system cannot confirm the existence of an emergency through direct voice interaction with the user (or vehicle occupant), a point-based algorithm can be used to determine the high / low probability of an emergency by using a pre-determined (or confidence) threshold algorithm and multiple context-based inputs.

[0115] For example, the system can use multiple built-in or aftermarket paired devices to help determine the probability of an emergency, such as cameras, microphones, biometrics, radar, occupant sensors, door sensors, Bluetooth® phones / devices, etc.

[0116] For example, the system uses vehicle usage data to help determine the probability of an emergency. Vehicle usage data includes diagnostic codes, current driving data, previous call or voice recognition attempts, driving modes, navigation routes, active safety events, hazard indicator status, etc.

[0117] For example, the system uses telematics connection data to help determine the probability of an emergency. Telematics connection data includes network status data, local emergency or crisis event data, weather, traffic information, geographic environment data, backend availability, call outcome data using other connection platforms, data from the 911 center, and emergency data from crowdsourced or third-party partners.

[0118] For example, some systems use context buttons to help determine the probability of an emergency, such as frantically pressing the emergency button, using the end call button, the duration of key presses, and the time interval between an emergency call and the end call.

[0119] The above description is merely exemplary and is not intended to limit the disclosure, its application, or use. The broad doctrines of this disclosure can be implemented in various forms. Therefore, while this disclosure includes specific examples, its true scope should not be so limited, as other modifications will become apparent from a study of the accompanying drawings, specification, and the following claims. It should be understood that one or more steps in the method may be performed in different orders (or simultaneously) without altering the principles of this disclosure. Furthermore, while each embodiment has been described above, indicating that it possesses certain features, any one or more features described with respect to any embodiment of this disclosure may be implemented in any other embodiment and / or combined with features of any other embodiment, even if such combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and various combinations of one or more embodiments remain within the scope of this disclosure.

[0120] Spatial and functional relationships between components (e.g., between modules, between circuit elements, between semiconductor layers, etc.) are described using various terms, including “connection,” “joint,” “coupled,” “adjacent,” “closely adjacent,” “on top of,” “above,” “below,” and “arrangement.” Unless explicitly described as “direct,” the relationship between the first and second components described in the foregoing disclosure can be a direct relationship where no other intermediate components exist between the first and second components, or an indirect relationship where one or more intermediate components (spatial or functional) exist between the first and second components. The phrase “at least one of A, B, and C” as used herein should be interpreted as logical (A or B or C) using a non-exclusive logical OR, and should not be interpreted as “at least one of A, at least one of B, at least one of C.”

[0121] In a diagram, the direction of the arrows (as indicated by the arrows) typically represents the flow of information (e.g., data or instructions) that the illustration is concerned with. For example, when elements A and B exchange various kinds of information, but the information transmitted from element A to element B is relevant to the illustration, the arrow can point from element A to element B. This unidirectional arrow does not mean that there is no other information transmission from element B to element A. Furthermore, for information sent from element A to element B, element B can send an information request or information receipt confirmation to element A.

[0122] In this disclosure, including the following definitions, the term "module" or "controller" may be replaced with "circuit". The term "module" may refer to, constitute, or include the following: application-specific integrated circuit (ASIC); digital, analog, or mixed analog / digital discrete circuit; digital integrated circuit, analog integrated circuit, or mixed analog / digital integrated circuit; combinational logic circuit; field-programmable gate array (FPGA); processor circuitry (shared, dedicated, or grouped) for executing code; storage circuitry (shared, dedicated, or grouped) for storing code executed by the processor circuitry; other suitable hardware components capable of providing said functionality; or combinations of some or all of the above, such as in a system-on-a-chip.

[0123] This module may contain one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces that connect to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functionality of any given module in this disclosure can be distributed across multiple modules connected via the interface circuits. For example, multiple modules can implement load balancing. As another example, a server (also known as a remote or cloud) module can perform some functions on behalf of a client module.

[0124] As stated above, the term "code" can include software, firmware, and / or microcode, and may refer to programs, routines, functions, classes, data structures, and / or objects. A shared processor circuit refers to a single processor circuit executing some or all of the code from multiple modules. The term "group processor circuit" includes a processor circuit that, in combination with other processor circuits, executes some or all of the code from one or more modules. When referring to multiprocessor circuits, this includes multiple processor circuits on a discrete chip, multiple processor circuits on a single chip, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above. The term "shared memory circuit" refers to a single memory circuit that stores some or all of the code from multiple modules. The term "group memory circuit" includes a memory circuit used in conjunction with additional memory to store some or all of the code from one or more modules.

[0125] The term "computer-readable medium" is a subset of the term "computer-readable medium." As used herein, the term "computer-readable medium" does not include transient electrical or electromagnetic signals propagated through a medium (e.g., a carrier wave); therefore, the term "computer-readable medium" can be considered to be practically meaningful and non-transient. Non-limiting examples of non-transient, tangible computer-readable media include: non-volatile storage circuits (e.g., flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile storage circuits (e.g., static random access memory circuits or dynamic random access memory circuits), magnetic storage media (e.g., analog or digital magnetic tape or hard disk drives), and optical storage media (e.g., CDs, DVDs, or Blu-ray discs).

[0126] The apparatus and methods described in this disclosure can be implemented, in whole or in part, by a dedicated computer created by configuring a general-purpose computer to perform one or more specific functions embodied in a computer program. The aforementioned functional modules, flowchart components, and other elements can serve as software specifications, which can be converted into computer programs through the routine work of skilled technicians or programmers.

[0127] A computer program includes processor-executable instructions stored on at least one non-transitory, tangible, computer-readable medium. A computer program may also contain or depend on stored data. A computer program may include a basic input / output system (BIOS) that interacts with dedicated computer hardware, device drivers that interact with specific devices of the dedicated computer, one or more operating systems, user applications, background services, background applications, etc.

[0128] These computer programs may include: (i) descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation); (ii) assembly code; (iii) object code generated from source code by a compiler; (iv) source code executed by an interpreter; (v) source code compiled and executed by a just-in-time (JIT) compiler, and so on. For example, the source code can be written in the syntax of a variety of languages, including C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, JavaScript®, HTML5 (Hypertext Markup Language 5th Edition), Ada, ASP (Dynamic Server Web Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, MATLAB, SIMULINK, and Python®.

Claims

1. A background security call device, comprising: A memory configured to store: i) at least one factor table, the factor table comprising multiple factors related to the master vehicle and the user of the master vehicle; and ii) a call event file, wherein the call event file is generated when the user initiates an emergency call through the emergency interface of the master vehicle or when the master vehicle initiates an emergency call; An interactive virtual machine assistant (IVMA) module is configured to call back at least one of the user who initiated the emergency call and the host vehicle, and during an attempt to establish a connection to communicate with the user or during a current call with the user, determine whether the emergency call is i) related to an actual emergency, ii) related to a non-emergency, or iii) undetermined; as well as The unconfirmed emergency module is configured to: i) in response to the emergency call being unconfirmed, determine the probability that the emergency call is related to an actual emergency situation based on the multiple factors; ii) Based on the probability, send one of the current call and the call event file to the non-emergency advisor device or the emergency advisor device.

2. The background security call device according to claim 1, wherein, The unconfirmed emergency module is configured as follows: In response to the probability being greater than a predetermined threshold, at least one of the current call and the call event file is sent to the emergency advisor device; as well as In response to the probability being less than or equal to the predetermined threshold, at least one of the current call and the call event file is sent to the non-emergency advisor device.

3. The background security call device according to claim 1, wherein, The unconfirmed emergency module is configured to: i) weight the plurality of factors, and ii) determine the probability based on the weighted plurality of factors.

4. The background security call device according to claim 1, wherein, The multiple factors correspond to multiple categories, including built-in or aftermarket paired device categories, telematics connection data factor categories, vehicle usage data factor categories, and context button usage factor categories.

5. The background security call device according to claim 4, wherein, The unconfirmed emergency module is configured to: i) add several plus signs related to the factors observed in each of the multiple categories to obtain multiple totals; ii) weight the multiple totals using their respective weights; iii) Sum the weighted sums to provide a final total; iv) Compare the total result with a predetermined threshold; v) Based on the comparison, send at least one of the current call and the call event file to the non-emergency advisor device or the emergency advisor device.

6. The background security call device according to claim 1, wherein, The IVMA module is configured to determine, based on communication with the user and the multiple factors, whether to: i) terminate the emergency call, ii) forward the emergency call to the emergency advisor device, or iii) set a flag to determine the probability.

7. The background security call device according to claim 1 further includes a control module, the control module being configured to collect sensor data from the host vehicle and set the values ​​of at least some of the plurality of factors based on the sensor data.

8. The background security call device according to claim 1, wherein, The interactive virtual machine assistant IVMA module implements an artificial intelligence neural network for speech recognition when calling back the user and confirming whether the emergency call is related to an actual emergency or a non-emergency situation.

9. The background security call device according to claim 1, wherein, The unconfirmed emergency module is configured to implement a point-based algorithm to determine the probability based on a confidence threshold and multiple context-based inputs when the IVMA module cannot confirm whether an emergency or non-emergency situation exists.

10. The background security call device according to claim 1, wherein, The unconfirmed emergency module is configured to determine the probability based on the output of at least one camera, at least one microphone, at least one biometric sensor, radar sensor, at least one occupant sensor, at least one door sensor, and at least one mobile device.

Images