Electric vehicle charging with extended reality

Abstract

The present disclosure provides "Electric Vehicle Charging with Extended Reality." A computer includes a processor and a memory, and the memory stores instructions executable by the processor to track a sequence of steps to charge an electric vehicle (EV) by electric vehicle service equipment (EVSE), execute a large language model (LLM) to generate a handling operation for a user to handle a component of the EVSE, and output the handling operation to an extended reality (XR) device. The handling operation supports a step to charge the EV. The XR device displays the handling operation overlaid on the EVSE. The XR device highlights the component of the EVSE that is in the handling operation.

Description

Technical Field

[0001] This disclosure provides techniques for users to properly use electric vehicle service equipment (EVSE) for charging electric vehicles (EVs) or to troubleshoot problems with them. Background Technology

[0002] Electric vehicles require battery recharging, which can be done at public charging stations. The architecture and operating technology of charging station systems are emerging. For example, charging station systems can include various types of charging stations, such as Level 1 chargers, Level 2 chargers, and DC fast chargers (DCFC). Level 1 charging uses a 120-volt AC outlet and can charge a vehicle from 0% to 80% state of charge (i.e., a relative value indicating how much energy remains in the battery compared to its maximum capacity) in 40-50 hours (e.g., based on a 40-50 kWh battery). Level 2 chargers use a 240-volt AC outlet and can charge a vehicle from 0% to 80% state of charge in 4-10 hours. DCFC uses a DC outlet and can charge a vehicle from 0% to 80% state of charge in one hour or less. Summary of the Invention

[0003] A computer is programmed to track a sequence of steps involving charging an EV from an EVSE, execute a Large Language Model (LLM) to generate disposition actions for a user to handle the components of the EVSE, and output the disposition actions to an extended reality (XR) device. The disposition actions support the steps of charging the EV. The XR device displays the disposition actions overlaid on the EVSE. The XR device highlights the components of the EVSE involved in the disposition actions. The XR device can provide output in virtual reality or augmented reality. The XR device can be used by an EV operator to charge the EV, by a service technician to repair or troubleshoot the EVSE, or by a call center technician to instruct an EV operator about charging the EV. Using LLM provides disposition actions for various situations. When a user may be unfamiliar with the EVSE (e.g., an operator charging their EV for the first time at a public charging station) or when the EVSE's components may be complex (e.g., a technician troubleshooting internal components of the EVSE), the output of the XR device provides a technical means to direct the user to the correct component for the disposition action. These techniques can provide a greater chance of success for the user, even without any additional knowledge.

[0004] A computer includes a processor and a memory, wherein the memory stores instructions executable by the processor to track a sequence of steps for charging an electric vehicle (EV) by an electric vehicle service equipment (EVSE), to execute a large language model (LLM) to generate disposal operations for a user to handle components of the EVSE, and to output the disposal operations to an extended reality (XR) device. The disposal operations support the steps of charging the EV. The XR device displays the disposal operations overlaid on the EVSE. The XR device highlights the components of the EVSE that are involved in the disposal operations.

[0005] In the example, the sequence of steps may include a fault that interrupts the charging of the EV by the EVSE, and at least one of the disposal operations may be in response to the fault.

[0006] In the example, the XR device may be an augmented reality (AR) device, and the AR device may display the handling operation overlaid on the real-time display of the component of the EVSE.

[0007] In the example, the instructions may further include instructions to: determine the user's role relative to the EVSE, and the LLM may generate a user-specific role-based disposal action. In another example, the instructions may further include instructions to: select the role from a preset group that includes at least EV operators and service technicians. In yet another example, at least some of the disposal actions specific to the service technician may relate to components in the EVSE that are inaccessible to the EV operator.

[0008] In yet another example, the preset group may include call center technicians.

[0009] In the example, the instructions may also include instructions to perform the following: in response to the user's location at the EVSE, output the disposal operation in augmented reality.

[0010] In the example, the instructions may also include instructions to perform the following: in response to the user's position moving away from the EVSE, output the disposal operation in virtual reality.

[0011] In the example, the instructions may further include instructions to: in response to receiving a selection of the EVSE when the user is located away from the EVSE, transmit a message to the EVSE instructing the EVSE to perform a subset of the sequence of steps. In another example, the subset may include processing the user's payment data.

[0012] In another example, the instructions may also include instructions to display multiple alternative EVSEs in response to receiving an indication of a failure in the subset.

[0013] In the example, the LLM can be trained on training data that includes the technical documentation of the EVSE.

[0014] According to an embodiment, the instructions may further include instructions for performing the following: in response to receiving a selection of the EVSE from a plurality of EVSEs, tracking the sequence of steps by which the EVSE charges the EV.

[0015] A method includes tracking a sequence of steps for charging an electric vehicle (EV) by an electric vehicle service equipment (EVSE), performing a large language model (LLM) to generate disposal actions for a user to handle components of the EVSE, and outputting the disposal actions to an extended reality (XR) device. The disposal actions support the steps of charging the EV. The XR device displays the disposal actions overlaid on the EVSE. The XR device highlights the components of the EVSE that are involved in the disposal actions.

[0016] In the example, the sequence of steps may include a fault that interrupts the charging of the EV by the EVSE, and at least one of the disposal operations may be in response to the fault.

[0017] In one example, the instructions may further include determining the user's role relative to the EVSE, and the LLM may generate a user-specific role-based disposition. In another example, the method may further include selecting the role from a preset group that includes at least EV operators and maintenance technicians.

[0018] In the example, the method may further include outputting the disposal action in augmented reality in response to the user's location at the EVSE.

[0019] In the example, the method may further include outputting the disposal action in virtual reality in response to the user's position moving away from the EVSE. Attached Figure Description

[0020] Figure 1 This is a block diagram of an example system including electric vehicles.

[0021] Figure 2 This is an illustration of an example charging station where an electric vehicle is being charged.

[0022] Figure 3 This is an example extended reality output perspective of the processing operations used to charge electric vehicles at charging stations.

[0023] Figure 4 This is a flowchart of an example process for charging electric vehicles at a charging station.

[0024] Figure 5 This is a flowchart of an example process for resolving faults during charging. Detailed Implementation

[0025] Referring to the accompanying drawings, where like numbers indicate the same portions in several views, computers 105, 110, and 115 include a processor and a memory, and the memory stores instructions executable by the processor to track the sequence of steps for charging the electric vehicle (EV) 100 by the electric vehicle service equipment (EVSE) 205, execute a large language model (LLM) to generate handling operations for the user to handle the components 305 of the EVSE 205, and output the handling operations to an extended reality (XR) device 115. Computers 105, 110, and 115 may be the vehicle computer 105 of the EV 100, a remote computer 110 separate from the EV 100, and / or the XR device 115. The handling operations support the steps for charging the EV 100. The XR device 115 displays the handling operations overlaid on the EVSE 205. The XR device 115 highlights the components 305 of the EVSE 205 in the handling operation.

[0026] refer to Figure 1 EV 100 can be any passenger or commercial vehicle, such as a sedan, truck, SUV, crossover, van, minivan, taxi, bus, etc. EV 100 can be a plug-in hybrid electric vehicle (PHEV), a battery electric vehicle (BEV), or a range-extended BEV. EV 100 includes a vehicle computer 105, a communication network 120, sensors 125, a user interface 130, a transceiver 135, and a battery 140.

[0027] Battery 140 provides electrical power (i.e., electricity) to the components of EV 100. Specifically, battery 140 provides propulsion for EV 100, that is, it powers EV 100 to move itself around. As used in, for example, PHEVs or BEVs, battery 140 can be any suitable type for vehicle electrification, such as lithium-ion batteries, nickel-metal hydride batteries, lead-acid batteries, or supercapacitors. Battery 140 can be multiple batteries or cells wired together.

[0028] Vehicle computer 105 is a microprocessor-based computing device, such as a general-purpose computing device (including a processor and memory, electronic controllers, etc.), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a combination thereof. Typically, hardware description languages ​​such as VHDL (VHSIC (Very High Speed ​​Integrated Circuit) Hardware Description Language) are used in electronic design to describe digital and mixed-signal systems such as FPGAs and ASICs. For example, an ASIC is manufactured based on VHDL programming provided before manufacturing, while the logic components inside an FPGA can be configured based on VHDL programming (e.g., stored in memory electrically connected to the FPGA circuitry). Therefore, vehicle computer 105 may include a processor, memory, etc. The memory of vehicle computer 105 may include media for storing instructions executable by the processor and for electronically storing data and / or databases, and / or vehicle computer 105 may include structures such as those providing programming capabilities. Vehicle computer 105 may be multiple computers coupled together.

[0029] Vehicle computer 105 can transmit and receive data via communication network 120. Communication network 120 can be a controller area network (CAN) bus, Ethernet, WiFi, local area network (LIN), on-board diagnostic connector (OBD-II), and / or any other wired or wireless communication network. Vehicle computer 105 can be communicatively coupled to sensors 125, user interface 130, transceiver 135, and other components via communication network 120.

[0030] Sensor 125 can provide data about the operation of EV 100, such as wheel speed, wheel orientation, and transmission data (e.g., temperature, power consumption, state of charge of battery 140, etc.). Sensor 125 can detect the position and / or orientation of EV 100. For example, sensor 125 may include a Global Positioning System (GPS) sensor; an accelerometer, such as a piezoelectric system or microelectromechanical system (MEMS); a gyroscope, such as a rate gyroscope, a ring laser gyroscope, or a fiber optic gyroscope; an inertial measurement unit (IMU); and a magnetometer. Sensor 125 can detect the external world, including objects and / or characteristics of the environment surrounding EV 100, such as other vehicles, road lane markings, traffic lights and / or signs, road users, etc. For example, sensor 125 may include a radar sensor, an ultrasonic sensor, a scanning laser rangefinder, a light detection and ranging (LIDAR) device, and an image processing sensor (such as a camera).

[0031] User interface 130 presents and receives information to and from the operator of EV 100. User interface 130 may be located on the dashboard in the passenger compartment of EV 100, and / or anywhere easily visible to the operator. User interface 130 may include dials, digital readout devices, screens, speakers, etc., for providing information to the operator, such as known human-machine interface (HMI) elements. User interface 130 may include buttons, knobs, keypads, microphones, etc., for receiving information from the operator.

[0032] Transceiver 135 can be adapted to wirelessly transmit signals via any suitable wireless communication protocol, such as cellular, Bluetooth®, Bluetooth® Low Energy (BLE), Ultra Wideband (UWB), WiFi, IEEE 802.11a / b / g / p, Cellular-V2X (CV2X), Dedicated Short Range Communication (DSRC), other RF (radio frequency) communications, etc. Transceiver 135 can be adapted to communicate with a remote server (i.e., a server separate from and spaced from the vehicle). The remote server can be located outside the EV 100. For example, the remote server can be associated with another vehicle (e.g., V2V communication), with infrastructure components (e.g., V2I communication), with a first responder, with a mobile device associated with the operator of the EV 100, etc. The remote server can be a remote computer 110, an XR device 115, or an EVSE 205. Transceiver 135 can be a single device or can include separate transmitters and receivers.

[0033] Vehicle computer 105, remote computer 110, and / or XR device 115 may be communicatively coupled via network 145. Network 145 represents one or more mechanisms through which vehicle computer 105, remote computer 110, and / or XR device 115 can communicate with each other or with other remote servers. Therefore, network 145 may be one or more of a variety of wired or wireless communication mechanisms, including any desired combination of wired (e.g., cable and fiber optic) and / or wireless (e.g., cellular, wireless, satellite, microwave, and radio frequency) communication mechanisms, and any desired network topology (or topology utilizing multiple communication mechanisms). Exemplary communication networks include wireless communication networks (e.g., using Bluetooth®, IEEE 802.11, etc.), local area networks (LANs), and / or wide area networks (WANs) (including the Internet) to provide data communication services.

[0034] Remote computer 110 is a microprocessor-based computing device, such as a general-purpose computing device including a processor and memory. The memory of remote computer 110 may include media for storing instructions executable by the processor and for electronically storing data and / or databases, and / or remote computer 110 may include structures such as those providing programming capabilities. Remote computer 110 may be multiple computers coupled together. For example, remote computer 110 may be a mobile device. A mobile device is a portable computing device, such as a mobile phone (e.g., a smartphone) or a tablet computer. The mobile device is owned and carried by someone who may be the operator of EV100 or a technician who may be maintaining EVSE 205. As another example, remote computer 110 may be associated with a service center or call center that oversees EVSE 205.

[0035] XR device 115 is a device equipped to provide output in extended reality. Extended reality combines the physical world with a digital world, in which the digital world interacts with the physical world. Extended reality includes augmented reality, virtual reality, and mixed reality. From the user's perspective, augmented reality combines the real world with computer-generated content superimposed on it. Virtual reality provides the user with a computer-generated simulation of the physical world, possibly combined with additional digital features. Mixed reality is a combination of augmented reality and virtual reality.

[0036] For example, XR device 115 may be an augmented reality (AR) device (such as AR glasses), a portable computing device (such as a mobile phone (e.g., a smartphone) or a tablet computer equipped with AR output), or a display in EV 100 such as a head-up display (HUD) (e.g., as part of user interface 130). AR glasses have transparent lenses that allow users to see their surroundings. The lenses of the AR glasses also display content that the user sees while viewing the world through the lenses. The AR glasses can display content on the lenses at a specific size and position such that the content appears to the user to be in a particular location in the world. For example, the AR glasses can use attitude tracking to determine the position and orientation of the user's head relative to the surrounding environment (e.g., based on data from an inertial measurement unit (IMU), etc.). The portable computing device may include a camera and a screen. The portable computing device can display image data from the camera and content at a specific size and position on the image data on the screen such that the content appears to the user to be in a particular location in the world. The portable computing device can use attitude tracking to determine its position and orientation. The HUD may display information or graphics on the windshield of the EV 100, which is transparent and allows the user to see the surroundings of the EV 100. A HUD can display content on a lens at a specific size and position, making the content appear to the user as being in a specific location in the world. For example, a HUD can use data from sensor 125 to track the attitude of EV 100 relative to its surroundings.

[0037] XR device 115 includes a microprocessor-based computing device, such as a general-purpose computing device including a processor and memory. The memory of XR device 115 may include media for storing instructions executable by the processor and for electronically storing data and / or databases, and / or XR device 115 may include structures such as those providing programming capabilities. XR device 115 may include multiple computers coupled together.

[0038] refer to Figure 2Charging station 200 may include one or more EVSEs 205. An area surrounded by charging station 200 may be defined (e.g., as a perimeter), on which EVSEs 205 and other elements of charging station 200 are located. Charging station 200 may include one or more parking spaces 210 corresponding to the respective EVSEs 205. That is, parking spaces 210 are provided as areas where EV 100 can be parked while simultaneously receiving charging of battery 140 from the respective EVSE 205. Charging station 200 may also include areas where EV 100 can be parked and / or driven (e.g., while waiting to access one of the EVSEs 205) to stop and view another facility of charging station 200, to enter and leave charging station 200, etc. The area of ​​charging station 200 and any other areas thereof may be defined according to location coordinates, geofencing, or any other suitable method for defining location boundaries.

[0039] Each EVSE 205 is configured to charge the battery 140 of the EV 100. EVSE 205 typically charges one EV100 at a time. EVSE 205 draws power from the grid to deliver it to the battery 140 of the EV 100. EVSE 205 is typically stationary (i.e., fixed to a specific physical location and cannot be moved from it). One or more corresponding EVSE 205s in the charging station 200 can use any suitable mechanism for recharging the battery 140 of the EV 100 (e.g., plug-in connection, inductive charging, etc.). Plug-in connection involves connecting the plug 305a of the EVSE 205 (in...) to the battery 140 of the EV 100. Figure 3 (As shown in the diagram) The port connected to the EV 100. Inductive charging is a form of wireless power transfer that relies on electromagnetic induction. The component 305 for power transfer can be located below the parking space 210 corresponding to the EVSE 205.

[0040] refer to Figure 3The EVSE 205 may include a physical structure 310 to which components 305 of the EVSE 205 are mounted or connected. For the purposes of this disclosure, a “component” of the EVSE 205 is defined as any physical part of the EVSE 205 that a user can manipulate or interact with. Components 305 may include external components that may be visible to an operator of the EV 100 using the EVSE 205, and internal components that may be hidden by one or more panels 305f of the EVSE 205. External components may include one or more plugs 305a, one or more cables 305b leading to a corresponding plug 305a, a payment device 305c such as a credit card reader, a keypad 305d or other input device for inputting data, a screen 305e for displaying information, panels 305f, etc. Internal components (not shown) may include wiring, circuit boards, transformers, computing devices, and other electrical or computing components for transmitting power and processing data to facilitate power transmission.

[0041] To charge the EV 100 using the EVSE 205, the operator and the EVSE 205 together perform a sequence of steps for the EVSE 205 to charge the EV 100. These steps may include: transferring payment from the operator of the EV 100 to the operator of the EVSE 205; selecting a charging protocol; transferring power from the EVSE 205 to the EV 100; and terminating charging. For example, when no fault occurs, the actual steps performed may follow an expected sequence of steps. The anticipated sequence of steps includes: initiating charging by providing input to the EVSE 205 by the operator; outputting a payment instruction via the EVSE 205; inputting payment information by the operator; receiving payment information via the EVSE 205; verifying payment information via the EVSE 205; outputting a charging protocol selection instruction via the EVSE 205; inputting the selection of the charging protocol by the operator; configuring internal component 305 via the EVSE 205 to provide power via the selected protocol; connecting plug 305a to the EV 100 by the operator; transferring power from the EVSE 205 to the battery 140 via the EVSE 205; outputting a state of charge notification via the EVSE 205; outputting a notification that the battery 140 is fully charged via the EVSE 205; disconnecting plug 305a from the EV 100 by the operator; and outputting a notification that the charging process is complete via the EVSE 205. These steps can be preset by the operator of the EVSE 205 based on its functionality and programming. Some of the steps may be conditional upon the completion of an earlier step, or may occur simultaneously with other steps. Some of the steps in the foregoing example can be further subdivided.

[0042] The sequence of steps (as actually performed by the operator and EVSE 205) may include malfunctions that interrupt the charging of EVSE 205 to EV 100. The term "malfunction" in its computational sense is used as an incorrect step in the process that causes unintended behavior of the procedure or device. For example, a malfunction may occur because EVSE 205 fails to verify payment information, EVSE 205 fails to configure internal component 305 for the selected protocol, plug 305a is not connected or is incorrectly connected to EV 100, plug 305a is incorrectly connected to EV 100, and so on. A malfunction may cause the actual sequence of steps to deviate from the intended sequence of steps.

[0043] Computers 105, 110, and 115 are programmed to track a sequence of steps for charging the EV 100 by the EVSE 205. The intended sequence of steps can be stored in computers 105, 110, and 115, as well as in the computing device of the EVSE 205. Computers 105, 110, and 115 and / or the EVSE 205 can also store contingency steps corresponding to pre-specified faults; in other words, the sequence of steps is designed to occur in response to the occurrence of a pre-specified fault. For example, computers 105, 110, and 115 can track the sequence of steps by determining the state of the steps, such as which steps have been completed, which steps are in progress or pending, and which steps are not yet pending. A step may be pending if it is ready to be executed (e.g., if earlier steps necessary for the step have been completed, and / or if other prerequisites are met). A step is not pending if it has not been executed and is not ready to be executed (e.g., if earlier steps necessary for the step have not been completed, and / or if prerequisites are not met). Computers 105, 110, and 115 can receive data from EVSE 205 indicating the state of a step, such as indicating when a state change occurs in a step. When performing actions relative to EVSE 205, computers 105, 110, and 115 can repeatedly or continuously update the state of the step.

[0044] As described below, computers 105, 110, and 115 generate disposal operations for user disposal of component 305 of EVSE 205. For the purposes of this disclosure, a "disposal operation" is defined as a user-performed step of disposing of something. For example, a disposal operation for an operator of EV100 may include placing or inserting a payment card or mobile device into or onto payment device 305c, typing information into keypad 305d, inserting one of plugs 305a into a port of EV100, placing plug 305a back onto physical structure 310, and so on. As another example, a disposal operation for a technician repairing EVSE 205 may include removing or replacing panel 305f, disassembling or connecting wiring, adjusting internal components 305, etc. Thus, disposal operations support steps for charging EV100 (i.e., leading to progress from one step to the next). Handling actions may include malfunction-response actions such as using a different payment method after an attempt to pay cannot be verified, or unplugging and replugging plug 305a into the EV 100 port after no electrical connection to battery 140 of EV 100 is detected.

[0045] Computers 105, 110, and 115 are programmed to execute a Large Language Model (LLM) to generate disposal actions for user disposal of EVSE 205, part 305. The term "Large Language Model" is used in the machine learning sense of a computational model used for natural language processing tasks. An LLM takes a sequence of steps (e.g., the current state of the steps or the most recently executed step) as input and provides the next disposal action as output. The disposal action output by the LLM can be in the form of a text instruction.

[0046] LLMs can be trained on training data including technical documentation from EVSE 205. Technical documentation can include, for example, instruction manuals, service manuals, answers to frequently asked questions (FAQs), troubleshooting guides, etc. Therefore, an LLM can be trained to provide actions consistent with the recommendations in the EVSE 205 technical documentation. For example, an LLM can be a customized version of a pre-existing base model. In other words, an LLM can be a base model that has already been trained on a general text corpus and then further trained on technical documentation. An LLM can use any suitable base model as its foundation, such as GPT, LLaMA, Claude, Gemini, Nemotron, etc.

[0047] Computers 105, 110, and 115 can be programmed to determine a user's role relative to EVSE 205. For the purposes of this disclosure, a person's "role" is defined as their function or position (e.g., job) in a given context. For example, a user's roles relative to EVSE 205 may include EV operator, maintenance technician, call center technician, etc. Computers 105, 110, and 115 can select roles from a preset group that includes at least EV operators and maintenance technicians, and possibly call center technicians. Computers 105, 110, and 115 can determine a user's role based on login information provided by the user. For example, roles may be stored in a user's profile or account. Computers 105, 110, and 115 can select an EV operator without the user indicating that they have a different role (i.e., EV operator is the default role).

[0048] The LLM can generate user-specific handling actions. For example, the LLM can be trained to output handling actions based on data from the instruction manual in response to the user's role being an EV operator, and to output handling actions based on data from the service manual in response to the user's role being a maintenance technician. At least some of the handling actions specific to the maintenance technician relate to components 305 of the EVSE 205 that are inaccessible to the EV operator. For example, in response to the user's role being a maintenance technician, the LLM can output handling actions for removing panel 305f and adjusting wiring, circuit boards, transformers, computing devices, etc., inside the physical structure 310 of the EVSE 205. In response to the user's role being an EV operator, the LLM can avoid outputting handling actions for panel 305f or internal components 305 of the EVSE 205.

[0049] Computers 105, 110, and 115 are programmed to output actions to XR device 115. In general, computers 105, 110, and 115 (e.g., based on the user's location) determine whether to output actions in virtual reality or augmented reality. When outputting actions in VR or AR, XR device 115 displays the actions overlaid on EVSE 205.

[0050] The XR device 115 displays the processing operation overlay on the EVSE 205. The XR device 115 can output the processing operation at an apparent location corresponding to component 305 of the EVSE 205 mentioned in the processing operation. The output processing operation may include a highlighted display and / or identifier of component 305 mentioned in the processing operation. For example, the XR device 115 can display a processing operation for grasping the plug 305a by highlighting and marking the plug 305a, or the XR device 115 can display a processing operation for placing the card on the payment reader by highlighting and marking the payment reader, such as... Figure 3 As shown. The processing operations may be useful to the user, for example, locating the payment device 305c, distinguishing the appropriate plug 305a (e.g., Class 1 to Class 2 to DCFC), etc. In the output, the highlighted and marked areas appear to be overlaid on the EVSE 205.

[0051] When outputting in virtual reality, the XR device 115 displays the processing operation overlaid on the image of the EVSE 205. The image can be, for example, recorded video or computer-generated video.

[0052] When outputting in augmented reality, the XR device 115 displays the handling operation overlaid on the real-time display of component 305 of the EVSE 205 (in other words, the display of the EVSE 205 while it is currently present in the physical world). For example, the real-time display could be a video feed from the camera of the XR device 115 to the screen of the XR device 115 (in the case of a mobile device). As another example, the real-time display could be the EVSE 205 itself as seen through the lenses of AR glasses.

[0053] Computers 105, 110, and 115 can be programmed to determine the user's location. Specifically, computers 105, 110, and 115 can determine whether the user's location is at or away from EVSE 205. If the user is within reach of EVSE 205 (e.g., when EV 100 is parked in parking space 210 corresponding to EVSE 205), then the user is likely at EVSE 205. For example, computers 105, 110, and 115 can determine the location based on data from a GPS sensor of sensor 125 of EV 100, or a GPS sensor of remote computer 110 (if a mobile device) or XR device 115. If the location from the GPS sensor is within a threshold distance of a known location of EVSE 205, then the user is likely at EVSE 205; otherwise, the user is away from EVSE 205. In another example, computers 105, 110, and 115 can determine whether the transceiver 135 of EV 100, or the remote computer 110 (if a mobile device), or the XR device 115, is within range of the transmitter of EVSE 205. If the transceiver 135, the remote computer 110, or the XR device 115 is within range of EVSE 205, then the user is likely at EVSE 205; otherwise, they are away from EVSE 205.

[0054] Computers 105, 110, and 115 can be programmed to select virtual reality or augmented reality to output a processing operation based on the user's position. Computers 105, 110, and 115 can output a processing operation in augmented reality in response to the user's position being at EVSE 205. Computers 105, 110, and 115 can output a processing operation in virtual reality in response to the user's position moving away from EVSE 205.

[0055] Before a user (e.g., an EV operator) drives EV 100 to EVSE 205, computers 105, 110, and 115 can be programmed to assist the user in selecting EVSE 205 and performing at least some of the steps in a sequence. In general, computers 105, 110, and 115 can display data about charging station 200 and EVSE 205, facilitate selection of EVSE 205 at charging station 200, and remotely instruct the selected EVSE 205 to perform a subset of the steps before the user arrives at EVSE 205. Computers 105, 110, and 115 can facilitate switching the selection to a different EVSE 205 in response to a failure during the execution of the subset of steps.

[0056] Computers 105, 110, and 115 can display data about charging station 200 and / or EVSE 205. For example, a user can enter a request for charging nearby, and computers 105, 110, and 115 can display a list of charging stations 200 and, in response to a selection of one of the charging stations 200, display detailed data about the selected charging station 200. The list can be displayed as a list (e.g., ordered by proximity), a set of locations on a map, or both. Computers 105, 110, and 115 can populate the list with charging stations 200 within the driving range of EV 100 (i.e., the driving range that EV 100 can travel using the current state of charge of battery 140). Data about charging stations 200 may include, for example, the number of EVSE 205, the occupancy of EVSE 205, the frequency of EVSE 205 malfunctions, the driving time to charging station 200, the layout of charging station 200, the rating or reviews of charging station 200, etc. The XR device 115 can output a depiction of the charging station 200 and a simulation of its handling operations in virtual reality in the manner described above.

[0057] A user can select charging station 200 (e.g., by consulting data about nearby charging stations 200) for charging the EV 100. The user can input this selection using user interface 130 or a remote computer 110. The user can also select EVSE 205 at charging station 200, or computers 105, 110, and 115 can select EVSE 205 at the selected charging station 200 based on availability. Computers 105, 110, and 115 can transmit a request to charging station 200 to reserve the selected EVSE 205.

[0058] Computers 105, 110, and 115 can be programmed to transmit messages to EVSE 205 in response to receiving a selection of EVSE 205 when the user is positioned away from EVSE 205, instructing EVSE 205 to perform a subset of a sequence of steps. This subset of steps may include most or all of the steps prior to inserting plug 305a into EV 100 (or inductively delivering power). For example, the subset of steps may include processing the user's payment data and / or configuring EVSE 205 to charge EV 100. Further breaking down the steps, the subset of steps may include initiating charging by providing input to EVSE 205 via an EV operator, outputting a payment instruction via EVSE 205, inputting payment information via an EV operator, receiving payment information via EVSE 205, verifying payment information via EVSE 205, outputting a instruction to select a charging protocol via EVSE 205, inputting a selection of a charging protocol via an EV operator, and configuring internal components 305 via EVSE 205 to deliver power via the selected protocol.

[0059] Furthermore, computers 105, 110, and 115 are programmed to, in response to receiving a selection of an EVSE 205 from a plurality of EVSEs 205, track the sequence of steps by which the EVSE 205 charges the EV 100, as described above. Tracking includes steps performed before and after the user is at the EVSE 205.

[0060] By performing a subset of steps before EV 100 reaches EVSE 205, a user can completely switch to a different EVSE 205 or a different charging station 200 in the event of a fault, instead of leaving EV 100 stopped at EVSE 205 and then experiencing the fault. Computers 105, 110, and 115 can be programmed to display multiple alternative EVSE 205s in response to receiving an indication of a fault in the subset of steps performed. For example, if a fault occurs in payment processing or configuration internals 305, EVSE 205 can transmit a notification to computers 105, 110, and 115. Computers 105, 110, and 115 can output a message indicating the fault and display a list of charging stations 200 or EVSE 205s as described above, excluding the previously selected EVSE 205 that experienced the fault.

[0061] Computers 105, 110, 115 and / or EVSE 205 may transmit a fault report to a remote server in response to an indication of a fault in an execution step (a subset of the steps or steps after EV 100 at EVSE 205). The report may include data about the fault condition, such as the identity of EVSE 205, the identity of EV 100, the step in which the fault occurred, data generated by sensors of EVSE 205, data generated by sensor 125 of EV 100, error messages returned by EVSE 205, etc.

[0062] Figure 4 This is a flowchart illustrating an example process 400 for initiating charging of EV 100 at charging station 200. The memories of computers 105, 110, and 115 store executable instructions for performing the steps of process 400, and / or can be programmed using structures such as those mentioned above. As a general overview of process 400, while a user is browsing charging station 200, computers 105, 110, and 115 display multiple EVSEs 205 at multiple charging stations 200. In response to receiving a selection of EVSE 205, computers 105, 110, and 115 transmit messages to the selected EVSE 205 to execute a subset of the step sequence. In response to a failure, computers 105, 110, and 115 transmit a report and return to browsing an alternative EVSE 205. In response to the successful execution of the subset of steps, computers 105, 110, and 115 execute LLM to generate disposal operations in augmented reality. The sequence of steps is then executed to charge EV 100. In response to a fault, computers 105, 110, and 115 proceed to process 500, as described below. Otherwise, process 400 terminates.

[0063] Process 400 begins in box 405, where computers 105, 110, and 115 display multiple alternative EVSEs 205, as described above.

[0064] Next, in box 410, computers 105, 110, 115 display data about the charging station 200 selected from the list, including a VR simulation of the charging station 200, as described above.

[0065] Next, in decision box 415, computers 105, 110, and 115 determine whether EVSE 205 has been selected as described above. In response to receiving a selection for EVSE 205 when the user is positioned away from EVSE 205, process 400 proceeds to box 420. Otherwise, process 400 returns to box 405 so that the user can continue browsing charging station 200.

[0066] In box 420, computers 105, 110, and 115 transmit messages to the selected EVSE 205 instructing the selected EVSE 205 to perform a subset of the sequence of steps, as described above.

[0067] Next, in decision box 425, computers 105, 110, and 115 determine whether a fault occurred during the execution of a subset of steps, as described above. In response to receiving an indication of a fault in the executed subset, process 400 proceeds to box 430. In response to the subset of steps completing without a fault indication, process 400 proceeds to decision box 435.

[0068] In box 430, computers 105, 110, and 115 transmit a report to a remote server, as described above. After box 430, process 400 returns to box 405 to browse alternative EVSE 205.

[0069] In decision box 435, computers 105, 110, and 115 determine whether the user is at EVSE 205, as described above. In response to the user being at EVSE 205, process 400 proceeds to box 440. In response to the user still being away from EVSE 205, process 400 remains at decision box 435 to wait for the user to arrive at EVSE 205.

[0070] In box 440, computers 105, 110, and 115 execute LLM to generate processing operations for user processing of component 305 of EVSE 205. Computers 105, 110, and 115 instruct XR device 115 to overlay the processing operation output onto the real-time display of component 305 of EVSE 205 in augmented reality, as described above.

[0071] Next, in box 445, computers 105, 110, and 115 track the sequence of steps as power transfer occurs, as described above.

[0072] Next, in decision box 450, computers 105, 110, and 115 determine whether a fault has occurred at EVSE 205, as described above. In response to a fault, computers 105, 110, and 115 execute process 500, as follows regarding... Figure 5 As described. In response to the charging process completing without failure, process 400 ends.

[0073] Figure 5 This is a flowchart illustrating an example process 500 for resolving a fault during the charging of EV 100. The memories of computers 105, 110, and 115 store executable instructions for performing the steps of process 500, and / or can be programmed using structures such as those mentioned above. The user executing process 500 can be an EV operator experiencing the fault, a maintenance technician, or a call center technician resolving a fault occurring for a different user. As a general overview of process 500, computers 105, 110, and 115 track a sequence of steps, collect sensor data and fault data, and determine the user's role. In response to the user being at EVSE 205, computers 105, 110, and 115 output handling actions in augmented reality. In response to the user moving away from EVSE 205, computers 105, 110, and 115 output handling actions in virtual reality. Finally, computers 105, 110, and 115 transmit the collected data to a remote server.

[0074] Process 500 begins in block 505, where computers 105, 110, 115 track the sequence of steps performed during the charging of EV 100 by EVSE 205, as described above.

[0075] Next, in box 510, computers 105, 110, and 115 receive data indicating a fault from the sensors and EVSE 205. This data includes data generated at the time of the fault from sensors 125 of EV 100 and EVSE 205, the status of the charging process for EV 100, error messages generated by EVSE 205, etc.

[0076] Next, in box 515, computers 105, 110, 115 determine the user's role relative to EVSE 205, as described above.

[0077] Next, in decision box 520, computers 105, 110, and 115 determine the user's position as described above. In response to the user's position being at EVSE 205, process 500 proceeds to box 525. In response to the user's position moving away from EVSE 205, process 500 proceeds to box 530.

[0078] In block 525, computers 105, 110, and 115 execute an LLM (Local Management Model) to generate handling actions for the user to handle component 305 of the EVSE 205. The LLM output is a handling action to resolve the fault. Computers 105, 110, and 115 instruct XR device 115 to overlay the handling action output onto a real-time display of component 305 of the EVSE 205 in augmented reality, as described above. After block 525, process 500 proceeds to block 535.

[0079] In block 530, computers 105, 110, and 115 execute an LLM (Limited Ledger Model) to generate handling operations for user intervention of component 305 of EVSE 205. The LLM output is a handling operation for resolving the fault. Computers 105, 110, and 115 instruct XR device 115 to overlay the handling operation output onto the image of component 305 of EVSE 205 in virtual reality, as described above. After block 530, process 500 proceeds to block 535.

[0080] In box 535, computers 105, 110, and 115 transmit reports to the remote server as described above. After box 535, process 500 ends.

[0081] Typically, the described computing system and / or device may employ any of a number of computer operating systems, including, but not limited to, versions and / or variants of: Ford Sync® applications, AppLink / Smart Device Link middleware, Microsoft Automotive® operating system, Microsoft Windows® operating system, Unix operating system (e.g., Solaris® operating system released by Oracle Corporation of Redwood Coast, California), AIX UNIX operating system released by International Business Machines Corporation of Armonk, New York, Linux operating system, Mac OSX and iOS operating systems released by Apple Inc. of Cupertino, California, BlackBerry OS released by BlackBerry Ltd. of Waterloo, Canada, and Android operating system developed by Google and the Open Handset Alliance, or QNX® CAR infotainment platform provided by QNX Software Systems, Inc. Examples of computing devices include, but are not limited to, in-vehicle computers, computer workstations, servers, desktop computers, laptops, notebook computers or handheld computers, or any other computing system and / or device.

[0082] Computing devices typically include computer-executable instructions, which can be executed by one or more computing devices such as those listed above. Computer-executable instructions can be compiled or interpreted from computer programs created using a variety of programming languages ​​and / or technologies, which, individually or in combination, include, but are not limited to, Java™, C, C++, Matlab, Simulink, Stateflow, Visual Basic, JavaScript, Python, Perl, HTML, etc. Some of these applications can be compiled and executed on virtual machines such as the Java Virtual Machine, the Dalvik Virtual Machine, etc. Generally, a processor (e.g., a microprocessor) receives instructions (e.g., from memory, computer-readable media, etc.) and executes those instructions to perform one or more processes, including one or more processes described herein. Such instructions and other data can be stored and transferred using a variety of computer-readable media. Files in a computing device are typically collections of data stored on computer-readable media such as storage media, random access memory, etc.

[0083] Computer-readable media (also known as processor-readable media) include any non-transitory (e.g., tangible) medium that contributes to providing data (e.g., instructions) that can be read by a computer (e.g., by the computer's processor). Such media can take many forms, including but not limited to non-volatile and volatile media. Instructions can be transmitted via one or more transmission media, including optical fibers, wires, wireless communications, and internals that constitute a system bus coupled to the computer's processor. Common forms of computer-readable media include, for example, RAM, PROM, EPROM, flash EEPROM, any other memory chip or magnetic tape, or any other medium from which a computer can read.

[0084] The databases, data repositories, or other data stores described herein can include various mechanisms for storing, accessing / retrieving various types of data, including hierarchical databases, file sets in file systems, application databases in proprietary formats, relational database management systems (RDBMS), non-relational databases (NoSQL), graph databases (GDB), and so on. Each such data store is typically contained within a computing device employing a computer operating system such as those mentioned above, and is accessed via a network in any one or more of various ways. File systems can be accessed from the computer operating system and can include files stored in various formats. In addition to languages ​​used to create, store, edit, and execute the stored programs (such as PL / SQL as described above), RDBMS typically employs Structured Query Language (SQL).

[0085] In some examples, system elements may be implemented as computer-readable instructions (e.g., software) stored on one or more computing devices (e.g., servers, personal computers, etc.) and on computer-readable media (e.g., disks, storage, etc.) associated therewith. Computer program products may include such instructions stored on computer-readable media for performing the functions described herein.

[0086] In the accompanying drawings, the same reference numerals indicate the same elements. Furthermore, some or all of these elements may be changed. Regarding the media, processes, systems, methods, inspirations, etc., described herein, it should be understood that although the steps of such processes, etc., are described as occurring in a certain ordered order, such processes can be practiced by performing the steps in a different order than that described herein. It should also be understood that some steps may be performed simultaneously, other steps may be added, or some steps described herein may be omitted. The operations, systems, and methods described herein should always be implemented and / or performed in accordance with the applicable owner / user manual and / or safety guidelines.

[0087] This disclosure has been described in an illustrative manner, and it should be understood that the terminology used is intended to be descriptive in nature and not restrictive. The use of terms such as “in response to,” “after determining,” etc., indicates a causal relationship, not just a temporal one. In view of the above teachings, many modifications and variations of this disclosure are possible, and this disclosure may be practiced in ways other than those specifically described.

[0088] According to the present invention, a computer is provided having a processor and a memory, the memory storing instructions executable by the processor to: track a sequence of steps for charging an electric vehicle (EV) by an electric vehicle service equipment (EVSE); execute a large language model (LLM) to generate disposal operations for a user to dispose of components of the EVSE, the disposal operations supporting the steps of charging the EV; and output the disposal operations to an extended reality (XR) device, the XR device displaying the disposal operations overlaid on the EVSE, the XR device highlighting the components of the EVSE in the disposal operations.

[0089] According to an embodiment, the sequence of steps includes a fault that interrupts the charging of the EV by the EVSE, and at least one of the handling operations is in response to the fault.

[0090] According to an embodiment, the XR device is an augmented reality (AR) device, and the AR device displays the handling operation overlaid on the real-time display of the component of the EVSE.

[0091] According to an embodiment, the instructions further include instructions for performing the following: determining the user's role relative to the EVSE, and the LLM generating the disposal operation specific to the user's role.

[0092] According to an embodiment, the instructions also include instructions for performing the following: selecting the role from a preset group that includes at least EV operators and maintenance technicians.

[0093] According to an embodiment, at least some of the handling operations specific to the maintenance technician relate to components in the EVSE that are inaccessible to the EV operator.

[0094] According to an embodiment, the preset group includes call center technicians.

[0095] According to an embodiment, the instructions further include instructions to perform the following: in response to the user's location at the EVSE, output the disposal operation in augmented reality.

[0096] According to an embodiment, the instructions further include instructions for performing the following: in response to the user's position moving away from the EVSE, outputting the disposal operation in virtual reality.

[0097] According to an embodiment, the instructions further include instructions to perform the following: in response to receiving a selection of the EVSE when the user is located away from the EVSE, transmit a message to the EVSE instructing the EVSE to perform a subset of the sequence of steps.

[0098] According to an embodiment, the subset includes processing the user's payment data.

[0099] According to an embodiment, the instructions further include instructions for performing the following: in response to receiving an indication of a failure in the subset, displaying a plurality of alternative EVSEs.

[0100] According to an embodiment, the LLM is trained on training data including the technical documentation of the EVSE.

[0101] According to an embodiment, the instructions further include instructions for performing the following: in response to receiving a selection of the EVSE from a plurality of EVSEs, tracking the sequence of steps by which the EVSE charges the EV.

[0102] According to the present invention, a method includes: tracking a sequence of steps for charging an electric vehicle (EV) by an electric vehicle service equipment (EVSE); performing a large language model (LLM) to generate disposal operations for a user to dispose of components of the EVSE, the disposal operations supporting the steps of charging the EV; and outputting the disposal operations to an extended reality (XR) device, the XR device displaying the disposal operations overlaid on the EVSE, the XR device highlighting the components of the EVSE in the disposal operations.

[0103] In one aspect of the invention, the sequence of steps includes a fault that interrupts the charging of the EV by the EVSE, and at least one of the handling operations is in response to the fault.

[0104] In one aspect of the invention, the method includes determining the user's role relative to the EVSE, wherein the LLM generates the disposal action specific to the user's role.

[0105] In one aspect of the invention, the method includes selecting the role from a preset group that includes at least EV operators and maintenance technicians.

[0106] In one aspect of the invention, the method includes outputting the disposal action in augmented reality in response to the user's location at the EVSE.

[0107] In one aspect of the invention, the method includes outputting the disposal operation in virtual reality in response to the user's position moving away from the EVSE.

Claims

1. A method comprising: Track the sequence of steps involved in charging an electric vehicle (EV) using an electric vehicle service equipment (EVSE); Execute a large language model (LLM) to generate disposal operations for the user to dispose of the EVSE, the disposal operations supporting the steps of charging the EV; as well as The action is output to an extended reality (XR) device, which displays the action overlaid on the EVSE and highlights the component of the EVSE in the action.

2. The method of claim 1, wherein the sequence of steps includes a fault that interrupts charging of the EV by the EVSE, and at least one of the disposal operations is in response to the fault.

3. The method of claim 1, wherein the XR device is an augmented reality (AR) device, and the AR device displays the handling operation overlaid on a real-time display of the component of the EVSE.

4. The method of claim 1, further comprising determining the user's role relative to the EVSE, wherein the LLM generates the disposal action specific to the user's role.

5. The method of claim 4, further comprising selecting the role from a preset group comprising at least EV operators and maintenance technicians.

6. The method of claim 5, wherein at least some of the handling operations specific to the maintenance technician relate to the components in the EVSE that are inaccessible to the EV operator.

7. The method of claim 5, wherein the preset group includes call center technicians.

8. The method of claim 1, further comprising, in response to the user's location at the EVSE, outputting the disposal action in augmented reality.

9. The method of claim 1, further comprising outputting the disposal operation in virtual reality in response to the user's position moving away from the EVSE.

10. The method of claim 1, further comprising, in response to receiving a selection of the EVSE when the user is located away from the EVSE, transmitting a message to the EVSE instructing the EVSE to perform a subset of the sequence of steps.

11. The method of claim 10, wherein the subset includes processing the user's payment data.

12. The method of claim 10, further comprising displaying a plurality of alternative EVSEs in response to receiving an indication of a failure in the subset.

13. The method of claim 1, wherein the LLM is trained on training data including the technical documentation of the EVSE.

14. The method of claim 1, further comprising, in response to receiving a selection of the EVSE from a plurality of EVSEs, tracking the sequence of steps of charging the EV by the EVSE.

15. A computer comprising: processor; And a memory that stores instructions that can be executed by the processor to perform the method as described in any one of claims 1–14.

Images