CHARGING ELECTRIC VEHICLES WITH EXTENDED REALITY

Abstract

A computer contains a processor and memory. The memory stores instructions that the processor can execute to follow a sequence of steps for charging an electric vehicle (EV) using an EV service equipment (EVSE). It executes a large-language model (LLM) to generate handling operations for a user to manipulate components of the EVSE and outputs these operations to an extended-reality (XR) device. The handling operations support the steps of charging the EV. The XR device displays the handling operations overlaid on the EVSE. The XR device highlights the EVSE components that are part of the handling operations.

Description

AREA OF TECHNOLOGY

[0001] This disclosure provides techniques to enable a user to correctly use an electric vehicle service equipment (EVSE) for charging an electric vehicle (EV) or to troubleshoot problems with it. GENERAL STATE OF THE ART

[0002] Electric vehicles require batteries to be charged, which can be done at public charging stations. This is driving the development of new technologies for the architecture and operation of charging station systems. For example, charging station systems can include different types of charging stations, such as Stage 1 chargers, Stage 2 chargers, DC fast chargers (DCFC), and so on. Stage 1 charging can use a 120-volt alternating current (AC) outlet, which can charge a vehicle from 0% to 80% state of charge (i.e., a relative value indicating how much energy remains in the battery compared to its maximum capacity) in 40–50 hours (e.g., based on a 40–50 kWh battery). Stage 2 chargers can use a 240-volt AC outlet, which can charge a vehicle from 0% to 80% state of charge in 4–10 hours.DCFCs can use a direct current (DC) outlet, which can charge a vehicle from 0% charge to 80% charge in 1 hour or less. SUMMARY

[0003] A computer is programmed to track a sequence of steps for charging the EV through the EVSE, execute a large-language model (LLM) to generate user handling instructions for manipulating EVSE components, and output these instructions to an extended-reality (XR) device. The handling instructions support the EV charging steps. The XR device displays the handling instructions overlaid on the EVSE. The XR device highlights the EVSE components that are part of the handling process. The XR device can provide output in virtual reality or augmented reality. The XR device can be used by the EV operator to charge the EV, by a repair technician to repair or troubleshoot EVSE issues, or by a call center technician to instruct the EV operator on charging the EV.The use of a Low Load Management (LLM) device can provide handling instructions for a wide variety of situations. The output from the XR device provides a technological means to guide the user to the correct components for handling procedures, especially when the user may be unfamiliar with the EVSE (such as an operator charging their EV at a public charging station for the first time) or when the EVSE components may be complex (such as a technician troubleshooting internal EVSE components). These techniques can increase the user's likelihood of success, even without additional user knowledge.

[0004] A computer contains a processor and memory. The memory stores instructions that the processor can execute to follow a sequence of steps for charging an electric vehicle (EV) using an EV service equipment (EVSE). It executes a large-language model (LLM) to generate handling operations for a user to manipulate components of the EVSE and outputs these operations to an extended-reality (XR) device. The handling operations support the steps of charging the EV. The XR device displays the handling operations overlaid on the EVSE. The XR device highlights the EVSE components that are part of the handling operations.

[0005] In one example, the sequence of steps may include an error that interrupts the charging of the EV by the EVSE, and at least one of the handling operations may react to the error.

[0006] In one example, the XR device can be an augmented reality device (AR device), and the AR device can display the handling operations superimposed on a real-time display of the EVSE components.

[0007] In one example, the instructions may also include instructions for determining a user's role in relation to the EVSE, and the LLM may generate the handling operations specific to that user role. In another example, the instructions may also include instructions for selecting the role from a predefined group that includes at least one EV Operator and one Repair Technician. In yet another example, at least some of the handling operations specific to the Repair Technician may relate to EVSE components that are inaccessible to the EV Operator.

[0008] In yet another example, the pre-configured group could include a call center technician.

[0009] In one example, the instructions may also include instructions for issuing handling instructions in augmented reality in response to the user's location being at the EVSE.

[0010] In one example, the instructions may also include instructions for issuing handling instructions in augmented reality in response to the user's location being at the EVSE.

[0011] In one example, the instructions might also include instructions to transmit a message to the EVSE in response to receiving a selection from the EVSE while the user is away from the EVSE, instructing the EVSE to perform a subset of the sequence of steps. In another example, the subset might include processing payment data for the user.

[0012] In another further example, the instructions may also include instructions to display a variety of alternative EVSEs in response to receiving an indication of an error in executing the subset.

[0013] In one example, the LLM can be trained using training data that includes the technical documentation of the EVSE.

[0014] In one example, the instructions may further include instructions to track the sequence of steps of charging the EV by the EVSE in response to receiving a selection of EVSEs from a variety of EVSEs.

[0015] This involves tracking a sequence of steps for charging an electric vehicle (EV) using an EV service equipment (EVSE), executing a large-language model (LLM) to generate handling operations for a user to manipulate EVSE components, and outputting these handling operations to an extended-reality (XR) device. The handling operations support the EV charging steps. The XR device displays the handling operations overlaid on the EVSE. The XR device highlights the EVSE components that are part of the handling operations.

[0016] In one example, the sequence of steps may include an error that interrupts the charging of the EV by the EVSE, and at least one of the handling operations may react to the error.

[0017] In one example, the procedure may further include determining a user's role in relation to the EVSE, and the LLM may generate the handling operations specific to that user's role. In another example, the procedure may further include selecting the role from a predefined group that includes at least one EV operator and one repair technician.

[0018] In one example, the procedure can also include, in response to the fact that a user's location is at the EVSE, the output of the handling operations in augmented reality.

[0019] In one example, the procedure can further involve, in response to the fact that a user's location is at the EVSE, outputting the handling operations in virtual reality. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 is a block diagram of an example system that includes an electric vehicle. Fig. Figure 2 is a diagram of an example charging station where the electric vehicle charges. Fig. Figure 3 is a perspective view in an exemplary extended reality output of handling procedures for charging the electric vehicle at the charging point. Fig. Figure 4 is a flowchart of an example process for charging the electric vehicle at the charging point. Fig. Figure 5 is a flowchart of an example process for addressing an error during loading. DETAILED DESCRIPTION

[0020] Referring to the figures, in which identical reference numerals denote identical parts in the different views, a computer 105, 110, 115 includes a processor and a memory, and the memory stores instructions that can be executed by the processor to follow a sequence of steps for charging an electric vehicle (EV) 100 by an electric vehicle service equipment (EVSE) 205, executing a large-language model (LLM) to generate handling operations for a user to handle components 305 of the EVSE 205, and outputting the handling operations to an extended-reality (XR) device 115. The computer 105, 110, or 115 can be the vehicle computer 105 of the EV 100, a remote computer 110 that is different from the EV 100, and / or the XR device 115. The handling procedures support the steps for charging the EV 100.The XR device 115 displays the handling operations superimposed on the EVSE 205. The XR device 115 highlights the components 305 of the EVSE 205 that are involved in the handling operations.

[0021] With reference to Fig. The EV 100 can be any passenger car or any commercial vehicle, such as a car, truck, SUV, crossover, van, minivan, taxi, bus, etc. The EV 100 can be a plug-in hybrid electric vehicle (PHEV), a battery electric vehicle (BEV), or an extended-range BEV. The EV 100 includes the vehicle computer 105, a communication network 120, sensors 125, a user interface 130, a transceiver 135, and a battery 140.

[0022] Battery 140 provides power (i.e., electricity) to components of the EV 100. Specifically, Battery 140 provides the power to drive the EV 100, meaning it enables the EV 100 to move itself. Battery 140 can be of any suitable type for vehicle electrification, such as lithium-ion batteries, nickel-metal hydride batteries, lead-acid batteries, or ultracapacitors, like those used in PHEVs or BEVs, etc. Battery 140 can consist of multiple batteries or cells wired together.

[0023] The vehicle computer 105 is a microprocessor-based computing device, such as a generic computing device, which includes: a processor and memory, an electronic control unit or the like, a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a combination of the above, etc. Typically, a hardware description language, such as VHDL (very high speed integrated circuit hardware description language), is used in the electronic design to describe digital and mixed-signal systems, such as FPGAs and ASICs.For example, an ASIC is manufactured based on VHDL programming, which is provided prior to manufacturing, whereas logical components within an FPGA can be configured based on VHDL programming (e.g., stored in memory electrically connected to the FPGA circuit). The vehicle computer 105 can thus include a processor, memory, etc. The memory of the vehicle computer 105 can include media for storing instructions executable by the processor, as well as for electronically storing data and / or databases, and / or the vehicle computer 105 can include structures, such as those mentioned above, through which programming is provided. The vehicle computer 105 can consist of several interconnected computers.

[0024] The vehicle computer 105 can transmit and receive data via the communication network 120. The communication network 120 can be a Controller Area Network bus (CAN bus), Ethernet, WiFi, a Local Interconnect Network (LIN), an On-Board Diagnostics (OBD-II) port, and / or any other wired or wireless communication network. The vehicle computer 105 can communicate with the sensors 125, the user interface 130, the transceiver 135, and other components via a communication network 120.

[0025] The sensors 125 can provide data on the operation of the EV 100, such as wheel speed, wheel alignment, and transmission data (e.g., temperature, power consumption, battery 140 charge level, etc.). The sensors 125 can detect the location and / or orientation of the EV 100. For example, the sensors 125 can include: sensors of a global positioning system (GPS); accelerometers, such as piezoelectric or microelectromechanical systems (MEMS); gyroscopes, such as reversing, ring laser, or fiber optic gyroscopes; inertial measurement units (IMEs); and magnetometers. The sensors 125 can detect the external environment, including objects and / or features of the EV 100's surroundings, such as other vehicles, lane markings, traffic lights and / or signs, road users, etc.For example, the sensors can include 125 radar sensors, ultrasonic sensors, laser scanner rangefinders, light detection and ranging devices (LIDAR devices) and image processing sensors, such as cameras.

[0026] The user interface 130 presents information to and receives information from the operator of the EV 100. The user interface 130 can be located on a dashboard in a passenger compartment of the EV 100 and / or anywhere else that is readily visible to the operator. The user interface 130 can include dials, digital displays, screens, speakers, and so on for providing information to the operator, similar to familiar elements of a human-machine interface (HMI). The user interface 130 can include buttons, knobs, keypads, a microphone, and so on for receiving information from the operator.

[0027] The EV 135 transceiver can be configured to transmit signals wirelessly using any suitable wireless communication protocol, such as cellular, Bluetooth®, Bluetooth® Low Energy (BLE), ultra-wideband (UWB), Wi-Fi, IEEE 802.11a / b / g / p, cellular-V2X (CV2X), dedicated short-range communications (DSRC), other RF (radio frequency) communications, etc. The EV 135 transceiver can also be configured to communicate with a remote server, meaning a server that is separate and located at a distance from the vehicle. The remote server can be located outside the EV 100. For example, the remote server could be another vehicle (e.g., V2V communication), an infrastructure component (e.g., V2I communication), an emergency responder, a mobile device assigned to the EV 100 operator, etc.The remote server could be the remote computer 110, the XR device 115, or the EVSE 205. The transceiver 135 could be a device or include a separate transmitter and receiver.

[0028] The vehicle computer 105, the remote computer 110, and / or the XR device 115 can be communicatively coupled via a network 145. The network 145 represents one or more mechanisms by which the vehicle computer 105, the remote computer 110, and / or the XR device 115 can communicate with each other or with remote servers. Accordingly, the network 145 can be one or more of various wired or wireless communication mechanisms, including any desired combination of wired (e.g., cable and fiber optic) and / or wireless (e.g., cellular, wireless, satellite, microwave, and radio frequency) communication mechanisms and any desired network topology (or topologies if multiple communication mechanisms are used). Examples of communication networks include wireless communication networks (e.g., using Bluetooth®, IEEE 802).11 etc.), local area networks (LAN) and / or wide area networks (WAN), including the Internet, which provide data communication services.

[0029] The remote computer 110 is a microprocessor-based computing device, such as a generic computing device, that includes a processor and memory. The memory of the remote computer 110 may include media for storing instructions executable by the processor, as well as for electronically storing data and / or databases, and / or the remote computer 110 may include structures, such as those mentioned above, through which programming is provided. The remote computer 110 may consist of multiple computers connected together. For example, the remote computer 110 may be a mobile device. The mobile device is a portable computing device, such as a mobile phone, e.g., a smartphone, or a tablet. The mobile device belongs to and is carried by a person, who may be the operator of the EV 100 or a technician who can maintain the EVSE 205.In another example, the remote computer 110 may be assigned to a service center or call center that oversees the EVSE 205.

[0030] The XR device 115 is a device equipped to provide output in Extended Reality. Extended Reality combines the physical world with a digital world that interacts with the physical world within the Extended Reality environment. Extended Reality encompasses Augmented Reality, Virtual Reality, and Mixed Reality. Augmented Reality combines the real world with computer-generated content that is superimposed on the real world from the user's perspective. Virtual Reality provides the user with a computer-generated simulation of the physical world, possibly combined with additional digital features. Mixed Reality is a combination of Augmented Reality and Virtual Reality.

[0031] For example, the XR device 115 can be an augmented reality (AR) device, such as AR glasses, a portable computing device, such as a mobile phone (e.g., a smartphone), or a tablet equipped for AR output, or a display in the EV 100, such as a head-up display (HUD) (e.g., as part of the user interface 130). The AR glasses have transparent lenses that allow the user to see their surroundings. The lenses of the AR glasses also display content that the user sees at the same time as they view the world through the lenses. The AR glasses can display content at a specific size and location on the lenses so that the content appears to the user as if they are at a specific location in the world. For example, the AR glasses can use pose tracking to determine the position and orientation of the user's head relative to the environment (e.g.,based on data from inertial measurement units (IMUs) or the like. The portable computing device may include a camera and a screen. The portable computing device can display image data from the camera on the screen, along with content in a size and location within the image data, so that the content appears to the user to be at a specific location in the world. The portable computing device can use pose tracking to determine its position and orientation. The HUD can display information or graphics on a transparent windscreen of the EV 100, allowing the user to see the EV 100's surroundings. The HUD can display content in a size and location on the lenses so that the content appears to the user as if they were at a specific location in the world.For example, the HUD can use data from sensors 125 to track the pose of the EV 100 in relation to its surroundings.

[0032] The XR Device 115 comprises a microprocessor-based computing device, such as a generic computing device, which includes a processor and memory. The memory of the XR Device 115 may include media for storing instructions executable by the processor, as well as for electronically storing data and / or databases, and / or the XR Device 115 may include structures, such as those mentioned above, through which programming is provided. The XR Device 115 comprises multiple interconnected computers.

[0033] With reference to Fig. 2. A charging point 200 can include one or multiple EVSEs 205. An area enclosed by the charging point 200 can be defined (e.g., as a radius) within which the EVSEs 205 and other elements of the charging point 200 are located. The charging point 200 can include one or more parking spaces 210 corresponding to the respective EVSEs 205. That is, the parking spaces 210 are provided as areas where an EV 100 can be parked while receiving an electrical charge for its battery 140 from a respective EVSE 205. The charging point 200 can also include a region where EVs 100 can park and / or drive (e.g., while waiting for access to one of the EVSEs 205) to park, to visit another facility of the charging point 200, to enter and exit the charging point 200, etc.The areas of loading point 200 and any other regions thereof can be defined according to location coordinates, a geofence, or any other suitable ways of defining location boundaries.

[0034] Each EVSE 205 is equipped to charge the battery 140 of an EV 100. The EVSE 205 can typically charge one EV 100 at a time. The EVSE 205 draws power from a power grid to transfer it to the battery 140 of the EV 100. The EVSE 205 is typically stationary (i.e., fixed to a specific physical location and unable to move from there). The one or more EVSEs 205 at the charging point 200 can use any suitable mechanism to charge the battery 140 of the EV 100 (e.g., a plug connection, inductive charging, etc.). A plug connection involves connecting a plug 305a of the EVSE 205 (in Fig. (3 shown) with a connection of the EV 100. Inductive charging is a form of wireless power transfer that relies on electromagnetic induction. The power transfer components 305 can be located under the parking space 210, which corresponds to the EVSE 205.

[0035] With reference to Fig. 3. The EVSE 205 may include a physical structure 310 to which the components 305 of the EVSE 205 are mounted or connected. For the purposes of this disclosure, a “component” of the EVSE 205 is defined as any physical part of the EVSE 205 that a user can handle or interact with. The components 305 may include external components that may be visible to the operator of an EV 100 using the EVSE 205, as well as internal components that may be concealed by one or more plates 305f of the EVSE 205. The external components may include one or more connectors 305a, one or more cables 305b leading to the respective connectors 305a, a payment device 305c such as a credit card reader, a keypad 305d or other input device for entering data, a screen 305e for displaying information, the panels 305f, etc.The internal components (not shown) may include wiring, circuit boards, transformers, computing devices and other electrical or computing components for transmitting power and processing data to facilitate the transmission of power.

[0036] To charge the EV 100 using the EVSE 205, the operator and the EVSE 205 jointly execute a sequence of steps to charge the EV 100. These steps may include making a payment from the EV 100 operator to the EVSE 205 operator, selecting a charging protocol, transferring power from the EVSE 205 to the EV 100, and completing the charging process. For example, if no errors occur, the steps actually performed may correspond to the intended sequence of steps. The intended sequence of steps includes the operator initiating the loading process by providing an input for the EVSE 205, the EVSE 205 issuing an instruction to provide a payment, the operator entering payment information, the EVSE 205 receiving payment information, and the EVSE 205 verifying the payment information.Issuing an instruction to select a charging protocol by the EVSE 205, inputting the selected charging protocol by the operator, configuring the internal components 305 to provide electricity via the selected protocol by the EVSE 205, connecting plug 305a to the EV 100 by the operator, transferring power from the EVSE 205 to the battery 140 by the EVSE 205, issuing a notification about the state of charge by the EVSE 205, issuing a notification that the battery 140 is fully charged by the EVSE 205, disconnecting plug 305a from the EV 100 by the operator and issuing a notification that the charging process is complete.through the EVSE 205. The intended steps may be preset by an operator of the EVSE 205 according to the functionality and programming of the EVSE 205. Some of the steps may depend on the completion of previous steps or may be performed concurrently with other steps. Some of the steps in the preceding example may be further subdivided.

[0037] The sequence of steps (as actually performed by the operator and the EVSE 205) may contain an error that interrupts the charging of the EV 100 by the EVSE 205. The term "error" is used in its computational sense as an incorrect step in a process that is responsible for unintended behavior of a program or device. For example, an error may occur because the EVSE 205 is unable to verify the payment information, the EVSE 205 cannot configure the internal components 305 for the selected protocol, connector 305a is not connected or is incorrectly connected to the EV 100, an incorrect connector 305a is connected to the EV 100, etc. The error can cause the actual sequence of steps to deviate from the intended sequence.

[0038] The computer 105, 110, 115 is programmed to track the sequence of steps for loading the EV 100 by the EVSE 205. The intended sequence of steps can be stored in the computer 105, 110, 115 as well as in a computing device of the EVSE 205. The computer 105, 110, 115 and / or the EVSE 205 can also store conditional intended steps corresponding to predefined errors; in other words, a sequence of steps is to be performed in response to the occurrence of a predefined error. For example, the computer 105, 110, 115 can track the sequence of steps by determining the status of the steps, such as which steps are completed, which steps are being executed or are pending, and which steps are not yet pending. A step can be pending if the step is ready to be executed (e.g.,A step is not pending if it has not been performed and is not ready to be performed (e.g., if a required previous step has not been completed and / or if a precondition has not been met). Computers 105, 110, and 115 can receive data from the EVSE 205 indicating the status of the steps, for example, whenever a status change of one of the steps occurs. Computers 105, 110, and 115 can repeatedly or continuously update the status of the steps when actions are performed with respect to the EVSE 205.

[0039] As described below, the computer 105, 110, 115 generates handling operations for the user to handle the components 305 of the EVSE 205. For the purposes of this disclosure, a "handling operation" is defined as a step performed by a user in which the user handles something. For example, the handling operations for an operator of an EV 100 might include placing or inserting a payment card or mobile device into or onto the payment device 305c, typing information into the keypad 305d, plugging one of the connectors 305a into the port of the EV 100, placing the connector 305a back onto the physical structure 310, and so on. As another example, the handling operations for a technician servicing the EVSE 205 might include removing or replacing a panel 305f, disconnecting or connecting wiring, adjusting the internal components 305, and so on.Accordingly, the handling procedures support the steps of charging the EV 100 (i.e., they initiate progress from one step to the next). The handling procedures may include actions in response to an error, such as using a different payment method after an attempted payment could not be verified, unplugging and replugging connector 305a into the EV 100's port after electrical connections to the EV 100's battery 140 were not detected, etc.

[0040] The computer 105, 110, 115 is programmed to execute a large-language model (LLM) to generate handling operations for the user to operate the components 305 of the EVSE 205. The term "large-language model" is used in the context of machine learning, referring to a computational model for natural language processing tasks. The LLM takes the sequence of steps (e.g., the current states of the steps or a last executed step) as input and provides the next handling operation as output. The handling operation, as output by the LLM, can take the form of a text instruction.

[0041] The LLM can be trained using training data that includes the EVSE 205 technical documentation. This documentation might include operating manuals, maintenance manuals, answers to frequently asked questions (FAQs), troubleshooting guides, and so on. The LLM can then be trained to provide handling procedures that align with the recommendations in the EVSE 205 technical documentation. For example, the LLM could be a modified version of an existing pen model. In other words, the LLM could be a baseline model already trained on a universal corpus of text, which is then further trained on the technical documentation. The LLM can use any suitable baseline model as its foundation, such as GPT, LLaMA, Claude, Gemini, Nemotron, and others.

[0042] Computer 105, 110, 115 may be programmed to determine a user's role in relation to the EVSE 205. For the purposes of this disclosure, a person's "role" in a given context is defined as a function or position (e.g., a job) held by that person. For example, roles a user may have in relation to the EVSE 205 may include EV Operator, Repair Technician, Call Center Technician, and so on. Computer 105, 110, 115 may select the role from a predefined group that includes at least EV Operator and Repair Technician, and possibly also Call Center Technician. Computer 105, 110, 115 may determine the user's role based on credentials provided by the user. For example, the role may be stored in a user profile or account.Computers 105, 110, and 115 can select the EV operator if there is no indication that the user has a different role (i.e., the EV operator is the default role).

[0043] The LLM can generate handling operations specific to the user's role. For example, the LLM can be trained to issue handling operations based on data from an operator's manual in response to the EV operator's role, and handling operations based on data from a maintenance manual in response to the repair technician's role. At least some of the repair technician-specific handling operations relate to the EVSE 205 components 305 that are inaccessible to the EV operator. For example, in response to the repair technician's role, the LLM can issue handling operations to remove a panel 305f and adjust wiring, circuit boards, transformers, computing devices, etc., within the physical structure 310 of the EVSE 205.In response to the EV operator role, the LLM may refrain from issuing handling operations for disk 305f or internal components 305 of the EVSE 205.

[0044] Computer 105, 110, 115 is programmed to output handling instructions to XR device 115. As a general overview, computer 105, 110, 115 determines whether the handling instructions should be output in virtual reality or augmented reality (e.g., based on the user's location). If the handling instructions are output in VR or AR, XR device 115 displays the handling instructions overlaid on EVSE 205.

[0045] The XR device 115 displays the handling operations superimposed on the EVSE 205. The XR device 115 can output the handling operation at a visible location corresponding to component 305 of the EVSE 205 mentioned in the handling operation. The output handling operation can include highlighting and / or an identifier of component 305 mentioned in the handling operation. For example, the XR device 115 can display a handling operation to grasp connector 305a by highlighting and labeling connector 305a, or the XR device 115 can display a handling operation to place a card on the payment reader by highlighting and labeling the payment reader, as shown in Fig. Figure 3 shows the handling procedures. These procedures can be useful for the user, for example, to locate the payment device 305c, to distinguish the appropriate connector 305a (e.g., level 1 versus level 2 versus DCFC), etc. In the output, the highlighting and labeling appear superimposed on the EVSE 205.

[0046] When the virtual reality output is activated, the XR device 115 displays the handling operations superimposed on the images from the EVSE 205. These images could be, for example, a recorded video or a computer-generated video.

[0047] When outputting in augmented reality, the XR device 115 displays the handling operations superimposed on a real-time display of the EVSE 205 components 305—in other words, a display of the EVSE 205 as it currently exists in the physical world. For example, the real-time display could be a video feed from a camera of the XR device 115 to a screen of the XR device 115 (in the case of a mobile device). Alternatively, the real-time display could be the EVSE 205 itself, as seen through the lenses of AR glasses.

[0048] Computer 105, 110, 115 can be programmed to determine the user's location. Specifically, computer 105, 110, 115 can determine whether the user's location is at or away from EVSE 205. The user can be at EVSE 205 if they are within range of EVSE 205 (e.g., if EV 100 is parked in parking space 210, which corresponds to EVSE 205). For example, computer 105, 110, 115 can determine the location based on data from a GPS sensor 125 of EV 100, or from a GPS sensor of the remote computer 110 (if it is a mobile device), or from the XR device 115. The user can be located at the EVSE 205 if the location of the GPS sensor is within a threshold distance of a known location of the EVSE 205, and otherwise is away from the EVSE 205.As another example, the computer 105, 110, 115 can determine whether the EV 100's transceiver 135, the remote computer 110 (if a mobile device), or the XR device 115 is within range of a transmitter of the EVSE 205. The user can be at the EVSE 205 if the transceiver 135, the remote computer 110, or the XR device 115 is within range of the EVSE 205, and otherwise be away from the EVSE 205.

[0049] Computers 105, 110, and 115 can be programmed to select virtual reality or augmented reality for the output of handling instructions based on the user's location. Computers 105, 110, and 115 can output handling instructions in augmented reality in response to the user's location at EVSE 205. Computers 105, 110, and 115 can output handling instructions in virtual reality in response to the user's location away from EVSE 205.

[0050] Before a user (e.g., an EV operator) drives the EV 100 to the EVSE 205, the computer 105, 110, 115 can be programmed to assist the user in selecting the EVSE 205 and performing at least part of the sequence of steps. As a general overview, the computer 105, 110, 115 can display data about the charging points 200 and the EVSEs 205, facilitate the selection of an EVSE 205 at a charging point 200, and remotely instruct the selected EVSE 205 to perform a subset of the sequence before the user arrives at the EVSE 205. The computer 105, 110, 115 can facilitate switching the selection to a different EVSE 205 in response to an error occurring during the execution of the subset of steps.

[0051] The computer 105, 110, 115 can display data about the charging points 200 and / or EVSEs 205. For example, the user can enter a request for a nearby charging point, and the computer 105, 110, 115 can display a list of charging points 200 and, in response to a selection of one of the charging points 200, display detailed data about the selected charging point 200. The list can be displayed as a list (e.g., in order of proximity), as a set of locations on a map, or both. The computer 105, 110, 115 can populate the list with charging points 200 within the driving range of the EV 100 (i.e., to which the EV 100 can travel using its current battery 140 charge level).The data for a charging point 200 can include, for example, the number of EVSEs 205, the occupancy of the EVSEs 205, the error rates occurring at the EVSEs 205, the travel time to the charging point 200, the layout of the charging point 200, ratings, or ratings of the charging point 200, etc. The XR device 115 can output a virtual reality image of the charging point 200 along with a simulation of the handling operations as described above.

[0052] The user can select a charging point 200 (e.g., by viewing the data on nearby charging points 200) to be used for charging the EV 100. The user can enter the selection using the user interface 130 or the remote computer 110. The user can also select an EVSE 205 at the charging point 200, or the computer 105, 110, or 115 can select an EVSE 205 at the selected charging point 200 based on availability. The computer 105, 110, or 115 can transmit a request to the charging point 200 to reserve the selected EVSE 205.

[0053] The computer 105, 110, 115 can be programmed to transmit a message to the EVSE 205 in response to receiving a selection from the EVSE 205 while the user is away from the EVSE 205. This message instructs the EVSE 205 to perform a subset of the sequence of steps. The subset of steps can include most or all of the steps prior to plugging the connector 305a into the EV 100 (or inductively transferring power). For example, the subset of steps can include processing payment data for the user and / or configuring the EVSE 205 to charge the EV 100.To further break down the steps, the subset of steps may include: initiating charging by the EV operator providing an input to the EVSE 205, issuing an instruction to provide a payment by the EVSE 205, entering payment information by the EV operator, receiving payment information by the EVSE 205, verifying the payment information by the EVSE 205, issuing an instruction to select a charging protocol by the EVSE 205, entering the charging protocol selection by the EV operator, and configuring the internal components 305 to provide electricity via the selected protocol by the EVSE 205.

[0054] Furthermore, the computer 105, 110, 115 is programmed, in response to receiving the selection of the EVSE 205 from a multitude of EVSEs 205, to track the sequence of steps for charging the EV 100 by the EVSE 205, as described above. This tracking includes the steps performed before and after the user is at the EVSE 205.

[0055] By performing the subset of steps before the EV 100 is at the EVSE 205, the user can switch to another EVSE 205 or another charging point 200 if an error occurs, instead of stopping the EV 100 at the EVSE 205 and then learning about the error. The computer 105, 110, 115 can be programmed to display a variety of alternative EVSEs 205 in response to receiving an error message while performing the subset of steps. For example, if an error occurs during payment processing or while configuring the internal components 305, the EVSE 205 can transmit a notification to the computer 105, 110, 115. The computer 105, 110, 115 can output a message indicating the error and display a list of charging points 200 or EVSEs 205 in the manner described above, except for the previously selected EVSE 205 that has experienced the error.

[0056] Computers 105, 110, 115, and / or EVSE 205 can transmit an error report to a remote server in response to a reported error while performing steps (either a subset of steps or steps after EV 100 is connected to EVSE 205). The report may include data about the circumstances of the error, such as the identity of EVSE 205, the identity of EV 100, the step at which the error occurred, data generated by sensors on EVSE 205, data generated by sensors on EV 100, an error message returned by EVSE 205, and so on.

[0057] Fig. Figure 4 is a flowchart illustrating an example process 400 for initiating the charging of the EV 100 at charging point 200. The memory of computer 105, 110, 115 stores executable instructions for carrying out the steps of process 400 and / or may contain programming implemented in structures such as those mentioned above. As a general overview of process 400, computer 105, 110, 115 displays a variety of EVSEs 205 at a variety of charging points 200 as the user browses the charging points 200. In response to receiving a selection of an EVSE 205, computer 105, 110, 115 transmits a message to the selected EVSE 205 to execute a subset of the sequence of steps. In response to an error, computer 105, 110, 115 transmits a report and returns to searching for alternative EVSEs 205.In response to the successful execution of the subset of steps, computer 105, 110, 115 executes the LLM to generate handling operations in augmented reality. The sequence of steps is executed to load the EV 100. In response to the occurrence of an error, computer 105, 110, 115 transitions to process 500 as described below. Otherwise, process 400 terminates.

[0058] Process 400 begins in a block 405 in which the computer 105, 110, 115 displays a variety of alternative EVSEs 205, as described above.

[0059] Next, the computer displays data 105, 110, 115 in block 410 about a charging point 200 selected from the list, including a VR simulation of charging point 200, as described above.

[0060] Next, in decision block 415, computer 105, 110, 115 determines whether an EVSE 205 has been selected, as described above. In response to receiving a selection of the EVSE 205 while the user is away from the EVSE 205, process 400 moves to block 420. Otherwise, process 400 returns to block 405 so the user can continue browsing the charging stations 200.

[0061] In block 420, computer 105, 110, 115 transmits a message to the selected EVSE 205, instructing the selected EVSE 205 to perform the subset of the sequence of steps as described above.

[0062] Next, in decision block 425, computer 105, 110, 115 determines whether an error occurred during the execution of the subset of steps, as described above. Upon receiving an error indication during the execution of the subset, process 400 proceeds to block 430. Upon receiving an error indication that the subset of steps completed without an error indication, process 400 proceeds to decision block 435.

[0063] In block 430, computer 105, 110, 115 transmits a report to a remote server, as described above. After block 430, process 400 returns to block 405 to search for alternative EVSEs 205.

[0064] In decision block 435, computers 105, 110, and 115 determine whether the user is at EVSE 205, as described above. If the user is at EVSE 205, process 400 moves to block 440. If the user is still away from EVSE 205, process 400 remains at decision block 435 to wait for the user to arrive.

[0065] In block 440, computer 105, 110, 115 executes the LLM to generate handling instructions for the user to handle the components 305 of the EVSE 205. Computer 105, 110, 115 instructs the XR device 115 to output the handling instructions in augmented reality, superimposed on a real-time display of the components 305 of the EVSE 205, as described above.

[0066] Next, the computer 105, 110, 115 tracks the sequence of steps in a block 445 while the power transfer takes place, as described above.

[0067] Next, in a decision block 450, the computer 105, 110, 115 determines whether a fault has occurred while the vehicle is connected to the EVSE 205, as described above. In response to the occurrence of a fault, the computer 105, 110, 115 executes process 500, which is described below with respect to Fig. As described in section 5. In response to the successful completion of the loading process without errors, the process terminates with error 400.

[0068] Fig.Figure 5 is a flowchart illustrating an example Process 500 for addressing a fault during the charging of the EV 100. The memory of computer 105, 110, 115 stores executable instructions for carrying out the steps of Process 500 and / or may contain programming implemented in structures such as those mentioned above. The user for whom Process 500 is executed may be the EV operator experiencing the fault, or a repair technician or call center technician addressing a fault experienced by another user. As a general overview of Process 500, computer 105, 110, 115 tracks the sequence of steps, collects sensor and fault data, and determines the user's role. In response to the user's location at EVSE 205, computer 105, 110, 115 may output the handling procedures in augmented reality.In response to the user's distance from the EVSE 205, computer 105, 110, or 115 can output the handling operations in virtual reality. Finally, computer 105, 110, or 115 transmits the collected data in a report to the remote server.

[0069] Process 500 begins in block 505, in which computer 105, 110, 115 tracks the sequence of steps performed by EVSE 205 when loading EV 100, as described above.

[0070] Next, the computer receives data from sensors and the EVSE 205 (block 510) indicating that an error has occurred. This data includes information from the EV 100's sensor (125) and the EVSE 205's sensor, generated concurrently with the error, the status of the EV 100's charging steps, error messages generated by the EVSE 205, and other relevant data.

[0071] Next, in block 515, the computer 105, 110, 115 determines the role of the user in relation to the EVSE 205, as described above.

[0072] Next, in decision block 520, computers 105, 110, and 115 determine the user's location, as described above. In response to the user being at EVSE 205, process 500 moves to block 525. In response to the user being away from EVSE 205, process 500 moves to block 530.

[0073] In block 525, computer 105, 110, 115 executes the LLM to generate handling instructions for the user to handle the components 305 of the EVSE 205. The LLM outputs handling instructions to address the fault. Computer 105, 110, 115 instructs the XR device 115 to output the handling instructions in augmented reality, superimposed on a real-time display of the components 305 of the EVSE 205, as described above. After block 525, process 500 transitions to block 535.

[0074] In block 530, computer 105, 110, 115 executes the LLM to generate handling instructions for the user to handle the components 305 of the EVSE 205. The LLM outputs handling instructions to address the fault. Computer 105, 110, 115 instructs the XR device 115 to output the handling instructions in virtual reality, superimposed on images of the components 305 of the EVSE 205, as described above. After block 530, process 500 transitions to block 535.

[0075] In block 535, computers 105, 110, and 115 transmit a report to a remote server, as described above. Process 500 terminates after block 535.

[0076] In general, the described computing systems and / or facilities can use any of a range of computer operating systems, including, but not limited to, versions and / or variants of the Ford Sync® application, the AppLink / Smart Device Link middleware, the Microsoft Automotive® operating system, the Microsoft Windows® operating system, the Unix operating system (e.g., the Solaris® operating system, distributed by Oracle Corporation in Redwood Shores, California), the AIX UNIX operating system, distributed by International Business Machines in Armonk, New York, the Linux operating system, the Mac OSX and iOS operating systems, distributed by Apple Inc. in Cupertino, California, the BlackBerry OS, distributed by Blackberry, Ltd. in Waterloo, Canada, and the Android operating system, developed by Google, Inc. and the Open Handset Alliance, or the QNX® CAR Platform for Infotainment, offered by QNX Software Systems.Examples of computing devices include, without limitation, an onboard vehicle computer, a computer workstation, a server, a desktop, notebook, laptop or handheld computer, or any other computing system and / or device.

[0077] Computing devices generally contain computer-executable instructions, which can be executed by one or more computing devices, such as those listed above. Computer-executable instructions can be compiled or interpreted by computer programs created using a variety of programming languages ​​and / or technologies, including, but not limited to, Java™, C, C++, Matlab, Simulink, Stateflow, Visual Basic, JavaScript, Perl, HTML, and others, either alone or in combination. Some of these applications can be compiled and executed on a virtual machine, such as the Java Virtual Machine, the Dalvik Virtual Machine, or similar. Generally, a processor (e.g., a microprocessor) receives instructions (e.g., from memory, a computer-readable medium, etc.).) and executes these instructions, thereby carrying out one or more processes, including one or more of the processes described herein. Such instructions and other data can be stored and transmitted using a variety of computer-readable media. A file in a computing device is generally a collection of data stored on a computer-readable medium, such as a storage medium, random-access memory, etc.

[0078] A computer-readable medium (also called a processor-readable medium) is any non-volatile (e.g., physical) medium involved in providing data (e.g., instructions) that can be read by a computer (e.g., by a computer's processor). Such a medium can take many forms, including both non-volatile and volatile media. Instructions can be transmitted through one or more transmission media, including optical fibers, wires, and wireless communication, as well as internal components that comprise a system bus connected to a computer's processor. Common forms of computer-readable media include, for example, RAM, PROM, EPROM, FLASH EEPROM, any other memory chip, memory cartridge, or any other medium from which a computer can read.

[0079] Databases, data repositories, or other data storage devices described herein may include various types of mechanisms for storing, accessing, and retrieving different types of data, including a hierarchical database, a set of files in a file system, an application database in a proprietary format, a relational database management system (RDBMS), a non-relational database (NoSQL), a graph database (GDB), and so on. Each such data storage device is generally contained within a computing device that uses a computer operating system, such as one of those listed above, and is accessed in one or more of a variety of ways over a network. A file system can be accessed by a computer operating system and may contain files stored in various formats.An RDBMS generally uses the Structured Query Language (SQL) in addition to a language for creating, storing, editing and executing stored procedures, such as the PL / SQL language mentioned above.

[0080] In some examples, system elements may be implemented as computer-readable instructions (e.g., software) on one or more computing devices (e.g., servers, personal computers, etc.) stored on computer-readable media associated with them (e.g., disks, memory, etc.). A computer program product may include such instructions stored on computer-readable media for performing the functions described herein.

[0081] In the drawings, identical reference symbols denote the same elements. Furthermore, some or all of these elements could be modified. Regarding the media, processes, systems, procedures, heuristics, etc., described herein, it is understood that although the steps of such processes, etc., have been described as following a specific, ordered sequence, such processes could be implemented in practice, with the described steps being carried out in a sequence that differs from the one described herein. It is also understood that certain steps could be performed simultaneously, that other steps could be added, or that certain steps described herein could be omitted.The processes, systems and procedures described in this document should always be implemented and / or carried out in accordance with applicable owner / user instructions and / or safety guidelines.

[0082] The revelation has been described in an illustrative manner, and it is understood that the terminology used is intended to be descriptive and not restrictive. The use of "in response to" and "in determining" indicates a causal relationship, not merely a temporal one. In light of the foregoing teachings, many modifications and variations of the present revelation are possible, and the revelation can be implemented in ways other than those specifically described.

[0083] A computer includes a processor and memory, and the memory stores instructions that can be executed by the processor to follow a sequence of steps for charging an electric vehicle (EV) by an electric vehicle service equipment (EVSE), to execute a large-language model (LLM) to generate handling operations for a user to handle components of the EVSE, the handling operations supporting the steps of charging the EV; and to output the handling operations to an extended-reality (XR) device, the XR device displaying the handling operations superimposed on the EVSE, the XR device highlighting the components of the EVSE that are in the handling operations.

[0084] According to one embodiment, the sequence of steps includes a fault that interrupts the charging of the EV by the EVSE, and at least one of the handling operations reacts to the fault.

[0085] According to one embodiment, the XR device is an augmented reality device (AR device) and the AR device displays the handling operations superimposed on a real-time display of the EVSE components.

[0086] According to one embodiment, the instructions further include instructions for determining a user's role in relation to the EVSE and the LLM generates the handling operations that are specific to the user's role.

[0087] According to one embodiment, the instructions also include instructions for selecting the role from a preset group that includes at least one EV operator and one repair technician.

[0088] According to one embodiment, at least some of the handling procedures specific to the repair technician relate to the EVSE components that are inaccessible to the EV operator.

[0089] According to one embodiment, the preset group includes a call center technician.

[0090] In one embodiment, the instructions also include instructions for displaying the handling operations in augmented reality in response to the user's location at the EVSE.

[0091] In one embodiment, the instructions also include instructions for issuing handling operations in virtual reality in response to the fact that the user's location is remote from the EVSE.

[0092] According to one embodiment, the instructions further include instructions to transmit a message to the EVSE in response to receiving a selection from the EVSE while the user is away from the EVSE, instructing the EVSE to perform a subset of the sequence of steps.

[0093] According to one embodiment, the subset includes processing payment data for the user.

[0094] According to one embodiment, the instructions further include instructions for displaying a plurality of alternative EVSEs in response to receiving an indication of an error in executing the subset.

[0095] According to one embodiment, the LLM is trained using training data that includes the EVSE's technical documentation.

[0096] According to one embodiment, instructions further include instructions to track the sequence of steps of charging the EV by the EVSE in response to receiving a selection of the EVSE from a plurality of EVSEs.

[0097] According to the present invention, a method comprises: tracking a sequence of steps for charging an electric vehicle (EV) by an electric vehicle service equipment (EVSE); executing a large-language model (LLM) to generate handling operations for a user to handle components of the EVSE, wherein the handling operations support the steps of charging the EV; and outputting the handling operations to an extended-reality (XR) device, wherein the XR device displays the handling operations superimposed on the EVSE, and the XR device highlights the components of the EVSE that are involved in the handling operations.

[0098] In one aspect of the invention, the sequence of steps includes a fault that interrupts the charging of the EV by the EVSE, and at least one of the handling operations reacts to the fault.

[0099] In one aspect of the invention, the method involves determining a user role in relation to the EVSE, wherein the LLM generates the handling operations that are specific to the user role.

[0100] In one aspect of the invention, the method involves selecting the role from a preset group that includes at least one EV operator and one repair technician.

[0101] In one aspect of the invention, the method involves, in response to the fact that a user's location is at the EVSE, outputting the handling operations in augmented reality.

[0102] In one aspect of the invention, the method involves, in response to the fact that the user's location is remote from the EVSE, outputting the handling operations in virtual reality.

Claims

Method comprising: tracking a sequence of steps for charging an electric vehicle (EV) by an electric vehicle service equipment (EVSE); executing a large-language model (LLM) to generate handling operations for a user to handle components of the EVSE, wherein the handling operations support the steps of charging the EV; and outputting the handling operations to an extended reality (XR) device, wherein the XR device displays the handling operations superimposed on the EVSE, the XR device highlighting the components of the EVSE that are in the handling operations. Method according to claim 1, wherein the sequence of steps includes a fault that interrupts the charging of the EV by the EVSE and at least one of the handling operations reacts to the fault. Method according to claim 1, wherein the XR device is an augmented reality device (AR device) and the AR device displays the handling operations superimposed on a real-time display of the EVSE components. The method of claim 1, further comprising determining a user role with respect to the EVSE, wherein the LLM generates the handling operations specific to the user role. The method of claim 4, further comprising selecting the role from a preset group comprising at least one EV operator and one repair technician. Method according to claim 5, wherein at least some of the handling operations specific to the repair technician relate to the components of the EVSE that are inaccessible to the EV operator. Method according to claim 5, wherein the preset group includes a call center technician. The method of claim 1, further comprising, in response to the fact that a user's location is at the EVSE, outputting the handling operations in augmented reality. The method of claim 1, further comprising, in response to the fact that a user's location is remote from the EVSE, outputting the handling operations in virtual reality. The method of claim 1, further comprising, in response to receiving a selection from the EVSE while the user is away from the EVSE, transmitting a message to the EVSE instructing the EVSE to perform a subset of the sequence of steps. Method according to claim 10, wherein the subset includes processing payment data for the user. The method of claim 10, further comprising, in response to receiving a notification of an error in the execution of the subset, displaying a plurality of alternative EVSEs. Method according to claim 1, wherein the LLM is trained using training data that includes the technical documentation of the EVSE. The method of claim 1, further comprising, in response to receiving a selection of the EVSE from a plurality of EVSEs, tracking the sequence of steps of charging the EV by the EVSE. Computer comprising a processor and a memory, wherein instructions are stored on the memory which can be executed by the processor to carry out the method according to any one of claims 1-14.

Images