Vehicle assistant device, platform, and system including same

Abstract

An AR display device linked with a vehicle and an operation method thereof are disclosed. The AR display device linked with a vehicle according to the present invention uses an AR graphic interface and thus can display a current traveling state of the vehicle while traveling and display, on a navigation screen, a guide related to a predicted next traveling situation. Here, the AR display device can guide, in advance, a predicted traveling situation by separating a part of an AR graphic interface at a predetermined distance or a predetermined time before a predicted point, and the separated AR guide can display a guide trajectory along which the vehicle should travel while moving according to the traveling of the vehicle. Therefore, a more intuitive and realistic AR guide is provided to the vehicle.

Description

Vehicle assistant device, platform, and system including the same

[0001] The present invention relates to a vehicle assistant device, a platform, and a system including the same, and more specifically, to a vehicle assistant device, a platform, and a system including the same that is installed in a vehicle and responds to situations occurring during driving.

[0002] To ensure the safety and convenience of users of vehicles, vehicles are equipped with various sensors and devices, and their functions are becoming more diverse. These vehicle functions can be divided into convenience functions designed to enhance driver comfort and safety functions designed to ensure the safety of drivers and / or pedestrians.

[0003] Specifically, as vehicles are equipped with various intelligent functions such as autonomous driving technology, advanced driver assistance systems (ADAS), and connected car services, they are evolving in a direction that collects and processes various events and external data occurring during driving in real time to provide appropriate responses to drivers and passengers.

[0004] In particular, due to advancements in Human-Machine Interfaces (HMI), such as voice recognition, gesture input, and the provision of user-customized content, vehicles are evolving beyond mere means of transportation into personalized service platforms. However, since these functions are often implemented independently at the individual system level, there are limitations in integrally managing various events and input conditions and performing optimized control actions according to the situation.

[0005] Conventional vehicle control systems typically performed a single predefined action upon the occurrence of a specific event or relied simply on sensor-based rule control. In such cases, there is a problem in that it is difficult for the system to provide accurate control actions in a timely manner when driving environments change complexly or user requirements diversify.

[0006] In addition, despite the increasing number of personalized services linked with external servers, mobile devices, and wearable devices, there is a lack of integrated architectures that effectively combine these heterogeneous resources and data to provide user-customized control and response.

[0007] Accordingly, the present invention aims to solve the aforementioned problems and other problems.

[0008] Existing vehicle control systems often perform only single-rule-based actions in response to sensor events or user inputs, making it difficult to provide integrated control that reflects complex driving contexts. Furthermore, the disconnected management of data between internal vehicle functions, external servers, and external devices has limited the provision of customized services and multi-domain cooperative control.

[0009] According to some embodiments of the present invention, in-vehicle driving-related events and route information are divided into time units and stored as context slots, and by analyzing the slots to generate a context model including event conditions, state transition rules, and corresponding control actions, a more sophisticated and context-appropriate response to multiple input conditions can be provided. Through this, a systematic structure that goes beyond a simple event-response structure and contextually understands and controls the entire driving situation becomes possible.

[0010] According to another embodiment of the present invention, the context model can be linked with multiple domain APIs within the vehicle to simultaneously output control signals and can provide user-customized services through cooperation with external servers and external devices. In addition, by including a security layer that performs source authentication, privacy protection, and audit log recording for context slot data, a personalized mobility control environment with ensured reliability and safety can be implemented.

[0011] To this end, the vehicle assistant device according to the present invention can continuously store driving-related events and route information, and then perform an appropriate response operation in response to a user's command input.

[0012] Specifically, as an assistant device installed in a vehicle according to an embodiment of the present invention and responding to situations occurring during driving, the driving assistant device may include: a context slot module that stores driving-related events and path information in a plurality of context slots divided into time units; a context model generation module that analyzes the driving-related events and path information stored in the plurality of context slots and generates a context model including event conditions, state transition rules, and corresponding control actions; a voice receiving module that receives user voice; and a response processing module that analyzes the received voice and input conditions occurring during driving, generates a response based on the context model and events stored in the context slots, or outputs a control signal corresponding to a driving control action.

[0013] In addition, in the embodiment, the context slot module can dynamically adjust the length of the slot by considering the frequency of occurrence of driving-related events and path variability.

[0014] In addition, in the embodiment, the event condition and the state transition rule can be represented in a machine form to generate a context model.

[0015] In addition, in the embodiment, the context model generation module can learn the repeating pattern of driving-related events using a machine learning algorithm and generate a plurality of context models by reflecting this.

[0016] In addition, in the embodiment, the response processing module can time-synchronize voice, gesture, touch, and sensor signals to fuse and analyze them into multidimensional input conditions.

[0017] In addition, in the embodiment, the response processing module can determine the response according to a predefined priority rule when multidimensional input conditions conflict.

[0018] In addition, in the embodiment, the response processing module can calculate weather, traffic, and road data received from the outside as uncertainty values ​​and adjust the response intensity and control range differently according to the values.

[0019] In addition, in an embodiment, the response processing module can predict the next driving-related event based on the generated context model and output a control signal to perform a corresponding control action when the reliability of the prediction result is greater than or equal to a threshold.

[0020] In addition, a vehicle assistant platform according to an embodiment of the present invention is an assistant platform executed in a vehicle, wherein the platform comprises: a slot manager that manages context slot data; a model generator that analyzes the context slot data to generate a context model including event conditions, state transition rules, and corresponding control actions; an input interface that receives a command from a user; and a response processor that interprets the command and input conditions occurring during driving, generates a response based on the context model and events stored in the context slot, or outputs a control signal corresponding to a driving control action through a vehicle function API.

[0021] In addition, in the embodiment, the slot manager is characterized by managing the context slot data in a rolling manner, sequentially storing the context slot data and deleting old data after a certain period of time.

[0022] In addition, in an embodiment, the model generator is characterized by performing federated learning between a server and a vehicle terminal to update model parameters without external transmission of individual vehicle data.

[0023] In addition, in the embodiment, the response handler is characterized by performing control operations independently using a locally stored context model when a network connection is unavailable.

[0024] In addition, in the embodiment, the response processor is characterized by being configured to perform a control operation differentiated for each user by analyzing the user's past selection pattern in response to the input of the same command.

[0025] In addition, in an embodiment, the platform is characterized by accessing APIs of the vehicle body, chassis, infotainment, ADAS, and V2X domains to simultaneously control functions of multiple domains based on the context model.

[0026] Furthermore, a mobility control system according to an embodiment of the present invention is a context-based mobility control system comprising a vehicle, a server, and an external device, wherein the vehicle stores driving-related events and path information in a plurality of context slots and generates a response corresponding to an input condition or outputs a control signal corresponding to a driving control operation based on the context slots and a context model, and the server collects and analyzes the context slot data to learn a context model including event conditions, state transition rules, and corresponding control operations, and provides control rules according to the context model to the vehicle. In addition, the external device communicates with the vehicle or the server to provide a user-customized service based on the response or control signal.

[0027] In addition, in an embodiment, the server can aggregate and analyze data collected from multiple vehicles to form a common context model and provide it to each vehicle.

[0028] In addition, in the embodiment, the server can analyze the user's repeated response pattern and automatically update the control policy for the same input condition.

[0029] In addition, in the embodiment, the external device is at least one of a mobile terminal, a wearable device, or a smart home device, and can provide a user-customized service based on a control signal output from the vehicle.

[0030] In addition, in an embodiment, the server can maintain service continuity by communicating directly with the external device when the vehicle's network is disconnected.

[0031] Additionally, in the embodiment, the system may further include a security layer for performing source authentication of context slot data, anonymization for user privacy protection, and slot-unit audit log generation.

[0032] The effects of the assistant device, platform, and system according to the present invention are described as follows.

[0033] Specifically, the present invention enables intelligent control that reflects complex driving contexts by storing driving-related events and route information in context slots on a time-based basis and generating a context model including event conditions, state transition rules, and control actions based thereon. Therefore, unlike conventional single-event-based control, it can provide more precise and context-appropriate responses by integrally considering the driving environment and user input.

[0034] Furthermore, according to the present invention, the context model can be linked with APIs of multiple domains, such as the vehicle body, chassis, infotainment, ADAS, and V2X, to simultaneously output control signals. Accordingly, various functions of the vehicle can be organically combined and controlled in response to changes in driving conditions, and customized services can also be provided by linking with external servers and external devices.

[0035] In addition, the present invention may include a security layer that performs source authentication of context slot data, anonymization processing of personal information, and generation of slot-unit audit logs. Through this, the reliability of collected data can be guaranteed, user privacy protected, and the traceability of system operation history can be secured.

[0036] FIG. 1 is a drawing illustrating an example of a vehicle related to an embodiment of the present invention.

[0037] FIG. 2 is a drawing of a vehicle related to an embodiment of the present invention viewed from various angles.

[0038] FIGS. 3 and FIGS. 4 are drawings illustrating the interior of a vehicle related to an embodiment of the present invention.

[0039] FIGS. 5 and 6 are drawings referenced to describe various objects related to the driving of a vehicle related to an embodiment of the present invention.

[0040] FIG. 7 is a block diagram referenced to describe a vehicle display device related to an embodiment of the present invention together with the components of a vehicle.

[0041] FIG. 8 is an exemplary block diagram for explaining the operation of a vehicle assistant device or platform according to an embodiment of the present invention.

[0042] FIG. 9 is an exemplary block diagram for explaining the detailed configuration of a vehicle assistant device according to an embodiment of the present invention.

[0043] FIG. 10 is a diagram illustrating the storage of driving-related events and route information in a plurality of context slots divided into time units according to an embodiment of the present invention.

[0044] FIG. 11 is a diagram illustrating various events related to the creation of a context model according to an embodiment of the present invention.

[0045] FIG. 12 is a flowchart illustrating the operation method of a vehicle assistant device according to an embodiment of the present invention.

[0046] FIG. 13 is an exemplary block diagram for explaining the architecture of a context-based mobility control system according to an embodiment of the present invention.

[0047] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components regardless of drawing symbols will be assigned the same reference number, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not inherently possess distinct meanings or roles. Furthermore, in describing embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the present invention.

[0048] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0049] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0050] A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0051] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0052] The concept of a vehicle as described in this specification may include automobiles and motorcycles. Hereinafter, vehicles will be described primarily in the context of automobiles.

[0053] The vehicle described in this specification may be a concept that includes all of the following: an internal combustion engine vehicle equipped with an engine as a power source, a hybrid vehicle equipped with an engine and an electric motor as a power source, and an electric vehicle equipped with an electric motor as a power source.

[0054] In the following description, the left side of the vehicle refers to the left side of the vehicle's direction of travel, and the right side of the vehicle refers to the right side of the vehicle's direction of travel.

[0055] FIGS. 1 and 2 are drawings showing the exterior of a vehicle related to an embodiment of the present invention, and FIGS. 3 and 4 are drawings showing the interior of a vehicle related to an embodiment of the present invention.

[0056] FIGS. 5 and 6 are drawings illustrating various objects related to the driving of a vehicle related to an embodiment of the present invention.

[0057] FIG. 7 is a block diagram referenced to describe a vehicle related to an embodiment of the present invention. FIG. 7 is a block diagram referenced to describe a vehicle according to an embodiment of the present invention.

[0058] Referring to FIGS. 1 to 7, the vehicle (100) may include a wheel that rotates by a power source and a steering input device (510) for controlling the direction of travel of the vehicle (100).

[0059] The vehicle (100) may be an autonomous vehicle. The vehicle (100) may be switched between an autonomous driving mode and a manual mode based on user input. For example, the vehicle (100) may be switched from a manual mode to an autonomous driving mode or from an autonomous driving mode to a manual mode based on user input received through a user interface device (hereinafter referred to as a 'user terminal') (200).

[0060] The vehicle (100) may switch to an autonomous driving mode or a manual mode based on driving situation information. The driving situation information may be generated based on object information provided by the object detection device (300). For example, the vehicle (100) may switch from a manual mode to an autonomous driving mode or from an autonomous driving mode to a manual mode based on the driving situation information generated by the object detection device (300). For example, the vehicle (100) may switch from a manual mode to an autonomous driving mode or from an autonomous driving mode to a manual mode based on the driving situation information received through the communication device (400).

[0061] The vehicle (100) can be switched from manual mode to autonomous driving mode or from autonomous driving mode to manual mode based on information, data, and signals provided by an external device.

[0062] When the vehicle (100) is operated in an autonomous driving mode, the autonomous vehicle (100) may be operated based on the driving system (700). For example, the autonomous vehicle (100) may be operated based on information, data, or signals generated from the driving system (710), the vehicle exit system (740), and the parking system (750).

[0063] When the vehicle (100) is operated in manual mode, the autonomous vehicle (100) can receive user input for driving through the driving control device (500). Based on the user input received through the driving control device (500), the vehicle (100) can be operated.

[0064] The overall length refers to the length from the front to the rear of the vehicle (100), the overall width refers to the width of the vehicle (100), and the overall height refers to the length from the bottom of the wheels to the roof. In the following description, the overall length direction (L) may refer to the direction that serves as the reference for measuring the overall length of the vehicle (100), the overall width direction (W) may refer to the direction that serves as the reference for measuring the overall width of the vehicle (100), and the overall height direction (H) may refer to the direction that serves as the reference for measuring the overall height of the vehicle (100).

[0065] As illustrated in FIG. 7, the vehicle (100) may include a user interface device (hereinafter referred to as a 'user terminal') (200), an object detection device (300), a communication device (400), a driving operation device (500), a vehicle driving device (600), a driving system (700), a navigation system (770), a sensing unit (120), a vehicle interface unit (130), a memory (140), a control unit (170), and a power supply unit (190).

[0066] According to an embodiment, the vehicle (100) may include additional components other than those described herein, or may not include some of the described components.

[0067] The user interface device (200) is a device for communication between the vehicle (100) and the user. The user interface device (200) can receive user input and provide information generated by the vehicle (100) to the user. The vehicle (100) can implement a UI (User Interfaces) or UX (User Experience) through the user interface device (which may be referred to as a 'user terminal') (200).

[0068] The user interface device (200) may include an input unit (210), an internal camera (220), a bio-sensing unit (230), an output unit (250), and a processor (270). According to an embodiment, the user interface device (200) may include additional components other than the components described, or may not include some of the components described.

[0069] The input unit (210) is for receiving information from a user, and the data collected from the input unit (120) can be analyzed by the processor (270) and processed into a control command of the user.

[0070] The input section (210) may be positioned inside the vehicle. For example, the input section (210) may be positioned in an area of ​​the steering wheel, an area of ​​the instrument panel, an area of ​​the seat, an area of ​​each pillar, an area of ​​the door, an area of ​​the center console, an area of ​​the head lining, an area of ​​the sun visor, an area of ​​the windshield, or an area of ​​the window.

[0071] The input unit (210) may include a voice input unit (211), a gesture input unit (212), a touch input unit (213), and a mechanical input unit (214).

[0072] The voice input unit (211) can convert the user's voice input into an electrical signal. The converted electrical signal can be provided to the processor (270) or the control unit (170). The voice input unit (211) may include one or more microphones.

[0073] The gesture input unit (212) can convert the user's gesture input into an electrical signal. The converted electrical signal can be provided to the processor (270) or the control unit (170).

[0074] The gesture input unit (212) may include at least one of an infrared sensor and an image sensor for detecting a user's gesture input. According to an embodiment, the gesture input unit (212) may detect a user's three-dimensional gesture input. To this end, the gesture input unit (212) may include a light output unit that outputs a plurality of infrared lights or a plurality of image sensors.

[0075] The gesture input unit (212) can detect the user's three-dimensional gesture input through the Time of Flight (TOF) method, the structured light method, or the disparity method.

[0076] The touch input unit (213) can convert the user's touch input into an electrical signal. The converted electrical signal can be provided to the processor (270) or the control unit (170).

[0077] The touch input unit (213) may include a touch sensor for detecting a user's touch input. According to an embodiment, the touch input unit (213) may be formed integrally with the display unit (251) to implement a touch screen. Such a touch screen may provide both an input interface and an output interface between the vehicle (100) and the user.

[0078] The mechanical input unit (214) may include at least one of a button, a dome switch, a jog wheel, and a jog switch. An electrical signal generated by the mechanical input unit (214) may be provided to a processor (270) or a control unit (170). The mechanical input unit (214) may be placed on a steering wheel, center fascia, center console, cockpit module, door, etc.

[0079] The internal camera (220) can acquire an image of the vehicle interior. The processor (270) can detect the state of the user based on the image of the vehicle interior. The processor (270) can acquire the user's gaze information from the image of the vehicle interior. The processor (270) can detect the user's gesture from the image of the vehicle interior.

[0080] The biometric detection unit (230) can acquire the user's biometric information. The biometric detection unit (230) includes a sensor capable of acquiring the user's biometric information, and can acquire the user's fingerprint information, heart rate information, etc. by using the sensor. The biometric information can be used for user authentication.

[0081] The output unit (250) is intended to generate an output related to sight, hearing, or touch. The output unit (250) may include at least one of a display unit (251), an acoustic output unit (252), and a haptic output unit (253).

[0082] The display unit (251) can display graphic objects corresponding to various information. The display unit (251) may include at least one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, a 3D display, and an e-ink display.

[0083] The display unit (251) can be formed as a layered structure with the touch input unit (213) or as an integral unit to implement a touch screen.

[0084] The display unit (251) can be implemented as a HUD (Head Up Display). When the display unit (251) is implemented as a HUD, the display unit (251) may be equipped with a projection module to output information through an image projected onto a windshield or window.

[0085] The display unit (251) may include a transparent display. The transparent display may be attached to a windshield or a window. The transparent display may display a predetermined screen while having a predetermined transparency. To have transparency, the transparent display may include at least one of a transparent TFEL (Thin Film Electroluminescent), a transparent OLED (Organic Light-Emitting Diode), a transparent LCD (Liquid Crystal Display), a transparent transparent display, and a transparent LED (Light Emitting Diode) display. The transparency of the transparent display may be adjusted.

[0086] Meanwhile, the user interface device (200) may include a plurality of display units (251a to 251g).

[0087] The display unit (251) may be placed in a part of the steering wheel, a part of the instrument panel (521a, 251b, 251e), a part of the seat (251d), a part of each pillar (251f), a part of the door (251g), a part of the center console, a part of the head lining, a part of the sun visor, or implemented in a part of the windshield (251c) or a part of the window (251h).

[0088] The sound output unit (252) converts an electrical signal provided from the processor (270) or the control unit (170) into an audio signal and outputs it. To this end, the sound output unit (252) may include one or more speakers.

[0089] The haptic output unit (253) generates a tactile output. For example, the haptic output unit (253) can operate by vibrating the steering wheel, seat belt, and seat (110FL, 110FR, 110RL, 110RR) so that the user can perceive the output.

[0090] A processor (hereinafter referred to as a 'control unit') (270) can control the overall operation of each unit of the user interface device (200). According to an embodiment, the user interface device (200) may include a plurality of processors (270) or may not include a processor (270).

[0091] If the user interface device (200) does not include a processor (270), the user interface device (200) may be operated under the control of a processor or control unit (170) of another device in the vehicle (100).

[0092] Meanwhile, the user interface device (200) may be named as a vehicle display device. The user interface device (200) may be operated under the control of the control unit (170).

[0093] The object detection device (300) is a device for detecting an object located outside the vehicle (100). The object may be various objects related to the operation of the vehicle (100). Referring to FIGS. 5 and 6, the object (O) may include a lane (OB10), another vehicle (OB11), a pedestrian (OB12), a motorcycle (OB13), a traffic signal (OB14, OB15), light, a road, a structure, a speed bump, a topographical feature, an animal, etc.

[0094] A lane (OB10) may be a driving lane, a lane adjacent to a driving lane, or a lane on which an oncoming vehicle travels. A lane (OB10) may be a concept that includes the left and right lines forming the lane.

[0095] Other vehicles (OB11) may be vehicles driving in the vicinity of the vehicle (100). Other vehicles may be vehicles located within a predetermined distance from the vehicle (100). For example, other vehicles (OB11) may be vehicles that are ahead of or behind the vehicle (100).

[0096] A pedestrian (OB12) may be a person located near a vehicle (100). A pedestrian (OB12) may be a person located within a predetermined distance from the vehicle (100). For example, a pedestrian (OB12) may be a person located on a sidewalk or a roadway.

[0097] A two-wheeled vehicle (OB12) may refer to a vehicle that moves using two wheels and is located around a vehicle (100). A two-wheeled vehicle (OB12) may be a vehicle having two wheels located within a predetermined distance from the vehicle (100). For example, a two-wheeled vehicle (OB13) may be a motorcycle or bicycle located on a sidewalk or roadway.

[0098] Traffic signals may include traffic lights (OB15), traffic signs (OB14), patterns or text drawn on the road surface.

[0099] The light may be light generated from a lamp equipped in another vehicle. The light may be light generated from a street light. The light may be sunlight.

[0100] Roads may include road surfaces, curves, slopes such as uphill and downhill, etc.

[0101] A structure may be an object located near a road and fixed to the ground. For example, a structure may include streetlights, street trees, buildings, utility poles, traffic lights, and bridges.

[0102] Topographical features may include mountains, hills, etc.

[0103] Meanwhile, objects can be classified into moving objects and stationary objects. For example, moving objects may include other vehicles and pedestrians. For example, stationary objects may include traffic signals, roads, and structures.

[0104] The object detection device (300) may include a camera (310), a radar (320), a lidar (330), an ultrasonic sensor (340), an infrared sensor (350), and a processor (370).

[0105] According to an embodiment, the object detection device (300) may include additional components other than the described components, or may not include some of the described components.

[0106] The camera (310) may be positioned at an appropriate location outside the vehicle to acquire images of the vehicle's exterior. The camera (310) may be a mono camera, a stereo camera (310a), an AVM (Around View Monitoring) camera (310b), or a 360-degree camera.

[0107] For example, the camera (310) may be positioned in the interior of the vehicle, close to the front windshield, to acquire an image of the front of the vehicle. Alternatively, the camera (310) may be positioned around the front bumper or radiator grille.

[0108] For example, the camera (310) may be positioned in the interior of the vehicle, close to the rear glass, to obtain an image of the rear of the vehicle. Alternatively, the camera (310) may be positioned around the rear bumper, trunk, or tailgate.

[0109] For example, the camera (310) may be positioned in close proximity to at least one of the side windows in the interior of the vehicle to obtain an image of the side of the vehicle. Alternatively, the camera (310) may be positioned around the side mirror, fender, or door.

[0110] The camera (310) can provide the acquired image to the processor (370).

[0111] The radar (320) may include an electromagnetic wave transmitter and a receiver. The radar (320) may be implemented as a pulse radar or a continuous wave radar based on the principle of radio wave emission. Among the continuous wave radar methods, the radar (320) may be implemented as a Frequency Modulated Continuous Wave (FMCW) or Frequency Shift Keying (FSK) method depending on the signal waveform.

[0112] The radar (320) can detect an object based on the Time of Flight (TOF) method or the phase-shift method using electromagnetic waves, and can detect the location of the detected object, the distance from the detected object, and the relative speed.

[0113] The radar (320) can be placed at an appropriate location outside the vehicle to detect objects located in front, behind, or to the side of the vehicle.

[0114] The lidar (330) may include a laser transmitter and a receiver. The lidar (330) may be implemented using a Time of Flight (TOF) method or a phase-shift method.

[0115] The lidar (330) can be implemented as a driven or non-driven type.

[0116] When implemented as a driven type, the lidar (330) is rotated by a motor and can detect objects around the vehicle (100).

[0117] When implemented as a non-driven type, the lidar (330) can detect an object located within a predetermined range relative to the vehicle (100) by optical steering. The vehicle (100) may include a plurality of non-driven lidars (330).

[0118] Lidar (330) can detect an object based on the Time of Flight (TOF) method or the phase-shift method using laser light medium, and can detect the position of the detected object, the distance from the detected object, and the relative velocity.

[0119] The lidar (330) can be placed at an appropriate location outside the vehicle to detect objects located in front, behind, or to the side of the vehicle.

[0120] The ultrasonic sensor (340) may include an ultrasonic transmitter and a receiver. The ultrasonic sensor (340) can detect an object based on ultrasound and detect the position of the detected object, the distance from the detected object, and the relative speed.

[0121] The ultrasonic sensor (340) can be placed at an appropriate location outside the vehicle to detect an object located in front, behind, or to the side of the vehicle.

[0122] The infrared sensor (350) may include an infrared transmitter and a receiver. The infrared sensor (340) can detect an object based on infrared light and detect the position of the detected object, the distance from the detected object, and the relative speed.

[0123] An infrared sensor (350) can be placed at an appropriate location outside the vehicle to detect an object located in front, behind, or to the side of the vehicle.

[0124] The processor (370) can control the overall operation of each unit of the object detection device (300).

[0125] The processor (370) can detect and track objects based on the acquired image. The processor (370) can perform operations such as calculating the distance to the object and calculating the relative speed to the object through an image processing algorithm.

[0126] The processor (370) can detect and track an object based on the reflected electromagnetic waves that are reflected back from the object after the transmitted electromagnetic waves are reflected. The processor (370) can perform operations such as calculating the distance to the object and calculating the relative speed to the object based on the electromagnetic waves.

[0127] The processor (370) can detect and track an object based on the reflected laser light that is reflected back from the object by the transmitted laser. The processor (370) can perform operations such as calculating the distance to the object and calculating the relative speed to the object based on the laser light.

[0128] The processor (370) can detect and track an object based on the reflected ultrasound that is reflected back from the object after the transmitted ultrasound is reflected. The processor (370) can perform operations such as calculating the distance to the object and calculating the relative speed to the object based on the ultrasound.

[0129] The processor (370) can detect and track an object based on the reflected infrared light that is reflected back from the object after the transmitted infrared light is reflected. The processor (370) can perform operations such as calculating the distance to the object and calculating the relative speed to the object based on the infrared light.

[0130] According to an embodiment, the object detection device (300) may include a plurality of processors (370) or may not include a processor (370). For example, each of the camera (310), radar (320), lidar (330), ultrasonic sensor (340), and infrared sensor (350) may individually include a processor.

[0131] If the object detection device (300) does not include a processor (370), the object detection device (300) may be operated under the control of the processor or control unit (170) of the device in the vehicle (100).

[0132] The object detection device (400) can be operated under the control of the control unit (170).

[0133] The communication device (400) is a device for performing communication with an external device. Here, the external device may be another vehicle, a mobile terminal, or a server.

[0134] A communication device (400) may include at least one of a transmitting antenna, a receiving antenna, an RF (Radio Frequency) circuit capable of implementing various communication protocols, and an RF element to perform communication.

[0135] The communication device (400) may include a short-range communication unit (410), a location information unit (420), a V2X communication unit (430), an optical communication unit (440), a broadcast transmission and reception unit (450), and a processor (470).

[0136] According to an embodiment, the communication device (400) may include additional components other than the described components, or may not include some of the described components.

[0137] The short-range communication unit (410) is a unit for short-range communication. The short-range communication unit (410) can support short-range communication using at least one of Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0138] The short-range communication unit (410) can form a short-range wireless communication network (Wireless Area Networks) to perform short-range communication between the vehicle (100) and at least one external device.

[0139] The location information unit (420) is a unit for obtaining location information of the vehicle (100). For example, the location information unit (420) may include a Global Positioning System (GPS) module or a Differential Global Positioning System (DGPS) module.

[0140] The V2X communication unit (430) is a unit for performing wireless communication with a server (V2I: Vehicle to Infra), other vehicles (V2V: Vehicle to Vehicle), or pedestrians (V2P: Vehicle to Pedestrian). The V2X communication unit (430) may include an RF circuit capable of implementing communication protocols with infrastructure (V2I), communication between vehicles (V2V), and communication with pedestrians (V2P).

[0141] The optical communication unit (440) is a unit for communicating with an external device via light. The optical communication unit (440) may include an optical transmitter that converts an electrical signal into an optical signal and transmits it externally, and an optical receiver that converts a received optical signal into an electrical signal.

[0142] According to an embodiment, the light emitter may be formed to be integrated with a lamp included in the vehicle (100).

[0143] The broadcast transmission / reception unit (450) is a unit for receiving a broadcast signal from an external broadcast management server or transmitting a broadcast signal to a broadcast management server via a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal.

[0144] The processor (470) can control the overall operation of each unit of the communication device (400).

[0145] According to an embodiment, the communication device (400) may include a plurality of processors (470) or may not include a processor (470).

[0146] If the communication device (400) does not include a processor (470), the communication device (400) may be operated under the control of a processor or control unit (170) of another device in the vehicle (100).

[0147] Meanwhile, the communication device (400) can implement a vehicle display device together with the user interface device (200). In this case, the vehicle display device may be named a telematics device or an AVN (Audio Video Navigation) device.

[0148] The communication device (400) can be operated under the control of the control unit (170).

[0149] The driving control device (500) is a device that receives user input for driving.

[0150] In manual mode, the vehicle (100) can be operated based on a signal provided by the driving control device (500).

[0151] The driving control device (500) may include a steering input device (510), an acceleration input device (530), and a brake input device (570).

[0152] The steering input device (510) can receive input regarding the direction of travel of the vehicle (100) from a user. The steering input device (510) is preferably formed in the shape of a wheel so that steering input is possible by rotation. According to an embodiment, the steering input device may be formed in the shape of a touch screen, a touch pad, or a button.

[0153] The acceleration input device (530) can receive input from a user for accelerating the vehicle (100). The brake input device (570) can receive input from a user for decelerating the vehicle (100). The acceleration input device (530) and the brake input device (570) are preferably formed in the form of pedals. According to an embodiment, the acceleration input device or the brake input device may be formed in the form of a touch screen, a touch pad, or a button.

[0154] The driving operation device (500) can be operated according to the control of the control unit (170).

[0155] The vehicle drive unit (600) is a device that electrically controls the operation of various devices within the vehicle (100).

[0156] The vehicle drive unit (600) may include a power train drive unit (610), a chassis drive unit (620), a door / window drive unit (630), a safety device drive unit (640), a lamp drive unit (650), and an air conditioning drive unit (660).

[0157] According to an embodiment, the vehicle drive unit (600) may include additional components other than the described components, or may not include some of the described components.

[0158] Meanwhile, the vehicle drive unit (600) may include a processor. Each unit of the vehicle drive unit (600) may individually include a processor.

[0159] The power train drive unit (610) can control the operation of the power train device.

[0160] The power train drive unit (610) may include a power source drive unit (611) and a transmission drive unit (612).

[0161] The power source drive unit (611) can control the power source of the vehicle (100).

[0162] For example, when a fossil fuel-based engine is the power source, the power source drive unit (610) can perform electronic control of the engine. By doing so, the output torque of the engine can be controlled. The power source drive unit (611) can adjust the engine output torque according to the control of the control unit (170).

[0163] For example, if the power source is an electric energy-based motor, the power source drive unit (610) can perform control over the motor. The power source drive unit (610) can adjust the rotational speed, torque, etc. of the motor according to the control of the control unit (170).

[0164] The transmission drive unit (612) can perform control over the transmission. The transmission drive unit (612) can adjust the state of the transmission. The transmission drive unit (612) can adjust the state of the transmission to forward (D), reverse (R), neutral (N), or parking (P).

[0165] Meanwhile, when the engine is the power source, the transmission drive unit (612) can adjust the gear engagement state in the forward (D) state.

[0166] The chassis drive unit (620) can control the operation of the chassis device. The chassis drive unit (620) may include a steering drive unit (621), a brake drive unit (622), and a suspension drive unit (623).

[0167] The steering drive unit (621) can perform electronic control of the steering apparatus within the vehicle (100). The steering drive unit (621) can change the direction of travel of the vehicle.

[0168] The brake drive unit (622) can perform electronic control of the brake apparatus within the vehicle (100). For example, it can reduce the speed of the vehicle (100) by controlling the operation of the brakes placed on the wheels.

[0169] Meanwhile, the brake drive unit (622) can control each of the plurality of brakes individually. The brake drive unit (622) can control the braking force applied to the plurality of wheels differently.

[0170] The suspension drive unit (623) can perform electronic control of the suspension apparatus within the vehicle (100). For example, the suspension drive unit (623) can control the suspension apparatus to reduce vibration of the vehicle (100) when there is a curvature in the road surface. Meanwhile, the suspension drive unit (623) can control each of the multiple suspensions individually.

[0171] The door / window drive unit (630) can perform electronic control of the door apparatus or window apparatus within the vehicle (100).

[0172] The door / window drive unit (630) may include a door drive unit (631) and a window drive unit (632).

[0173] The door drive unit (631) can perform control over the door device. The door drive unit (631) can control the opening and closing of a plurality of doors included in the vehicle (100). The door drive unit (631) can control the opening or closing of the trunk or tail gate. The door drive unit (631) can control the opening or closing of the sunroof.

[0174] The window drive unit (632) can perform electronic control of the window apparatus. It can control the opening or closing of a plurality of windows included in the vehicle (100).

[0175] The safety device drive unit (640) can perform electronic control of various safety apparatuses within the vehicle (100).

[0176] The safety device drive unit (640) may include an airbag drive unit (641), a seatbelt drive unit (642), and a pedestrian protection device drive unit (643).

[0177] The airbag drive unit (641) can perform electronic control of the airbag apparatus within the vehicle (100). For example, the airbag drive unit (641) can control the airbag to deploy when danger is detected.

[0178] The seatbelt drive unit (642) can perform electronic control of the seatbelt apparatus within the vehicle (100). For example, the seatbelt drive unit (642) can control the seatbelt to secure the occupant to the seat (110FL, 110FR, 110RL, 110RR) using the seatbelt when danger is detected.

[0179] The pedestrian protection device drive unit (643) can perform electronic control of the hood lift and pedestrian airbag. For example, the pedestrian protection device drive unit (643) can control the hood lift up and the pedestrian airbag to deploy when a collision with a pedestrian is detected.

[0180] The lamp driving unit (650) can perform electronic control of various lamp apparatuses within the vehicle (100).

[0181] The air conditioning drive unit (660) can perform electronic control of the air cinditioner in the vehicle (100). For example, the air conditioning drive unit (660) can control the air cinditioner to operate and supply cold air into the vehicle when the temperature inside the vehicle is high.

[0182] The vehicle drive unit (600) may include a processor. Each unit of the vehicle drive unit (600) may individually include a processor.

[0183] The vehicle drive unit (600) can be operated under the control of the control unit (170).

[0184] The driving system (700) is a system that controls various operations of the vehicle (100). The driving system (700) can be operated in an autonomous driving mode.

[0185] The operation system (700) may include a driving system (710), an exit system (740), and a parking system (750).

[0186] According to an embodiment, the operating system (700) may include additional components other than the components described, or may not include some of the components described.

[0187] Meanwhile, the operating system (700) may include a processor. Each unit of the operating system (700) may individually include a processor.

[0188] Meanwhile, according to an embodiment, if the operating system (700) is implemented in software, it may be a sub-concept of the control unit (170).

[0189] Meanwhile, according to an embodiment, the driving system (700) may be a concept including at least one of a user interface device (200), an object detection device (300), a communication device (400), a vehicle driving device (600), and a control unit (170).

[0190] The driving system (710) can drive the vehicle (100).

[0191] The driving system (710) can perform driving of the vehicle (100) by receiving navigation information from the navigation system (770) and providing a control signal to the vehicle driving device (600). The driving system (710) can perform driving of the vehicle (100) by receiving object information from the object detection device (300) and providing a control signal to the vehicle driving device (600). The driving system (710) can perform driving of the vehicle (100) by receiving a signal from an external device through the communication device (400) and providing a control signal to the vehicle driving device (600).

[0192] The vehicle dispatch system (740) can perform the dispatch of the vehicle (100).

[0193] The vehicle exit system (740) can receive navigation information from the navigation system (770) and provide a control signal to the vehicle driving device (600) to perform the exit of the vehicle (100). The vehicle exit system (740) can receive object information from the object detection device (300) and provide a control signal to the vehicle driving device (600) to perform the exit of the vehicle (100). The vehicle exit system (740) can receive a signal from an external device through the communication device (400) and provide a control signal to the vehicle driving device (600) to perform the exit of the vehicle (100).

[0194] The parking system (750) can perform parking of the vehicle (100).

[0195] The parking system (750) can perform parking of the vehicle (100) by receiving navigation information from the navigation system (770) and providing a control signal to the vehicle driving device (600). The parking system (750) can perform parking of the vehicle (100) by receiving object information from the object detection device (300) and providing a control signal to the vehicle driving device (600). The parking system (750) can perform parking of the vehicle (100) by receiving a signal from an external device through the communication device (400) and providing a control signal to the vehicle driving device (600).

[0196] The navigation system (770) can provide navigation information. The navigation information may include at least one of map information, set destination information, path information according to the set destination, information about various objects on the path, lane information, and current location information of the vehicle.

[0197] The navigation system (770) may include memory and a processor. The memory may store navigation information. The processor may control the operation of the navigation system (770).

[0198] According to an embodiment, the navigation system (770) can receive information from an external device through a communication device (400) and update previously stored information.

[0199] According to an embodiment, the navigation system (770) may be classified as a sub-component of the user interface device (200).

[0200] The sensing unit (120) can sense the state of the vehicle. The sensing unit (120) may include an attitude sensor (e.g., yaw sensor, roll sensor, pitch sensor), a collision sensor, a wheel sensor, a speed sensor, an inclination sensor, a weight sensing sensor, a heading sensor, a yaw sensor, a gyro sensor, a position module, a vehicle forward / reverse sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor based on steering wheel rotation, a vehicle interior temperature sensor, a vehicle interior humidity sensor, an ultrasonic sensor, an illuminance sensor, an accelerator pedal position sensor, a brake pedal position sensor, etc.

[0201] The sensing unit (120) can acquire sensing signals regarding vehicle attitude information, vehicle collision information, vehicle direction information, vehicle location information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward / reverse information, battery information, fuel information, tire information, vehicle lamp information, vehicle interior temperature information, vehicle interior humidity information, steering wheel rotation angle, vehicle exterior illumination, pressure applied to the accelerator pedal, pressure applied to the brake pedal, etc.

[0202] The sensing unit (120) may further include, in addition, an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor (AFS), an intake air temperature sensor (ATS), a water temperature sensor (WTS), a throttle position sensor (TPS), a TDC sensor, a crank angle sensor (CAS), etc.

[0203] The vehicle interface unit (130) can serve as a channel for various types of external devices connected to the vehicle (100). For example, the vehicle interface unit (130) may be equipped with a port that can be connected to a mobile terminal, and may be connected to a mobile terminal through said port. In this case, the vehicle interface unit (130) can exchange data with the mobile terminal.

[0204] Meanwhile, the vehicle interface unit (130) can serve as a channel for supplying electrical energy to a connected mobile terminal. When a mobile terminal is electrically connected to the vehicle interface unit (130), the vehicle interface unit (130) can provide electrical energy supplied from the power supply unit (190) to the mobile terminal under the control of the control unit (170).

[0205] The memory (140) is electrically connected to the control unit (170). The memory (140) can store basic data for the unit, control data for controlling the operation of the unit, and input / output data. The memory (140) can be a hardware storage device such as ROM, RAM, EPROM, flash drive, hard drive, etc. The memory (140) can store various data for the overall operation of the vehicle (100), such as programs for processing or control by the control unit (170).

[0206] According to an embodiment, the memory (140) may be formed integrally with the control unit (170) or implemented as a sub-component of the control unit (170).

[0207] The control unit (170) can control the overall operation of each unit within the vehicle (100). The control unit (170) may be named an ECU (Electronic Control Unit).

[0208] The power supply unit (190) can supply power necessary for the operation of each component according to the control of the control unit (170). In particular, the power supply unit (190) can receive power from a battery inside the vehicle, etc.

[0209] One or more processors and control units (170) included in the vehicle (100) may be implemented using at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

[0210] The “vehicle assistant device” disclosed in this specification refers to a device that generates and outputs appropriate responses or control signals in response to various events and user inputs occurring while driving within a vehicle. For example, it may be any one of a vehicle infotainment system, a driver assistance controller, or a separate dedicated ECU (Electronic Control Unit), and may include hardware and software components that interact with sensors, microphones, displays, communication modules, etc., within the vehicle.

[0211] In addition, the "Context Slot" disclosed in this specification refers to a logical data unit that stores events and route information occurring during a vehicle driving process by dividing them into time units or distance units. The Context Slot stores events (e.g., road congestion, weather changes, user voice commands, etc.) that occurred in a specific driving segment in a structured format along with tags, and multiple slots can be connected consecutively to represent the entire driving context (Mobility Context).

[0212] Furthermore, the "Context Model" disclosed in this specification refers to a representation of a control policy that includes event conditions, state transition rules, and corresponding control actions, by analyzing event and path data stored in context slots. The Context Model may be implemented as a Finite State Machine, a rule-based policy, or a machine learning-based model, and serves to define what response or control action the vehicle assistant device should perform when a specific input condition occurs.

[0213] Hereinafter, with reference to FIGS. 7 and FIGS. 9, a vehicle assistant device (800) according to an embodiment of the present invention and its operation will be described in more detail.

[0214] FIG. 7 is a diagram illustrating the configuration of a vehicle assistant device (800) according to an embodiment of the present invention. As shown in FIG. 7, the assistant device (800) functions as a core component for generating and executing control actions suitable for the situation by comprehensively processing various events and user inputs that occur while driving in a vehicle.

[0215] An assistant device (800) according to one embodiment of the present invention includes a structure that first divides driving events and route information into time units and stores them in a plurality of context slots. These context slots are not merely simple data storage areas, but structure situational contexts corresponding to each driving segment and are subsequently utilized for analysis and control decision-making.

[0216] In addition, the assistant device (800) generates a context model by analyzing the event and path information stored in the slot. This context model includes an event condition and a state transition rule, and defines what state the vehicle should transition to and what control action should be performed when a specific input condition occurs. Accordingly, the assistant device (800) according to the present invention can perform policy-based control based on complex context, rather than a simple event-response method.

[0217] Additionally, the assistant device (800) includes a voice receiving function to accept user input and can also be coupled with gesture, touch, and sensor-based inputs as needed. These various inputs can be fused and interpreted as multidimensional input conditions in the response processing stage and are used to clearly identify the user's intent.

[0218] The assistant device (800) combines and analyzes the received user input and various events occurring during driving, and generates a final response or control signal by referring to a context model.

[0219] Here, the response can be provided to the user as guidance messages, warning notifications, etc. Additionally, the control signal can be linked with multiple domain APIs, such as the vehicle body, chassis, infotainment, ADAS, and V2X, to lead to the control of the actual driving environment.

[0220] In particular, the assistant device (800) according to the present invention is designed to clearly separate application operation in User Space from control decision-making related to vehicle safety and security. Through this, the core control logic is protected from security threats or abnormal application execution, and stable and reliable vehicle control becomes possible.

[0221] Accordingly, the assistant device (800) as illustrated in FIG. 7 provides an intelligent mobility control environment that is differentiated from existing vehicle control systems through a series of processes including context slot-based data structuring, context model-based policy generation, multidimensional input fusion interpretation, priority-based collision resolution, and control signal output for multiple domains.

[0222] FIG. 9 is a block diagram illustrating the detailed configuration of a vehicle assistant device (800) according to one embodiment of the present invention.

[0223] The assistant device (800) is installed in the vehicle and can perform functions to respond to various situations that occur while driving. The assistant device (800) consists of a context slot module (810), a context model creation module (820), a voice reception module (830), and a response processing module (840). Each module is interconnected to analyze complex driving events and, based on this, provides vehicle control and user response.

[0224] The context slot module (810) divides driving-related events and route information into time units and stores them in multiple slots. The events may include vehicle sensor data (lane departure, distance to the vehicle ahead, detection of sudden acceleration, etc.), user input (voice, touch, gesture), and external environment information (weather, road congestion, danger alerts). The context slot module (810) does not stop at a simple recording function but dynamically adjusts the slot length considering the frequency of event occurrence and route variability to ensure optimal data representation.

[0225] The context model generation module (820) generates a context model by analyzing event and path data stored in slots. The context model is a set of policies that defines control actions that a vehicle must perform when a specific input condition occurs by structuring event conditions and state transition rules. This can be implemented as a fixed rule-based, Finite State Machine (FSM), or learning-based policy model.

[0226] In this case, the event condition serves as a trigger that initiates driving control and is based on various sources, such as vehicle sensors, user input, the external environment, and network data. For example, “reducing the distance to the vehicle ahead below a threshold” can be an event condition that triggers deceleration control. Additionally, for example, “a user uttering a voice command such as ‘find a parking lot’” can also be an event condition that triggers a new state transition.

[0227] Furthermore, state transition rules refer to a set of rules that transition from the current state to another state or control action when an event condition is satisfied. For example, it is defined as: State A (driving straight) + Event Condition B (Detection of sudden braking by the vehicle ahead) → State C (Perform emergency deceleration). These rules can be applied not only to safe driving but also to user convenience services (e.g., destination guidance, in-vehicle environment control).

[0228] A context model is not merely a structure for storing data, but rather a logic that enables a vehicle and platform to understand complex driving contexts and perform appropriate control. For example, when a complex context such as “night + rainy conditions + traffic ahead” is recognized, the context model can simultaneously direct multiple controls, such as turning on interior lights, operating wipers, and increasing the distance between vehicles.

[0229] The voice receiving module (830) receives voice input from a user, converts it into text or command form, and transmits it to the response processing module (840). The voice receiving module (830) is not limited to simply performing a microphone function, but may include background noise removal, keyword detection, and user-specific speaker recognition functions. Through this, it functions as one of the main input means for vehicle control.

[0230] The response processing module (840) temporally synchronizes different inputs, such as voice, gestures, touch, and sensor signals, and semantically fuses them to interpret them as a single multidimensional input condition. For example, if a user says “open the window” and points toward the window, the two inputs can be fused to clearly produce a “window open” control signal. This reduces the probability of input misrecognition and provides an intuitive user experience.

[0231] The voice receiving module (830) applies a predefined priority rule when a conflict occurs between multidimensional inputs. For example, if a user issues a command to open a window while an external air quality sensor reports a “hazardous” state, the window opening command is ignored and the control for air quality protection is executed first, as safety and health-related conditions take precedence according to the priority rule. This ensures safety while maintaining the user experience.

[0232] The response processing module (840) refers to the stored event data and transmits a response (e.g., guidance message, voice output, display display) or a control signal (e.g., distance control, steering assistance, infotainment control) to the vehicle system according to the analysis result. The control signal generated at this time is linked with APIs of the body, chassis, infotainment, ADAS, and V2X domains to simultaneously perform multi-domain control.

[0233] The vehicle assistant device (800) according to the present invention can protect core control logic from security threats or malicious manipulation by processing event analysis and response control in a safe environment separated from User Space. In addition, the context model-based structure enables predictive and preemptive control beyond a simple event-response method, thereby simultaneously realizing vehicle safety and user-customized services.

[0234] In this way, the vehicle assistant device (800) is a system that structures data using a context slot module (810), forms policies using a context model generation module (820), and integrally interprets and controls user input and vehicle events through a voice reception module (830) and a response processing module (840). Through this, intelligent driving control based on complex context, which was not provided by existing technology, becomes possible.

[0235] FIG. 8 is a block diagram illustrating the operation process of a vehicle assistant device or platform according to an embodiment of the present invention. FIG. 8 includes the entire process from input event processing to context model creation, event handling, response generation, and vehicle and external device control. Through this, it demonstrates an operation and process that provides a safe and personalized driving experience by integrally reflecting the vehicle and user context, which is the core of the invention.

[0236] Referring to the left side of FIG. 8, various mobility events (910) are illustrated. Mobility events (910) are classified into environment-related events, weather-related events, passenger-related events, security-related events, safety-related events, use-case-related events, driving-related events, traffic-related events, vehicle-related events, and other events. These mobility events (910) occur through internal vehicle sensors, external sensors, networks, and user input, and are used as basic data for context analysis.

[0237] User input (830) can be provided directly through means such as voice, gesture, or touchscreen, and this input is transmitted to a response processing module (840) via a separate voice receiving module or sensor input interface. User input (830) acts as a major trigger affecting the entire system together with mobility events (910).

[0238] Various input mobility events (910) are first processed in the context slot generation module (810). The context slot generation module (810) divides the events and path data by time intervals and stores them in multiple slots. A slot is not a simple data set, but functions as a basic unit representing the driving context during a specific interval, and is utilized as an important unit of analysis in the subsequent model generation stage.

[0239] Data structured into slots is analyzed by a context model creation module (820). The context model creation module (820) creates a context model (905) that includes event conditions, state transition rules, and control action mappings. The created context model (905) is instantiated by user and use case, and multiple models can be managed simultaneously. For example, a 'commute route driving' model and a 'leisure driving' model can be maintained independently in the same vehicle.

[0240] The context model generation module (820) is trained and generated based on slot data, but conversely, the result of the model generation can again affect the operation process of the context slot generation module (810). For example, if a specific event pattern is repeatedly detected, the time unit of the slot can be reduced to allow for more detailed data collection.

[0241] The context model (905) is closely linked with the event handler service (930). The event handler service (930) is responsible for the functions of event registration, event triggering, and action request. That is, when a new event occurs, it is registered; when conditions are met, a trigger occurs; and an action corresponding to the event is requested.

[0242] The event handler service (930) is connected to in-vehicle applications and external use cases (906). For example, when a user requests “start navigation” or an external app sends a vehicle control command, the event handler service (930) registers this and maps it to the context model (905) to generate an appropriate control signal. This allows various applications to access vehicle resources while separated from the safety layer.

[0243] The generated context model (905) is connected to an environment configuration and control service (950). The environment configuration and control service (950) is linked with a vehicle domain API and an external device (940) to provide an extended mobility service.

[0244] Specifically, the environment configuration and control service (950) calls control operations defined in the context model through vehicle domain APIs (Body, Chassis, ADAS, IVI, V2X, etc.). In addition, the environment configuration and control service (950) also links with external vehicle devices (mobile terminals, wearables, smart home devices, etc.) to provide an extended mobility service that includes not only the interior of the vehicle but also the surrounding environment.

[0245] Meanwhile, the response processing module (840) interprets user input (voice, gesture, touch, etc.) (830) and generates a response based on the context model (905) and slot events. The response is provided to the user in the form of a guidance message or warning notification, or is converted into a driving control signal and executed via a vehicle domain API.

[0246] The response processing module (840) calculates the weather, traffic, and road data received from external sensors as an uncertainty value and adjusts the response intensity and control range according to this value. For example, if the reliability of the road icing data is low, the sensitivity of the braking assist is increased, or the warning indication is provided in a limited manner. This allows safety to be maintained while preventing over-control based on incorrect data.

[0247] In the process of creating context slots and models, route data (map data) (901), policies (902), action lists (903), and vehicle resources (904) are used as external reference data. Additionally, through integration with a server (920), model parameters are synchronized or federated learning is performed. Accordingly, individual vehicle data can be used to improve the common model without external transmission.

[0248] Meanwhile, the present invention adopts a structure that separates the use case / application layer from the core control layer. Through this, applications installed by the user can bypass the event handler service and the context model without directly accessing the vehicle control logic. Consequently, it is possible to prevent infringement of the security and safety layer and maintain system consistency.

[0249] For example, if a vehicle ahead suddenly stops while driving and a congestion warning is simultaneously received from an external traffic sensor, the context slot creation module (810) records the event in a slot. The context model creation module (820) defines a transition to an 'emergency deceleration' state according to state transition rules. The event handler service (830) registers this as a trigger, and the response processing module provides a warning notification to the driver while simultaneously transmitting a control signal to the environment configuration and control service to activate the braking assist system.

[0250] As such, the operational structure of the vehicle assistant device or platform illustrated in FIG. 8 forms a closed-loop operation of event collection, slot structuring, model creation, event handling, and response / control execution. Through this, the present invention overcomes the limitations of existing technologies, namely the problem of dependence on a specific vehicle brand or a single service, and realizes a differentiated mobility environment characterized by user customization, context-based, and multi-domain connectivity.

[0251] FIG. 10 is a diagram illustrating the process of a vehicle assistant device (800) according to one embodiment of the present invention storing various events occurring on a driving path (route length) in a plurality of context slots (CS1 to CS7) divided into time or distance units.

[0252] The context slot module (810) of the present invention can efficiently and precisely represent the driving context by dynamically adjusting the slot length in consideration of the frequency of events and the variability of the path.

[0253] Conventionally, a static slot method was often adopted to store all data in fixed units. However, this presented problems, such as information compression leading to the loss of detailed context in high-event density areas like urban intersections, and the waste of unnecessary computational and storage resources in low-event areas like highways. Accordingly, the present invention implements a dynamic slot partitioning method that adjusts slot lengths differently by considering event frequency and path complexity.

[0254] The mobility event (910) illustrated in FIG. 10 is expressed, for example, as “Hot Weather, Rain, No Network, No Street Lights, Heavy Fog, Bad Roads, Heavy Traffic,” etc. This represents weather, road, traffic, and network-related events that may be encountered on the driving path (1020) while the vehicle (100) is moving toward a set destination (1010). Here, the event (910) is mapped to a specific slot and defines the driving context of that section.

[0255] In an embodiment of the present invention, the calculation of the target slot length (L*) according to the invention can be defined as follows.

[0256]

[0257] Here,

[0258] L* stands for Target Slot Length. This refers to the size of the slot in time or distance units that the context slot module actually needs to apply.

[0259] Additionally, K represents the scaling constant. This is a proportionality factor used by the overall system designer to determine the base size for slot length calculation. For example, setting K to a large value increases the overall slot length, while setting it to a small value shortens it.

[0260] α represents the Event Frequency Weight. This is a parameter that adjusts the influence of the driving event frequency (λ) on the slot length.

[0261] λ is the event frequency. It represents the average number of events occurring within a unit of time or unit of distance. For example, it may include entering an intersection, sudden acceleration / braking, changing lanes, etc.

[0262] β represents the Route Variability Weight. This is a parameter that adjusts the influence of the Route Variability Index (RVI) on the slot length.

[0263] RVI stands for Route Variability Index. It is an indicator representing the complexity of a driving route, calculated by combining factors such as intersection density, rate of change in curvature and slope, frequency of road type changes, and the occurrence of sudden events. A higher RVI value indicates that the route is more complex and variable.

[0264] ε stands for the stabilizing term. It is a correction term used to prevent the formula from becoming unstable as the denominator approaches zero.

[0265] L_min is the minimum slot length. This is a lower limit value set to prevent the system from unnecessarily consuming resources by ensuring that the slots become excessively short.

[0266] L_max: Maximum Slot Length. This is an upper limit set to prevent events from being diluted or important situations from being missed due to excessively long slots.

[0267] Finally, clamp is a clamping function. It refers to an operation that forcibly limits the calculated L* value to L_min if it is smaller than L_min, and to L_max if it is larger than L_max.

[0268] Meanwhile, in the formula for the target slot length (L*), as λ (event frequency) increases, the slot length L* decreases. For example, in complex sections where many events occur, context is recorded in detail using short slots.

[0269] Additionally, as the RVI (Path Volatility Index) increases, the slot length L* decreases. This means that in sections with turns, branches, or high signal density, the context for each section is recorded using shorter slots.

[0270] In sections where both λ and RVI are low, the slot length L* increases and is integrated into a long slot. Accordingly, resource efficiency is maximized in simple sections, such as straight driving on a highway.

[0271] Continuing, in FIG. 10, the vehicle (100) moves from the starting point to the destination (1010) and creates multiple slots (CS1 to CS7).

[0272] For example, a “Hot Weather” event can be recorded in CS1, and a “Rain” event in CS2. “No Network” and “Heavy Fog” can be recorded simultaneously in CS3, and “Heavy Traffic” and “No Street Lights” can be recorded overlappingly in CS6. In this way, multiple events can be recorded simultaneously in a single slot, and the system interprets them comprehensively to derive appropriate control actions.

[0273] The slot length is dynamically calculated based on the frequency of driving events (λ) and / or the Route Variability Index (RVI). When the event frequency is high or route variability is high, the slot length is shortened, enabling detailed data collection. Conversely, when events are infrequent and the route is simple, the slot length is lengthened to improve computational efficiency.

[0274] For example, in urban areas with high intersection density, both λ and RVI are evaluated highly, so the slots are subdivided. Conversely, in straight highway sections, λ and RVI are low, so the slots are set to be long. This allows the same system to perform data collection and control policies optimized for the driving environment.

[0275] Meanwhile, the present invention does not necessarily require a destination to be set to operate. Even when there is no destination, slots can be set based on Points of Interest (POI) or a certain driving distance.

[0276] For example, if a driver sets only sequential POIs such as “Convenience Store → School → Gas Station” and does not specify a final destination, the system can record events by dividing the intervals between each POI into slots. For instance, a “Rain” event is recorded in the segment to the convenience store, a “Heavy Traffic” event in the segment to the school, and a “Bad Roads” event in the segment to the gas station.

[0277] Additionally, multiple events can be recorded simultaneously in a single slot. For example, if "No Network" and "Heavy Fog" occur simultaneously in CS3, the context model can recognize the complex context of limited visibility in a situation where the network is disconnected. In this case, the response processing module can provide voice-based navigation guidance and generate control signals to enhance the forward collision avoidance function.

[0278] Vehicle resources (e.g., powertrain, brakes, steering), sensor resources (radar, cameras, GPS), actuator resources, device resources, data, and channel bandwidth can be mapped to each slot.

[0279] As the slot length decreases, the resource allocation unit becomes more granular, enabling a concentrated response to specific events. Conversely, in longer slot sections, resources can be managed in an integrated manner to increase efficiency.

[0280] In the event of a safety-critical event, such as emergency braking or ABS intervention, the current slot length is temporarily fixed, even if it is being dynamically adjusted. This is to prevent data loss and ensure that slot adjustments are made again only after the safety event has been fully processed.

[0281] In the event of a discrepancy between sensor data or a communication error, the system evaluates the uncertainty value highly and applies slot length changes conservatively. For example, if the temperature sensor reports 2 degrees above freezing in a rainy situation, the system defers slot length changes to reflect the discrepancy between the two data, or records a collision in the slot metadata.

[0282] In addition, the slot module of the present invention can record the reason for a length change (λ value, RVI value, length before and after the change, time of application, etc.) in the metadata. This can be utilized for subsequent analysis or audit logs, thereby enhancing system transparency and enabling integration with security layers.

[0283] For example, if a vehicle travels through a continuous series of urban intersections immediately after departure, a short slot is generated, and events such as intersection entry, waiting at a signal, and departure are recorded separately. Subsequently, upon entering a highway, the slot length increases, integrating long driving sections without identical events into a single slot. If information regarding sudden traffic congestion is received before reaching the destination, the slot length is shortened again to record events within the congested section in detail.

[0284] As such, FIG. 10 can secure resource efficiency while minimizing information loss by flexibly adjusting the slot length according to the frequency of vent occurrence and path complexity. In addition, it can be applied to various scenarios such as destination-based, POI-based, or simple driving distance-based, thereby enabling intelligent driving control that reflects the user's customized context.

[0285] FIG. 11 is a diagram illustrating various events related to the creation of a context model according to an embodiment of the present invention.

[0286] As illustrated in FIG. 11, the vehicle driving environment is not composed of a single factor, but rather various categories of events occur simultaneously or sequentially and interact with each other, directly affecting vehicle control and user experience. Therefore, the present invention enables intelligent control by structurally collecting and analyzing these events and integrating them into a context model.

[0287] Environment Events (1101) correspond to external weather and road conditions. Examples include rainfall, snowfall, fog, heatwaves, road surface temperature, and changes in the road surface friction coefficient. Since these events are directly linked to driving safety, they are used as the most basic conditions when creating a context model.

[0288] Passenger Events (1102) reflect the status and needs of passengers inside the vehicle. For example, if a child is traveling, voice commands from the passenger (e.g., “Lower the temperature”), seat occupancy sensors, biosignal sensors, etc., correspond to passenger-related events. The present invention can perform user-customized control by reflecting this information as contextual conditions.

[0289] Security & Safety Events (1103) refer to situations directly related to the protection of the vehicle and the user. These include events such as attempted vehicle theft, abnormal access, accident collision detection, emergency braking intervention, and airbag deployment. The present invention incorporates these events into the state transition rules of a context model to enable rapid and automated control in the event of an emergency.

[0290] Driving Events (1104) are associated with direct driving actions of the driver or the vehicle system. These include lane changes, acceleration and deceleration, changes in steering angle, stopping and starting, and activation of cruise control. These driving events are stored in context slots along with time tags and subsequently used as input for a State Machine Based Model or a Learning Based Model.

[0291] Traffic Events (1105) refer to surrounding traffic conditions occurring outside of individual vehicles. These include road congestion, traffic light cycles, traffic accidents, and V2X-based vehicle-to-vehicle communication data. Traffic Events have a significant impact on the Route Variability Index (RVI) and are directly related to slot length adjustment or uncertainty-reflecting control.

[0292] Use case events (1106) relate to specific user purposes or service requirements. Examples include “work commute mode,” “leisure travel mode,” “taxi hailing service,” and “autonomous shuttle mode.” Even in the same traffic environment, the control policy selected by the vehicle may vary depending on the use case events, and this is a key element that characterizes the customized control operation of the present invention.

[0293] Other Events (1107) encompass various situations that do not fall under the categories listed above. For example, this includes factors that indirectly affect the vehicle's operating environment, such as network disconnection, failure to update map data, and requests for software updates.

[0294] As such, the events illustrated in FIG. 11 are not mutually exclusive, and multiple events occur simultaneously in actual operating situations. The present invention stores these events in a slot-based data structure rather than a simple list and achieves situation-specific optimal control by linking them with state transition rules.

[0295] In particular, the context model generation module can express event conditions and state transition rules in the form of a Finite State Machine (FSM). For example, it is a method of transitioning to "State C (Wiper operation, distance between vehicles increased)" when "Driving State A (Normal driving) + Event Condition B (Rain detection)" is satisfied. This ensures predictable and stable control by leveraging the advantages of rule-based control.

[0296] In another embodiment, the context model generation module may adopt a machine learning-based approach. That is, it learns repeatedly observed event patterns to automatically generate new transition rules or correct existing rules based on weights. For example, if the pattern “commute time + specific intersection = sudden traffic congestion” is continuously learned in a specific area, it can automatically provide detour route guidance when the same conditions are detected in the future.

[0297] Meanwhile, the event classification of Fig. 11 can be utilized not only for context models generated at the individual vehicle level but also for server-based aggregate analysis. By collectively analyzing event data from multiple vehicles, a common context model is formed, which can be distributed to individual vehicles via OTA. In this case, predictability specialized for specific regions or time periods is enhanced, and fleet learning occurs among vehicles.

[0298] Furthermore, this event data is linked to security and privacy layers. The source of the event is always authenticated, and data that could be directly related to a user's personal information is anonymized or pseudonymized before transmission. By generating slot-based audit logs, it is possible to track data tampering and the history of changes to control policies.

[0299] As such, the various events illustrated in FIG. 11 serve as key inputs in the context model generation process of the present invention, through which the vehicle contextually understands multidimensional and multi-domain situations and performs customized and intelligent control. Unlike conventional single-event-based control methods, this provides a more precise, safe, and user-friendly vehicle control environment.

[0300] FIG. 12 is a flowchart for explaining the operation method of a vehicle assistant device (800) according to an embodiment of the present invention.

[0301] Referring to FIG. 12, first, the vehicle assistant device (800) stores driving-related events and route information in multiple context slots divided by time units (1210). In this step, various driving events collected from internal vehicle sensors, external networks, map data, etc. are slotted by time intervals and managed as a structured dataset.

[0302] Next, driving-related events and path information stored in multiple context slots are analyzed. Through this analysis, a context model is generated that includes event conditions, state transition rules, and corresponding control actions (1220). That is, slot-based structured data is not merely stored but is transformed into a control policy that tracks the vehicle's state according to the driving context and learns and updates transition rules.

[0303] Subsequently, the vehicle assistant device (800) receives a user voice (1230). The voice receiving module (830) recognizes the user's speech (e.g., “Open the window,” “Find a nearby parking lot”) and transmits it to the response processing module (840).

[0304] The response processing module (840) continuously analyzes the received voice and input conditions occurring during driving. At this time, it generates a response based on the context model and events stored in the context slot, or outputs a control signal corresponding to the driving control operation (1240).

[0305] For example, if a voice command conflicts with a traffic situation or safety-related event, an appropriate control action can be determined and executed according to priority rules.

[0306] Below, each step (1210 to 1240) will be explained in more detail.

[0307] First, the step (1210) of storing multiple context slots is described.

[0308] The vehicle assistant device (800) collects event candidate data from internal vehicle sensors (camera, radar / lidar, IMU, GPS, vehicle speed / steering / braking signals), vehicle status signals (CAN / LIN / Ethernet), external network inputs (V2X messages, traffic / weather / road APIs), map / route data (including HD maps), user inputs (voice, gesture, touch), etc. When collecting, timestamps, location coordinates, and data source metadata are assigned together.

[0309] The context slot module (810) basically uses a time-based window, but applies a distance-based auxiliary policy when passing through meaningful sections on the map, such as intersections, ramps, and merging points. The slot length is calculated by dynamic splitting logic and is automatically scaled down / up according to the event occurrence frequency (λ) and route variability (RVI) indicators. Hysteresis and interpolation are applied when changing to prevent slot boundary jitter.

[0310] Additionally, timestamps from different sources are time-axis aligned (clock sync, NTP correction), and sensor drops and outliers are refined through missing value imputation and smoothing. Duplicate events at the same point in time are merged using priority rules (sensor quality / source reliability). Source identification fields are anonymized or pseudonymized by the security layer.

[0311] In each slot, (i) an event list (time of occurrence, coordinates, source, confidence), (ii) route / map context (road type, number of lanes, curvature / slope), (iii) external data snapshots (weather / traffic / road construction), (iv) resource allocations (computing / sensor / bandwidth quotas), and (v) quality metrics (data recency, uncertainty ε) are stored together as metadata. Storage is managed in a sequential rolling manner, and old slots are summarized / discarded according to policy.

[0312] Next, the context model creation step (1220) is described in detail as follows.

[0313] Explanatory variables such as rate of change, co-occurrence, intervals between events, segment dwell time, lane change patterns, and ABS / ESC intervention frequency are extracted from the time series accumulated in the slots. They are combined with map, traffic, and weather features to construct a multidimensional vector sequence.

[0314] Context models can be broadly classified into (a) state machine-based (FSM / HMM) models and (b) learning-based (ML) models.

[0315] (a) State machine-based (FSM / HMM) represents the set of states (e.g., normal / caution / danger / emergency), transition conditions (critical rules), transition probabilities, and control action mappings during transitions in a table.

[0316] (b) The learning base (ML) is trained to predict the event, state, and control of the next slot using RNN / LSTM / Transformer, Bayes filter, reinforcement learning policy, etc. The two methods can be configured as a hybrid and are operated in a rule-first / ML-correction form.

[0317] Independent context model instances are created for each user / use case within the same vehicle (e.g., Commuting / Leisure, Driver A / B). Each instance retains version, expiration date, and hit rate statistics, and if performance degrades, an automatic rollback or retraining trigger is activated.

[0318] When network connectivity is available, parameter synchronization and federated learning are performed with the server, and transmitted data is limited to the slot summary / feature level. At this time, the entire process of creation, update, and deployment is recorded in the slot-level audit log.

[0319] Next, the user voice reception step (1230) is described in detail as follows.

[0320] The voice receiving module (830) collects input via a microphone array, and converts it into STT (on-device / onboard) after detecting a wake word through beamforming, noise suppression, and echo canceling. At this time, multilingual, speaker identification, and command phrase parsing are supported. In addition, it falls back to a local dictionary / on-device ASR in the event of network failure.

[0321] At this point, the NLU extracts the command intent and entity (e.g., “Window / Open / Crew Seat”), and safety and security filters block dangerous commands (such as unblocking the UI while driving) or require approval. User profiles and preferences are linked with the personalization library and reflected as parameters.

[0322] Next, the response generation and control signal output step (1240) is described in detail.

[0323] The response processing module (840) synchronizes voice, gesture, touch, and sensor signals in time and fuses them into a single multidimensional input condition. In the event of a conflict, a conclusion is reached by applying the priority order of “Safety / Health > Compliance > Vehicle Protection > Convenience.” Example: “Open the window” vs. “Harmful outside air quality” → A decision can be made to propose ventilation measures + postpone opening the window.

[0324] The response intensity (sensitivity, speed, torque) and range (spatial and temporal application intervals) are scaled by reflecting the uncertainty values ​​of external weather, traffic, and road data. Simultaneously, if the reliability of the future event prediction calculated by the context model exceeds a critical threshold, preemptive control (such as increasing the distance between vehicles or preparing steering assist) is performed.

[0325] The determined action is transmitted to vehicle domain APIs (Body / Chassis / ADAS / IVI / V2X) via environment configuration and control services. Example: “Rain+Night+No Streetlights” slot → Calls Wiper / Defogger / Lighting / ACC sets. User notifications and automation commands can be transmitted concurrently to external devices (mobile, wearable, smart home).

[0326] Upper / lower limits, ramps (gradual changes), timeouts, and driver overrides are applied to the control. If the situation is resolved or reliability drops, it is rolled back to its original state, and all decisions and outputs are recorded in the slot audit log.

[0327] In this case, the outputs are classified into (i) vehicle control signals, (ii) audiovisual guidance and warnings, (iii) alternative suggestions (route, stop, rest), and (iv) policy update requests. Additionally, the HMI adheres to the principle of minimizing driver distraction and may use voice / head-up display priority channels in dangerous situations.

[0328] Meanwhile, as another embodiment, additional slot length adjustment is performed by immediately reducing the length of the next two slots and reallocating the resource quota upwards upon receiving a forecast of sudden congestion.

[0329] In another embodiment, federated learning is performed between the server and the vehicle terminal to periodically exchange parameters with the same vehicle type and region cluster to fine-tune the local model.

[0330] In another embodiment, control operations are performed independently using a locally stored context model when a network connection is unavailable. That is, independent decisions are continuously made using the local model and cached policies even during a network disconnection, and logs are synchronized to the server after the connection is restored.

[0331] In another embodiment, the response processing module (840) responds to the input of the same command by analyzing the user's past selection pattern and performing a differentiated control action for each user. For example, even with the same command, User A may perform differentiated control as “quiet cooling”, while User B may perform differentiated control as “strong wind + fast dehumidification”.

[0332] In the embodiment, the context slot module (810) can dynamically adjust the slot length by considering the frequency of event occurrence and path variability. This enables precise control by subdividing into short slots in sections with high event density, such as urban intersections, and enables efficient resource usage by integrating into long slots in simple sections, such as highways.

[0333] Additionally, in an embodiment, the response processing module (840) can synchronize voice, gesture, touch, and sensor signals to interpret them as a single multidimensional input condition. For example, when a user says “Execute navigation” while simultaneously inputting a destination on a touchscreen, these inputs are fused into a single complex command to generate a more accurate control signal.

[0334] Additionally, in the embodiment, an uncertainty value is assigned to weather, traffic, and road data received from an external sensor, and the response intensity and control range can be adjusted. For example, if the uncertainty of road icing information is high, the control signal may be partially applied or set to be effective only for short time intervals.

[0335] In addition, in the embodiment, the context model can predict future events and preemptively perform control actions if the reliability of the prediction result is above a threshold. For example, if a congestion pattern that occurs repeatedly in a specific section is learned, the vehicle can output signals to increase the distance between vehicles and change the route in advance before reaching that section.

[0336] Meanwhile, the present invention can be implemented with a similar operation flow not only at the device (800) but also at the platform and system levels.

[0337] From a platform perspective, the in-vehicle software platform performs the same procedures through slot managers, model generators, and response handlers. However, the platform emphasizes integration with the User Interface Layer (API), so the generated control signals are immediately applied via vehicle domain APIs (body, chassis, ADAS, V2X, etc.).

[0338] In terms of the system, the server aggregates and analyzes slot data collected from multiple vehicles to form a common context model and provides it to the vehicles. External devices (mobile, wearable, smart home devices) interact with the vehicles or the server to execute user-customized services, and if the vehicle is disconnected from the network, the external devices can supplement response processing instead of the server.

[0339] FIG. 13 is an exemplary block diagram for explaining the architecture of a context-based mobility control system according to an embodiment of the present invention.

[0340] That is, FIG. 13 is an exemplary block diagram in which a vehicle, a server, and an external device are organically interconnected to form a P-MUSE (Personalized Mobility User Space Engine).

[0341] The described system (1300) structures the driving context in slot units, creates and manages a context model based on the slot data, and safely performs event-driven multi-domain (Body, Chassis, ADAS, IVI, V2X) control. In addition, it incorporates personalized services and security / privacy layers for each user and use case, providing a consistent user experience regardless of brand, model, or region.

[0342] Below, the detailed configurations of the illustrated system (1300) will be described in detail.

[0343] First, the P-MUSE Event Handler Service (PMEHS) (1301) serves as the central event hub of the system. It manages all mobility-related events, such as environment, traffic, passengers, driving, security, and safety, in units of Registration, Trigger, and Action Request.

[0344] Events are published as time-limited event objects with a lifetime, rather than as simple API calls. Accordingly, they are automatically discarded upon expiration, preventing malfunctions caused by reuse or redoing. Additionally, event-based security is enhanced by accepting only origins that have passed authentication and authorization.

[0345] The context slot generator (810) divides driving events and path information collected from the vehicle into multiple slots based on a time / distance hybrid standard. The context slot generator (810) applies the Event Projection & Context Slotting Algorithm (EPSA) to pre-project predictable external events (ITS / V2X, traffic, weather) onto the path and dynamically adjusts the slot length according to event density and path variability. At this time, each slot records event bundles, map context, uncertainty / reliability, and resource allocation values ​​as metadata.

[0346] Slot data is passed to the context model. As previously mentioned, the context model automatically corrects the policy by either (a) having event conditions, state transition rules, and control mappings in a state machine (FSM / HMM)-based representation, or (b) learning iterative patterns based on learning (ML / RNN / Transformer / Reinforcement Learning). The context model is instantiated per user / use case and manages reliability, version, and validity period.

[0347] The Personalized Mobility Network Service (PMoNS) (1302) forms a Virtual Mobility Network (VMNS) by combining vehicles, personal mobility devices (Segways, e-bikes, etc.), and personalized devices such as smartphones, wearables, and home appliances. The VMNS has an independent instance for each passenger and is activated only under subscription, authentication, and authorization policies. Additionally, the Personalized Mobility Network Service (1302) selectively uses standard or proprietary protocols such as BLE, Wi-Fi Direct, Ethernet, CAN, LIN, C-V2X, and DSRC.

[0348] Meanwhile, although not explicitly described, the system (1300) may further include a Passenger Identification and Monitoring Service (PIMoS). Specifically, this performs passenger identification and seat mapping using face / speaker / seat sensors, etc., and monitors their behavior and environmental status. The Passenger Identification and Monitoring Service (PIMoS) increases resource efficiency by synchronizing with the PMEL (Personalized Environment Library) and maintaining only the data currently required for the VMNS as a local instance. At this time, available functions are controlled by reflecting permission and age policies.

[0349] The Personalized Mobility Environment Library (PMEL) (1303) is a library that stores personalized environment templates such as vehicle seats, climate control, lighting, audio / video, displays, personal mobility devices, and content preferences. It operates in a cloud / local redundancy structure, and only the settings required for the context are maintained locally. The Personalized Mobility Environment Library (PMEL) (1303) is linked with PIMoS (1302) / PMECCS (1305) to support automatic environment reconfiguration based on events.

[0350] The Brand and Model Abstraction Service (PBaMAS) (1304) abstracts and corrects functional differences between different brands and models to provide a consistent user experience. For example, it integrates model-specific scales and constraints, such as vehicle seat memory parameters, lighting brightness units, and climate control levels, into a conversion table. This allows for the lossless provision of user profiles across rental cars, shared vehicles, and multiple vehicles in a home.

[0351] The environment configuration and control service (PMECCS) (1305) converts and executes the policy of the context model into actual vehicle domain API calls (Body / Chassis / ADAS / IVI / V2X). For example, when “night + rain + no streetlights” → vehicle lights ON, wipers / defogger, HUD contrast enhancement, ACC target speed reduction, distance between vehicles increased, etc., can be orchestrated for multi-domain simultaneous control.

[0352] The Collective Transport Drop-and-Pick-up Service (CDaPTS) (1306) manages drop-off and pick-up points in vehicle-free zones (parks / malls / stations) and plans and tracks movement in non-vehicle sections by linking external personal mobility devices with PMoNS. The CDaPTS (1306) recommends a transit point with good safety, lighting, and congestion conditions in combination with context slots.

[0353] The Personalized Health Warning and Management Service (PHWMS) (1307) incorporates health data (with user consent) to reflect driving suitability, the need for rest, guidance on medical facilities along the route, etc., into the context model. When necessary, it restricts specific functions (such as high-performance driving mode) or outputs warnings.

[0354] The Energy, Data, and Bandwidth Management / Warning Service (PEDBMWS) (1308) virtualizes energy, data, and bandwidth in slot units and distributes them according to priority, and performs warnings or disables non-critical services when limits are exceeded. For example, in network vulnerable slots, media streaming is restricted and ADAS and map updates are prioritized.

[0355] A personalized point of interest service (PPoIS) (1309) recommends contextualized PoIs that reflect visit history, preferences, and current context, and triggers automatic consumption events when demand is met. For example, under conditions such as 'preference for low-caffeine coffee, rain, and avoidance of crowds,' it prioritizes suggesting stores with higher ease of stopping.

[0356] The User Space Core Orchestrator (USCOS) (1310) performs distributed choreography and central orchestration across the P-MUSE components. The USCOS (1310) allocates and reallocates computing / memory / network / device resources based on event arrival rates and priorities, and coordinates service execution timing and data exchange through the use case registry.

[0357] Personalized AI-enabled service (PAIES) (1311) performs slot length calculation, event importance evaluation, and training, deployment, and version management of AI models for prediction / policy generation. It supports privacy-friendly learning such as federated learning and knowledge distillation, and only safe approved models are loaded at runtime.

[0358] The Personalized Connectivity Service (PCS) (1312) performs the function of providing network connectivity optimized for each individual user and situational context. Additionally, the PCS (1312) manages various communication paths between the vehicle, edge server, cloud, and ITS infrastructure, and dynamically distributes data and bandwidth resources required by the context model.

[0359] In particular, the personalized connection service (1312) operates in close alignment with the safety / security context. For example, in a rainy situation, the “close window” event is allowed, but conversely, the “open door” request may be blocked. This means that it is executed according to the rules of a predefined event-action list and context model, and has built-in context-based control policies that go beyond simple network connections.

[0360] In addition, the personalized connection service (1312) is linked with PONIMS, PSM (Personalized Subscription Manager) (1314), and PMCM (Personalized Mobility Context Manager) to prevent unnecessary consumption of network resources and to implement a connection with minimum cost and maximum efficiency tailored to each user's situation. For example, when a family rides in the same vehicle, adult users can be allowed to use high-bandwidth streaming services, while minor users can be managed by restricting their data usage according to safety rules.

[0361] In other words, PCS is not a simple connection service, but a context-based personalized connection optimization engine, and can be considered a core component that simultaneously ensures safety and efficiency.

[0362] The Neighbor Optimal Context Recommendation Service (PONIMS) (1313) is linked with the Personalized Mobility Context Model (820) and performs the function of recommending an optimized movement context based on the context slot data of other users. The PONIMS (1312) collects and compares anonymized movement data from multiple vehicles or user groups to reflect the experience of 'Neighbor Users' with similar movement patterns or environmental conditions into the current user's driving context.

[0363] For example, if there is a user context that has recorded past optimal driving strategies or setting data under the same vehicle model and similar environments (rain, congested roads, same destination, etc.), the PONIMS of the present invention provides such data to the current user's vehicle to realize a safer and more efficient driving environment. In other words, it is not limited to the experience of a single user but enables an expanded intelligent mobility experience by sharing and utilizing optimal context data from a group.

[0364] In this way, PONIMS (1312) goes beyond simple recommendation functions and simultaneously enhances driving efficiency and personalization accuracy through a cyclic structure of collective optimal experience aggregation → individualized application → new context slot update.

[0365] The Personalized Subscription Manager (PSM) (1314) manages the subscription status of various mobility / content / communication services in an integrated manner. It prevents duplicate subscriptions and optimizes costs and resources by activating only the necessary services for each context.

[0366] The input / output service (IOS) (840) is a virtual I / O layer for in-vehicle and out-of-vehicle sensors / actuators and cloud sensors, and standardizes the data streams required by P-MUSE.

[0367] Additionally, the input / output service (IOS) (840) corresponds to the response processing module (840) of the vehicle assistant device (800) described with reference to FIG. 9. Specifically, the IOS provides a virtualization layer that manages inputs and outputs from various sensors, actuators, and user interface devices inside and outside the vehicle. In this process, control signal output, driving-related response generation, and event-based control operation performed by the response processing module (840) are transmitted to and executed by the actual vehicle functions (e.g., lighting, windows, air conditioning, ADAS control module, etc.) through the IOS. Therefore, the IOS can be described as a component that extends and realizes the operation of the response processing module at the system level, serving as an integrated I / O hub with internal and external vehicle services.

[0368] Additionally, the voice input interface (830) corresponds directly to the voice receiving module (830) of the vehicle assistant device (800) described with reference to FIG. 9. The voice input interface (830) collects the driver's speech, performs preprocessing (noise removal, voice feature extraction, etc.), and then transmits it to the event handler service (PMEHS) (1301) and context model module (820) within the P-MUSE architecture. This process corresponds to the user speech reception step (S1230) performed by the voice receiving module in FIG. 12, and the speech is converted into a response or control action based on the context model and context slot data.

[0369] Meanwhile, in the embodiment, all slot data is collected after source authentication (key / signature / integrity verification). Additionally, personal information is anonymized or pseudonymized, and model / policy change and control outputs are recorded as slot-level audit logs. The logs are stored in a tamper-proof manner to support regulatory compliance and post-analysis.

[0370] Furthermore, the server aggregates and analyzes slot data from multiple vehicles to form and deploy common context models based on region, season, and vehicle type, while the vehicles perform fine-tuning reflecting user preferences through local adaptation. External devices handle notifications, remote control, and continuous services (such as alternative route guidance in the event of network disconnection) to ensure service continuity.

[0371] A specific example of the operation of the system (1300) of FIG. 13 is as follows. For example, when approaching a merging section with insufficient streetlights during a rainy night, the context slot is subdivided into high resolution. Then, the Environment Configuration & Control Service (PMECCS) (1305) uses the context model to generate a multi-domain control set of “deceleration, increased distance between vehicles, enhanced lighting, and improved HUD contrast.” Subsequently, the Event Handler Service (PMEHS) (1301) manages event triggers, and the User Space Core Orchestrator (USCOS) (1310) coordinates the execution order of related services. The Passenger Identification & Monitoring Service (PIMoS) / Personalized Connection Service (PCS) (1312) maintains network and device connections, and the Energy, Data, & Bandwidth Management / Warning (PEDBMWS) (1308) allocates bandwidth and energy according to priority. Furthermore, this entire process is recorded in the security and audit layer to ensure traceability.

[0372] Meanwhile, although not illustrated in the drawings, the present invention can be extended to various modified embodiments as follows.

[0373] The context model according to the present invention is not only generated and utilized at the unit of a single vehicle, but can also be shared among multiple vehicles when they are driving on the same road or the same route section.

[0374] For example, if a preceding vehicle detects a "road icing" event in a specific section and records it in a context slot, a following vehicle can receive the same event in real time and update its context model. Through such cooperative context model sharing between vehicles, risk factors that are difficult to detect immediately due to the limitations of individual vehicle sensors can be recognized early, and this can be extended to a cooperative driving safety system utilizing V2V (Vehicle-to-Vehicle) communication.

[0375] In another embodiment, the context slot in the present invention is not limited to driving event and path information, but can also be applied to the management of a vehicle's energy resources (battery, fuel, bandwidth, computational resources, etc.).

[0376] For example, "estimated energy consumption" can be calculated on a slot-by-slot basis, and the vehicle's climate control, audio and video output, and display brightness can be adjusted according to event conditions. Furthermore, the balance between energy efficiency and safety can be coordinated in real time, such as temporarily restricting power-intensive infotainment functions in slots where highly uncertain traffic data is detected, or conversely, enhancing user convenience features in slots with high stability.

[0377] In another embodiment, the response processing module according to the present invention may be linked with an AR (augmented reality) or VR (virtual reality) display device in addition to voice, gesture, and touch.

[0378] For example, when a user wearing a head-up display (HUD) or AR glasses visually points to a specific Point of Interest (POI), the response processing module can combine the corresponding eye-tracking data with context slot events to recognize it as a “destination setting” event. At this time, the context model combines with traffic conditions and weather events to suggest an optimal route, which extends into a driving assistant based on an immersive user experience that goes beyond simple voice / gesture input.

[0379] In another embodiment, the present invention can support federated learning-based adaptive model improvement in addition to a server collecting and analyzing data from multiple vehicles to form a common context model.

[0380] In this case, each vehicle does not transmit raw data externally, but sends only locally trained model parameters to the server. The server collectively integrates these parameters to generate an improved global model and redistributes it back to the vehicles. This allows for the improvement of driving safety and prediction accuracy for the entire vehicle fleet without the exposure of personal information, and enables the smooth application of the present invention, particularly in environments with strict data privacy regulations.

[0381] As described above, the vehicle assistant device according to an embodiment of the present invention, specifically, stores driving-related events and route information in context slots on a time-based basis and generates a context model including event conditions, state transition rules, and control actions based thereon, thereby enabling intelligent control that reflects complex driving contexts. Therefore, unlike conventional single-event-based control, it can provide more precise and context-appropriate responses by integrally considering the driving environment and user input. Furthermore, according to the present invention, the context model can be linked with APIs of multiple domains, such as the vehicle body, chassis, infotainment, ADAS, and V2X, to simultaneously output control signals. Accordingly, various functions of the vehicle can be organically combined and controlled in response to changes in driving conditions, and customized services can be provided by linking with external servers and external devices. In addition, the present invention may include a security layer that performs source authentication of context slot data, anonymization processing of personal information, and slot-based audit log generation. Through this, the reliability of collected data is guaranteed, user privacy is protected, and the traceability of system operation history is secured.

[0382] The above-described invention can be implemented as computer-readable code (or an application or software) on a medium on which a program is recorded. The above-described control method for an autonomous vehicle can be realized by code stored in memory, etc.

[0383] Computer-readable media include all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drives (HDDs), Solid State Disks (SSDs), Silicon Disk Drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc., and also include those implemented in the form of carrier waves (e.g., transmission over the Internet). Furthermore, the computer may include a processor or a control unit. Accordingly, the above detailed description should not be interpreted restrictively in all respects and should be considered exemplary. The scope of the invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.

Claims

1. As an assistant device installed in a vehicle and responding to situations occurring while driving, the driving assistant device comprises: A context slot module that stores driving-related events and route information in multiple context slots divided by time units; A context model generation module that analyzes driving-related events and path information stored in the plurality of context slots to generate a context model including event conditions, state transition rules, and corresponding control actions; A voice receiving module for receiving user voice; and An assistant device characterized by including a response processing module that analyzes the received voice and input conditions occurring during driving, generates a response based on events stored in the context model and context slot, or outputs a control signal corresponding to a driving control operation.

2. In Paragraph 1, The above context slot module is, A device characterized by dynamically adjusting the length of a slot by considering the frequency of occurrence of driving-related events and path variability.

3. In Paragraph 1, The above context model generation module is, A device characterized by generating a context model by expressing the above event conditions and the above state transition rules in a machine form.

4. In Paragraph 3, The above context model generation module is, A device characterized by learning the repetitive patterns of driving-related events using a machine learning algorithm and generating multiple context models by reflecting this.

5. In Paragraph 1, The above response processing module is, A device characterized by fusion analysis of voice, gesture, touch, and sensor signals into multidimensional input conditions by time synchronization.

6. In Paragraph 5, The above response processing module is, A device characterized by determining a response according to a predefined priority rule when multidimensional input conditions conflict.

7. In Paragraph 1, The above response processing module is, A device characterized by calculating weather, traffic, and road data received from the outside as uncertainty values, and adjusting response strength and control range differently according to said values.

8. In Paragraph 1, The above response processing module is, A device characterized by predicting the next driving-related event based on a generated context model and outputting a control signal to perform a corresponding control action when the reliability of the prediction result is above a threshold.

9. As an assistant platform running in a vehicle, the platform comprises: A slot manager that manages context slot data; A model generator that analyzes the above context slot data to generate a context model including event conditions, state transition rules, and corresponding control actions; An input interface that receives commands from a user; and A driving assistant platform characterized by including a response handler that interprets the above command and input conditions occurring during driving, generates a response based on events stored in the context model and the context slot, or outputs a control signal corresponding to a driving control operation through a vehicle function API.

10. In Paragraph 9, The above slot manager is, A platform characterized by managing the above context slot data in a rolling manner, sequentially storing the data and deleting old data after a certain period.

11. In Paragraph 9, The above model generator is, A platform characterized by performing federated learning between a server and a vehicle terminal to update model parameters without external transmission of individual vehicle data.

12. In Paragraph 9, The above response handler is, A platform characterized by performing control operations independently using a locally stored context model when network connectivity is unavailable.

13. In Paragraph 9, The above response handler is, A platform characterized by being configured to perform differentiated control actions for each user by analyzing the user's past selection patterns in response to the input of the same command.

14. In Paragraph 9, The above platform is, A platform characterized by accessing APIs of the vehicle body, chassis, infotainment, ADAS, and V2X domains to simultaneously control functions of multiple domains based on the context model.

15. A context-based mobility control system including a vehicle, a server, and an external device, comprising: The vehicle stores driving-related events and path information in a plurality of context slots, and generates a response corresponding to an input condition or outputs a control signal corresponding to a driving control operation based on the context slots and a context model. The server collects and analyzes the context slot data to learn a context model including event conditions, state transition rules, and corresponding control actions, and provides control rules according to the context model to the vehicle. A system characterized in that the above external device communicates with the vehicle or the server to provide a user-customized service based on the response or control signal.

16. In Paragraph 15, The above server is, A system characterized by aggregating and analyzing data collected from multiple vehicles to form a common context model and providing it to each vehicle.

17. In Paragraph 15, The above server is, A system characterized by analyzing a user's repetitive response patterns to automatically update control policies for the same input conditions.

18. In Paragraph 15, The above external device is, A system characterized by being at least one of a mobile terminal, a wearable device, or a smart home device, and providing a user-customized service based on a control signal output from the vehicle.

19. In Paragraph 15, The above server is, A system characterized by maintaining service continuity by communicating directly with the external device when the vehicle's network is disconnected.

20. In Paragraph 15, The above system is, A system characterized by further including a security layer for performing source authentication of context slot data, anonymization for user privacy protection, and slot-unit audit log generation.

Images