Digital Assistant Human-Machine Interaction Device

Abstract

The digital assistant device (10) includes input means (12, 14) configured to acquire information from a plurality of users within a predetermined area around the digital assistant device (10) and output a signal indicative of the acquired information, a processing means (16) configured to receive the signal output by the input means (12, 14) and generate individual user output information for some or each of the users, and an output means (18) configured to output the generated output information in a user-specific manner that is uniquely assigned to a particular user.

Description

Technical Field

[0001] The present invention relates to a digital assistant device, and more particularly to an in-vehicle digital assistant device.

Background Art

[0002] Over the past few years, various digital assistants have been proposed as digital assistants aiming for human-like interactions with users. However, the available digital assistants do not interact individually with multiple users in a multi-user environment.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Therefore, an object of the present invention is to provide a digital assistant device configured for individual interaction with multiple users in a multi-user environment.

Means for Solving the Problems

[0004] According to the present invention, this object is achieved by a digital assistant device according to claim 1, the digital assistant device comprising: input means configured to acquire information from a plurality of users within a predetermined area around the digital assistant device and output a signal indicating the acquired information; processing means configured to receive the signal output by the input means and generate individual user output information for several or each of the users; and output means configured to output the generated output information uniquely assigned to a specific user in a user-specific manner. The digital assistant device can be configured as an in-vehicle digital assistant device optionally attached to the dashboard of an automobile. In the in-vehicle environment, the plurality of users can include or consist of a driver and a relief driver.

[0005] In a predetermined area surrounding a digital assistant device, the distance between the user, some or all users, and the digital assistant device or at least some of its components may range from a few centimeters to a few meters. In the case of an in-car assistant device, the predetermined area may correspond to the interior of a car equipped with the digital assistant device according to the present invention.

[0006] The digital assistant device according to the present invention is configured to handle input from multiple users individually, process information received from multiple users individually, and output information in a user-specific manner. As a result, the digital assistant device according to the present invention is configured for individual interactions with multiple users in a multi-user environment.

[0007] In exemplary embodiments, the processing means may be configured to determine the user's position relative to the digital assistant device, or the positions of several users, or the position of each user, from the signal output by the input means. This may be done through either sound sensing, utilizing sound localization and / or visual sensing (e.g., a camera device). The relative positions of the user / multiple users may be used to optimize the output of the output means for each individual user, thereby improving the user experience.

[0008] The processing means may include one or more processors operably coupled to one or more memories, such as read-only memory (ROM) and / or random access memory (RAM). The processing means may be part of an embedded system and / or cloud-based processing means.

[0009] The output means may include a display. The display may be configured as or include an LCD, LED, μLED, or OLED display, and / or may be configured as a high-resolution display. The display may include a curved display portion. The curved display portion can increase the viewing angle compared to a flat display, thus facilitating the ability of a digital assistant device to interact with multiple users.

[0010] In exemplary embodiments, the curved display portion may be cylindrical or conical. Both cylindrical and conical curved display portions can achieve a wide viewing angle, which is advantageous in terms of interaction with multiple users. The viewing angle may range from 270° to 360°. Thus, partial to full 360° interaction can be implemented. With a conical curved display portion, the visible area can be expanded in multiple directions (horizontal and vertical) of the viewing angle compared to a cylindrical curved display portion.

[0011] In exemplary embodiments, a display may be configured to show output information uniquely assigned to a particular user on a portion of the display uniquely assigned to that particular user. In the case of a display including a curved display portion, the display may be configured to show output information uniquely assigned to a particular user on a portion of the curved display portion facing the position of that particular user. In this way, the user can fully concentrate on information specifically addressed to them. This may be particularly important in relation to a digital assistant device configured as an in-vehicle digital assistant device. In this case, if the driver, who is one of several users, does not visually recognize information addressed to another of the several users, the driver's level of distraction can be reduced. This can therefore contribute to improved driving performance and, consequently, driving safety.

[0012] In one or more exemplary embodiments, the digital assistant device may further include a housing defining a recess configured to accommodate a display, and a mechanism configured to house the display within the recess so that it is not visible from the outside of the digital assistant device, and to move the display out of the recess so that it is visible from the outside. This configuration may allow the user's needs to be better considered compared to a permanently visible display. This may be particularly important in relation to an in-vehicle digital assistant device that can effectively reduce the driver's level of distraction by housing the display within a recess.

[0013] If the display includes a touch-sensitive area at the edge of the curved display portion, the digital assistant device may be configured such that the touch-sensitive area remains accessible from outside the housing even when the display is housed in a recess. In this way, even when the display is not visible from the outside, the touch-sensitive area can still be used as an input means. The touch-sensitive area may have a contour that matches the contour of the housing around the recess. In this way, easy access to the touch-sensitive area can be ensured.

[0014] In one or more exemplary embodiments, the output means may further include illumination means formed separately from the display. The illumination means may function as an indicator of the touch-sensitive portion when the display is housed within a recess. The illumination means may be configured to provide user-specific light to clearly indicate the user interacting with the digital assistant device. The illumination means may include a plurality of colors that can be used to provide visual guidance to the user as part of multisensory input to the user. Examples include green to indicate a “positive” or “clear” state by the assistant device, or yellow to indicate that attention is needed. The illumination means may be configured to surround the recess.

[0015] A digital assistant device may be configured to visually represent a specific user to whom output information is uniquely assigned by displaying a user-specific still image or video (avatar). In an exemplary embodiment, the user-specific avatar may be displayed when the digital assistant device switches between different users or when information is output to a specific user.

[0016] In one or more exemplary embodiments, the input means may include one or more digital cameras and / or one or more microphones, optionally one or more omnidirectional microphones. The input means configured in this way can reliably determine the number of users relative to the digital assistant device and the users' relative positions, for example, by comparing signals received from different digital cameras or microphones. These sensors can facilitate human-like interaction between the digital assistant device and the user by monitoring the user's emotional state. In addition, the input means may further include radar, lidar, and infrared sensors for gesture and / or health sensing.

[0017] The digital assistant device may further include one or more speakers as output means to enable human-like interaction between the digital assistant device and multiple users. Furthermore, a voice-only mode may be implemented by the output means thus configured to reduce driver distraction levels and allow for different user-preferred driving modes. The voice-only mode may be provided when the display is retracted.

[0018] In one or more exemplary embodiments, the digital assistant device may include onboard or cloud-based artificial intelligence implemented by neural networks or simple object-oriented programming, partial / limited or full vehicle head unit access, a PC-based or fully internally developed embedded solution, and / or a secure network.

[0019] As described above, the digital assistant device can implement visual interaction in a multi-user environment by addressing each user individually. In addition, by capturing emotions and other sensory inputs, it is possible to provide necessary information visually and audibly, or prompt the user to trigger relevant functions, for example, in a connected car environment. Furthermore, by adapting to different situations initiated by the user or sensed through the digital assistant device, it is possible to address the user's human-machine interaction (HMI) needs in a variety of scenarios.

[0020] Examples of information and functions include displaying a welcome screen with integrated face detection, displaying an overview of the vehicle's status, and providing a service reservation interface, traffic information updates and navigation, fatigue detection, autonomous driving handover, and music selection based on mood detection.

[0021] The present invention will be described in more detail below with reference to the accompanying drawings. [Brief explanation of the drawing]

[0022] [Figure 1] This is a schematic diagram of an example digital assistant device. [Figure 2] This is a schematic diagram of the display, including the curved display section that implements the navigation function. [Figure 3] This is a schematic diagram of the curved display section that shows the user's unique avatar. [Modes for carrying out the invention]

[0023] FIG. 1 is a schematic diagram of a digital assistant device 10. The digital assistant device 10 is configured to obtain information from a plurality of users within a predetermined area around the digital assistant device 10 and output a signal indicating the obtained information. The digital assistant device 10 may include one or more input means such as one or more digital cameras (e.g., cameras of a driver monitoring system) 12 or one or more microphones (e.g., one or more omnidirectional microphones) 14, a processing means 16 configured to receive the signals output by the input means 12, 14 and generate individual user output information for several or each of the users, and an output means 18 configured to output the generated output information uniquely assigned to one or more specific users in a user-specific manner. The digital assistant device 10 may be configured as an in-vehicle digital assistant device optionally attached to the dashboard of an automobile.

[0024] With the above configuration, the digital assistant device 10 may be configured to individually handle inputs from a plurality of users, individually process the information received from the plurality of users, and output the information in a singular / plural user-specific manner. As a result, the digital assistant device 10 may be configured for individual interaction with a plurality of users in a multi-user environment such as inside a vehicle. In the vehicle environment, the plurality of users may include or consist of a driver and a relief driver. In this case, the predetermined area around the digital assistant device 10 may correspond to the inside of the vehicle.

[0025] The processing means 10 may be configured to identify the position of the user with respect to the digital assistant device 10 from the signals output by the input means 12, 14. The relative position of the user may be used to optimize the output of the output means 18 for the individual user, thereby improving the user experience.

[0026] The processing means 16 may include one or more processors operably coupled to one or more memories such as a read-only memory (ROM) and / or a random access memory (RAM).

[0027] The output means 18 may include a display 20. The display 20 may be configured as or include an LCD, LED, μLED, or OLED display, and / or may be configured as a high-resolution display. The display 20 may include a curved display portion 22. The curved display portion 22 can increase the viewing angle compared to a flat display, thus facilitating the digital assistant device 10's ability to interact with multiple users individually.

[0028] In exemplary embodiments, the curved display portion 22 may be cylindrical or conical, as shown in Figures 1-3. Both cylindrical and conical curved display portions 22 can achieve multi-directional (horizontal and / or vertical) wide viewing angles, which is therefore advantageous in terms of interaction with multiple users. In addition, in this way, partial to full 360° interaction can be implemented. The viewing angle may range from 270° to 360°. With a conical curved display portion 22, the visible area can be expanded compared to a cylindrical curved display portion.

[0029] At least a portion of the display 20 may be configured as a touch-sensitive display. In this way, the display 20 may function as an input means, i.e., the input and output means are combined into a common component, thereby contributing to a compact overall setup. In exemplary embodiments, the display 20 may include a touch-sensitive portion 24 separated from the curved display portion 22. The touch-sensitive portion 24 may be located at the end of the curved display portion 22 (e.g., the top in the direction of gravity), as shown in Figures 1-3. The touch-sensitive portion 24 may have an ergonomic shape to facilitate user input. In the example shown in the figures, the touch-sensitive portion 24 is tilted forward. This shape may be advantageous in an automotive environment as it is easily accessible by the driver and the relief driver, thereby facilitating input by the driver or relief driver via the touch-sensitive portion 24.

[0030] In exemplary embodiments, the display 20 may be configured to display output information uniquely assigned to a particular user on a portion of the display 20 that is uniquely assigned to that particular user. As shown in Figures 1-3, in the case of a display 20 including a curved display portion 22, the display 20 may be configured to display output information uniquely assigned to a particular user on a portion of the curved display portion 22 that faces the position of the particular user. In this way, other users are not distracted by information not addressed to them.

[0031] This is illustrated in Figure 2, where the navigation instructions 26 are displayed on one side of the curved display portion 22 so that they are not visible from the other side. This may be particularly important for the digital assistant device 10, which is configured as an in-vehicle digital assistant device where the navigation instructions are relevant only to the driver and not to the relief driver. Conversely, information that is not relevant to the driver may be displayed on the side of the curved display portion 22 that is not visible to the driver. In this way, the level of driver distraction can be minimized. This can therefore contribute to improving driving performance and, consequently, driving safety.

[0032] As shown in Figure 1, the digital assistant device 10 may further include a housing 28 defining a recess 30 configured to accommodate a display 20, and a drive mechanism 32 configured to house the display 20 within the recess 30 so that the curved display portion 22 is not visible from the outside of the digital assistant device 10, and to move the display 20 out of the recess 30 so that the curved display portion 22 is visible from the outside. This configuration may allow the user's needs to be better considered compared to a display 20 that is permanently visible from the outside of the digital assistant device 10. This may be particularly important for an in-vehicle digital assistant device that can effectively reduce the driver's level of distraction by housing the display 20 within the recess 30.

[0033] If the display 20 includes a touch-sensitive portion 24 at the edge (top) of the curved display portion 22, the digital assistant device 10 may be configured such that the touch-sensitive portion 24 is still accessible from outside the housing 28. In this way, even if the display 20 is not visible from the outside, the touch-sensitive portion 24 can still be used as an input means. The touch-sensitive portion 24 may have a contour that matches the contour of the housing 28 around the recess 30. In this way, access to the touch-sensitive portion 24 can be facilitated.

[0034] As shown in Figure 1, the output means 18 may further include an illumination means 34 formed separately from the display 20. The illumination means 34 may function as an indicator of the touch-sensitive portion 24 when the display 20 is housed within the recess 30. The illumination means 34 may be configured to provide user-specific light to clearly indicate the user interacting with the digital assistant device 10. The illumination means 34 may include a plurality of colors that can be used to provide visual guidance to the user as part of the multisensory input to the user. Examples include green to indicate a “positive” or “clear” state by the assistant device, or yellow to indicate that attention is needed. The illumination means 34 may be configured to surround the recess 30, as shown in Figure 1.

[0035] The digital assistant device 10 may be configured to visually indicate a specific user to whom output information is uniquely assigned by displaying a user-specific still image or video avatar (Figure 3). In an exemplary embodiment, the user-specific avatar 36 may be displayed when the digital assistant device 10 switches between different users or when information is output to a specific user.

[0036] As described above, the input means may include one or more digital cameras 12 and / or one or more microphones 14. With the input means configured in this way, the number of users relative to the digital assistant device 10 and the users' relative positions can be reliably determined, for example, by comparing signals received from different digital cameras 12 or microphones 14. These input means make it possible to facilitate human-like interaction between the digital assistant device 10 and the user by monitoring the user's emotional state. In addition, the input means may further include radar, lidar, and infrared sensors for gesture and / or health sensing.

[0037] The digital assistant device 10 may further include one or more speakers 38 as output means 18 to enable human-like interaction between the digital assistant device 10 and multiple users. Furthermore, a voice-only mode can be implemented by the output means 18 configured in this way to reduce the driver's level of distraction and enable different users to prefer different driving modes.

[0038] In one or more exemplary embodiments, the digital assistant device 10 may include onboard or cloud-based artificial intelligence implemented by a neural network or simple object-oriented programming, partial / limited or full vehicle head unit access, a PC-based or fully internally developed embedded solution, and / or a secure network.

[0039] As described above, the digital assistant device 10 enables visual interaction in a multi-user environment by addressing each user individually. In addition, by capturing emotions and other sensory inputs, it can provide necessary information visually and audibly, or prompt the user to trigger relevant functions, for example, in a connected car environment. Furthermore, by adapting to different situations initiated by the user or sensed through the digital assistant device 10, it can address the user's human-machine interaction (HMI) needs in various scenarios.

[0040] Examples of information and functions of the digital assistant device include displaying a welcome screen with integrated face detection, displaying an overview of the vehicle's status, and providing a service reservation interface, traffic information updates and navigation, fatigue detection, autonomous driving handover, and music selection based on mood detection. While this application relates to the invention described in the claims, it also includes the following other aspects. 1. A digital assistant device (10), Input means (12, 14) configured to acquire information from multiple users within a predetermined area surrounding the digital assistant device (10) and to output a signal indicating the acquired information, A processing means (16) is configured to receive the signals output by the input means (12, 14) and to generate individual user output information for several or each of the users, Output means (18) configured to output the generated output information, which is uniquely assigned to a specific user, in a user-specific manner. Digital assistant devices (10) including the following. 2. The processing means (16) is configured to identify the location of the user relative to the digital assistant device (10), or the locations of several users, or the location of each user, from the signals output by the input means (12, 14), as described in 1 above. 3. The output means (18) is a digital assistant device (10) according to 1 or 2 above, including a display (20). 4. The display (20) is the digital assistant device (10) described in 3 above, including a curved display portion (22). 5. The curved display portion (22) is a digital assistant device (10) as described in item 4, having a cylindrical or conical shape. 6. The digital assistant device (10) according to 4 or 5 above, wherein the display (20) includes a touch-sensitive portion (24) at the end of the curved display portion (22). 7. The digital assistant device (10) according to any one of 3 to 6 above, wherein the display (20) is configured to display the output information, which is uniquely assigned to a specific user, on a portion of the display (20) that is uniquely assigned to the specific user. 8. The digital assistant device (10) according to any one of the above 2 or 7 or 4 to 6, wherein the display (20) is configured to display output information uniquely assigned to a specific user on a portion of the curved display portion (22) facing the position of the specific user. 9. A digital assistant device (10) according to any one of 3 to 8 above, further comprising: a housing (28) defining a recess (30) configured to accommodate the display (20); and a drive mechanism (32) configured to house the display (20) in the recess (30) and to move the display (20) out of the recess (30). 10. In the stored state of the display (20), the touch-sensitive portion (24) at the end of the curved display portion (22) is accessible from outside the housing (28) of the digital assistant device (10) according to 6 or 9 above. 11. The output means (18) further includes a lighting means (34) formed separately from the display (20), the digital assistant device (10) according to any one of 3 to 10 above. 12. A digital assistant device (10) according to any one of 3 to 11 above, configured to visually indicate the specific user to whom the output information is uniquely assigned by displaying a user-specific still image or video (36). 13. The input means (12, 14) is a digital assistant device (10) according to any one of 1 to 12 above, comprising one or more digital cameras (12) and / or one or more microphones (14), and optionally one or more omnidirectional microphones (14). 14. The output means (18) further includes one or more speakers (38), the digital assistant device (10) according to any one of items 1 to 13 above. 15. A digital assistant device (10) described in any one of items 1 to 14 above, configured as an in-vehicle digital assistant device. [Explanation of Symbols]

[0041] 10 Digital Assistant Devices 12 Digital Cameras 14 Microphones 16 Processing means 18 Output means 20 displays 22. Curved display section 24 Touch-sensitive area 26 Navigation Instructions 28 Housing 30 recesses 32 Drive mechanism 34 Lighting means 36 Avatars 38 speakers

Claims

1. A digital assistant device (10), The aforementioned digital assistant device (10) Input means (12, 14) configured to acquire information from multiple users within a predetermined area surrounding the digital assistant device (10) and to output a signal indicating the acquired information, A processing means (16) configured to receive signals output by the input means (12, 14) and to generate individual user output information for several or each of the users, Output means (18) that are uniquely assigned to a specific user and configured to output the generated output information in a user-specific manner, and including, In the said digital assistant device (10), The output means (18) includes a display (20), and the display (20) includes a curved display portion (22). The processing means (16) is configured to identify the user's position relative to the digital assistant device (10), the positions of several users, or the position of each user from the signals output by the input means (12, 14). The display (20) is configured to display output information uniquely assigned to a specific user on a portion of the curved display portion (22) facing the position of the specific user. The display (20) is configured to display output information uniquely assigned to a specific user on a portion of the display (20) uniquely assigned to the specific user. A portion of the display (20) is the curved display portion (22) that faces the position of the specific user. Digital assistant device (10).

2. The curved display portion (22) has a cylindrical or conical shape, as described in claim 1, for the digital assistant device (10).

3. The digital assistant device (10) according to claim 1 or 2, wherein the display (20) includes a touch-sensitive portion (24) at the end of the curved display portion (22).

4. The digital assistant device (10) according to claim 3, further comprising: a housing (28) defining a recess (30) configured to accommodate the display (20); and a drive mechanism (32) configured to house the display (20) in the recess (30) and to move the display (20) out of the recess (30).

5. The digital assistant device (10) according to claim 4, wherein, in the stowed state of the display (20), the touch-sensitive portion (24) at the end of the curved display portion (22) is accessible from outside the housing (28).

6. The digital assistant device (10) according to any one of claims 1 to 5, wherein the output means (18) further includes an illumination means (34) formed separately from the display (20).

7. A digital assistant device (10) according to any one of claims 1 to 6, configured to visually indicate the specific user to whom the output information is uniquely assigned by displaying a user-specific still image or video (36).

8. The input means (12, 14) includes one or more digital cameras (12) and / or one or more microphones (14), and optionally one or more omnidirectional microphones (14), according to any one of claims 1 to 7, for the digital assistant device (10).

9. The output means (18) further comprises one or more speakers (38), the digital assistant device (10) according to any one of claims 1 to 8.

10. A digital assistant device (10) according to any one of claims 1 to 9, configured as an in-vehicle digital assistant device.

Images