A system for searching using multiple displays.

Abstract

This invention relates to a system for performing a search using multiple displays. The system includes a lower display located in the cockpit and an upper display located above the lower display. The system includes: a processor; and a computer-readable medium communicating with the processor and storing instructions that, when executed by the processor, cause the processor to control the system to perform: receiving a first user input including an integrated search command signal; and, in response to receiving the first user input, displaying an integrated search window associated with the integrated search function on at least one of the upper and lower displays.

Description

[0001] Cross-references to related applications

[0002] This application claims priority and benefit to Korean Patent Application No. 10-2021-0011861, No. 10-2021-0011862, No. 10-2021-0011863, No. 10-2021-0011864, No. 10-2021-0011865, No. 10-2021-0011866, and No. 10-2021-0011867, all of which are filed on January 27, 2021, the disclosures of which are incorporated herein by reference in their entirety. Technical Field

[0003] This invention relates to a system for performing searches using multiple displays. Background Technology

[0004] According to relevant technologies, there is a problem that only searching specific menus (e.g., searching for navigation destinations) can be performed within a limited area of ​​the vehicle's in-vehicle display. Summary of the Invention

[0005] The present invention aims to provide a device capable of performing integrated searches using separate upper / lower displays (multiple displays) located in different positions.

[0006] According to one aspect of this disclosure, an apparatus for performing a search using multiple displays is provided, the apparatus comprising: an input unit configured to receive an integrated search command signal; a memory storing a program for controlling the display of an integrated search function on at least one of an upper display and a lower display according to the integrated search command signal; and a processor configured to execute the program, wherein the processor controls the display of an integrated search window for at least one of a lower display disposed in a cockpit and an upper display disposed above the lower display.

[0007] The processor can use selection information about the user profile to display search results for each menu item corresponding to the user profile and search terms.

[0008] When an integrated search request is received, the processor can control the display of a list of items in the menu according to the most recently used order of each menu based on the user profile.

[0009] When an integrated search request is received and search terms are entered, the processor can control the display of a list corresponding to the search terms in a predetermined order.

[0010] The processor can control the arrangement and display of a list of each predetermined menu on the upper display, and the display of frequently used information for each predetermined menu on the lower display.

[0011] The processor can control the display on the upper screen to show information based on the most recently used order of each predefined menu item, and when a search term is entered on the lower screen, the processor can control the display on the upper screen to show a list corresponding to the search term in a predefined order.

[0012] When an application function is being executed on the upper display, the processor can control the display to show the integrated search window on the lower display.

[0013] When passenger mode is in operation, the processor can control the use of the lower display to show the integrated search window. Attached Figure Description

[0014] The above and other objects, features and advantages of the present invention will become more apparent to those skilled in the art from the detailed description of exemplary embodiments of the present invention with reference to the accompanying drawings, in which:

[0015] Figure 1 This is a concept diagram of an integrated cockpit according to an embodiment of the present invention;

[0016] Figure 2 An apparatus for searching using multiple displays according to an embodiment of the present invention is shown;

[0017] Figure 3 A global menu according to an embodiment of the present invention is shown;

[0018] Figure 4 A brief document illustrating an embodiment of the present invention is shown;

[0019] Figure 5 The process of modifying a profile according to an embodiment of the present invention is illustrated;

[0020] Figure 6 (a) and (b) respectively show the search pre-screen and search results screen according to an embodiment of the present invention;

[0021] Figure 7 (a) and (b) respectively show the search pre-search screen and search results screen of the lower display according to an embodiment of the present invention during navigation operation on the upper display;

[0022] Figures 8 to 10 An integrated search process according to an embodiment of the present invention is illustrated;

[0023] Figure 11 and Figure 12 An integrated search in passenger mode according to an embodiment of the present invention is shown;

[0024] Figure 13 An apparatus for controlling and monitoring a forward-facing camera image and a panoramic view using multiple displays according to an embodiment of the present invention is shown;

[0025] Figure 14 The upper and lower display screens in default mode are shown according to an embodiment of the present invention;

[0026] Figure 15 An image display considering image size is shown according to an embodiment of the present invention;

[0027] Figure 16 and Figure 17 A forward image display according to an embodiment of the present invention is shown;

[0028] Figure 18 and Figure 19 A surround-view monitoring screen display according to an embodiment of the present invention is shown;

[0029] Figure 20 A navigation control device using multiple displays according to an embodiment of the present invention is shown;

[0030] Figure 21 A navigation display prior to driving is shown according to an embodiment of the present invention;

[0031] Figure 22 A navigation display during driving is shown according to an embodiment of the present invention;

[0032] Figure 23 The operation of using the lower display according to an embodiment of the present invention is illustrated;

[0033] Figure 24 The operation of using the upper display according to an embodiment of the present invention is illustrated;

[0034] Figure 25 A navigation setup using an upper display is shown according to an embodiment of the present invention;

[0035] Figure 26 and Figure 27 This illustrates a scenario where navigation is being performed while the passenger mode changes, according to an embodiment of the present invention.

[0036] Figure 28This illustrates a scenario where, according to an embodiment of the invention, navigation is not performed based on a change in passenger mode;

[0037] Figure 29 The process of searching for a destination using the lower display according to an embodiment of the present invention is illustrated;

[0038] Figure 30 The process of searching for a destination using an upper display according to an embodiment of the present invention is illustrated;

[0039] Figure 31 A three-dimensional (3D) navigation and augmented reality (AR) navigation setup according to an embodiment of the present invention is shown;

[0040] Figure 32 Navigation control in passenger mode is illustrated according to an embodiment of the present invention;

[0041] Figure 33 A media audio multi-connection control device using multiple displays is shown according to an embodiment of the present invention;

[0042] Figure 34 Media multi-connectivity in driver mode and passenger mode is illustrated according to an embodiment of the present invention;

[0043] Figure 35 Bluetooth audio reproduction according to an embodiment of the present invention is shown.

[0044] Figure 36 A list of Bluetooth audio files according to an embodiment of the present invention is shown;

[0045] Figure 37 The Bluetooth audio reproduction process according to an embodiment of the present invention is illustrated.

[0046] Figure 38 This illustrates Universal Serial Bus (USB) music reproduction according to an embodiment of the present invention;

[0047] Figure 39 A USB music playback list is shown according to an embodiment of the present invention;

[0048] Figure 40 The USB music playback process according to an embodiment of the present invention is illustrated;

[0049] Figure 41 USB video playback according to an embodiment of the present invention is shown;

[0050] Figure 42 A list of USB video playbacks according to an embodiment of the present invention is shown;

[0051] Figure 43This illustrates video movement between displays according to an embodiment of the present invention;

[0052] Figure 44 The knob operation process according to an embodiment of the present invention is shown;

[0053] Figure 45 The simultaneous video playback process according to an embodiment of the present invention is illustrated.

[0054] Figure 46 An apparatus for providing meditation functionality using multiple displays according to an embodiment of the present invention is shown;

[0055] Figure 47 A screen showing meditation execution in driver mode according to an embodiment of the present invention is shown;

[0056] Figure 48 A meditation execution screen in passenger mode is shown according to an embodiment of the present invention;

[0057] Figure 49 The process of providing the meditation function according to an embodiment of the present invention is illustrated;

[0058] Figure 50 A passenger mode meditation screen is shown according to an embodiment of the present invention;

[0059] Figure 51 The process of providing the meditation function in passenger mode according to an embodiment of the present invention is illustrated;

[0060] Figure 52 A passenger mode meditation multitasking screen according to an embodiment of the present invention is shown;

[0061] Figure 53 The process of providing the meditation function in passenger mode according to an embodiment of the present invention is illustrated;

[0062] Figure 54 The bubble formation when the passenger mode meditation function is provided according to an embodiment of the present invention is illustrated.

[0063] Figure 55 A control device for media delivery between a mobile device and an in-vehicle infotainment (IVI) system according to an embodiment of the present invention is shown;

[0064] Figure 56 The home screen of a smartphone application according to an embodiment of the present invention is shown;

[0065] Figure 57 A screen for a video player in a smartphone application according to an embodiment of the present invention is shown;

[0066] Figure 58A screen for a music player in a smartphone application according to an embodiment of the present invention is shown;

[0067] Figure 59 A controller screen is shown during smartphone application deployment according to an embodiment of the present invention;

[0068] Figure 60 This illustrates a change in content position according to an embodiment of the present invention;

[0069] Figure 61 The video casting process according to an embodiment of the present invention is illustrated;

[0070] Figure 62 The music delivery process according to an embodiment of the present invention is illustrated;

[0071] Figure 63 An apparatus for providing progressive warnings based on driver status, according to an embodiment of the present invention, is shown;

[0072] Figure 64 An implementation of Driver Status Warning (DSW) in driver mode according to an embodiment of the present invention is shown;

[0073] Figure 65 A DSW implementation in passenger mode according to an embodiment of the present invention is shown;

[0074] Figure 66 The step-by-step warning delivery process in driver mode according to an embodiment of the present invention is illustrated; and

[0075] Figure 67 A step-by-step warning delivery process in passenger mode according to an embodiment of the present invention is illustrated. Specific Implementation

[0076] The advantages and features of the present invention, as well as the methods of implementing them, will become clear from the embodiments described in detail below with reference to the accompanying drawings.

[0077] However, the present invention can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, embodiments are provided to fully explain the invention and to provide a full understanding of its scope by those skilled in the art. The scope of the invention is defined by the appended claims.

[0078] Furthermore, the terminology used herein is for the purpose of describing embodiments and is not intended to limit the invention. As used herein, the singular form includes the plural form as well as the singular form unless the context clearly indicates otherwise. The terms “comprising” or “including” as used herein do not exclude the presence or addition of one or more other elements, steps, operations, and / or devices besides those described.

[0079] Figure 1 This is a concept diagram of an integrated cockpit according to an embodiment of the present invention.

[0080] According to an embodiment of the invention, a 12.3-inch upper display and a controller integrated with a 13.2-inch monitor are housed within an integrated cockpit.

[0081] The 12.3-inch upper display can be operated in passenger mode, and the controller integrated with the 13.2-inch monitor is connected to the following: instrument panel, driver status warning unit (DSW), loudspeaker, 12.3-inch upper display, three-dimensional (3D) pattern lighting unit, headrest speakers, surround view module (SVM), rotary operating system, vents, gradient lighting unit, and camera monitoring system (CMS) camera.

[0082] In the following text, it will be explained by Figure 1 The 12.3-inch monitor is named the upper monitor. Figure 1 The controller integrated with the 13.2-inch monitor is referred to as the lower display to describe an exemplary embodiment of the invention.

[0083] Figure 2 An apparatus for searching using multiple displays according to an embodiment of the present invention is shown.

[0084] According to an embodiment of the present invention, an apparatus for performing a search using multiple displays includes: an input unit 110 configured to receive an integrated search command signal; a memory 120 storing a program for controlling the display of an integrated search function on at least one of an upper display and a lower display according to the integrated search command signal; and a processor 130 configured to execute the program. The processor 130 controls the display of the integrated search window for at least one of the lower display disposed in the cockpit and the upper display disposed above the lower display.

[0085] The processor 130 uses selection information about the user profile to control the display of search results for each menu corresponding to the user profile.

[0086] When an integrated search request is received, the processor 130 controls the display of a list of items in the menu according to the most recently used order of each menu item in the user profile.

[0087] When an integrated search request is received and a search term is entered, the processor 130 controls the display of a list corresponding to the search term in a predetermined order.

[0088] The processor 130 controls the arrangement and display of a list of each predetermined menu on the upper display, and displays information on the high frequency of use of each predetermined menu on the lower display.

[0089] The processor 130 controls the display on the upper display to show information based on the most recently used order of each predefined menu item, and when a search term is entered on the lower display, the processor 130 controls the display on the upper display to show a list corresponding to the search term in a predefined order.

[0090] When an application function is being executed on the upper display, the processor 130 controls the use of the lower display to show the integrated search window.

[0091] The processor 130 controls the display to show the integrated search window when passenger mode is being executed.

[0092] Figure 3 A global menu according to an embodiment of the present invention is shown.

[0093] The upper display shows the temperature, a phrase (e.g., Life Is Good), and the current time (e.g., 11:23).

[0094] The content displayed on the upper monitor is not limited to Figure 3 A and Figure 3 The content shown in B.

[0095] refer to Figure 3 A and Figure 3 B. When the user taps or drags a specific area on the lower display, the existing navigation information display area and media display area are darkened or dimmed, and the global menu is displayed.

[0096] Figure 3 Each configuration of the global menu shown in B is defined as shown in Table 1 below.

[0097] [Table 1]

[0098]

[0099]

[0100] Figure 4 A brief document illustrating an embodiment of the present invention is shown.

[0101] refer to Figure 4 A. Each user's profile picture and name are displayed in the user profile on the lower display, and guest and profile addition buttons are provided according to the left / right scrolling area.

[0102] Figure 4 B illustrates a scenario where up to six user profiles are displayed on the lower monitor's screen, while Figure 4 C shows the case where the profile list is arranged in the center of the pop-up window.

[0103] Figure 4 Each configuration in the user profile shown is defined as shown in Table 2 below.

[0104] [Table 2]

[0105]

[0106]

[0107] Figure 5 The process of modifying a profile according to an embodiment of the present invention is illustrated.

[0108] When the profile for "Eric" is selected from the global menu on the lower display, the user profile screen is displayed, and the screen with content focused on "Eric" is shown.

[0109] Here, when the user selects "Aaron", the user changes to "Aaron".

[0110] When the profile for the name "Aaron" is selected from the global menu on the lower display, the user profile screen is displayed, and the screen displaying content focuses on "Aaron".

[0111] Here, when the user selects "Guest", the user is changed to "Guest".

[0112] Based on the selections in the user profile, display the profile image and name corresponding to the changed profile.

[0113] Figure 6 (a) and (b) respectively show the search pre-search screen and the search results screen according to an embodiment of the present invention.

[0114] Figure 6 (a) shows the case where media, contacts, and destinations are arranged in the most recent order before the search is performed, while Figure 6 (b) shows the media, contacts, and destinations arranged alphabetically based on the search results.

[0115] In the integrated cockpit according to an embodiment of the present invention, a widget can be moved from the lower display to the upper display, or from the upper display to the lower display.

[0116] When performing an integrated search, the integrated search screen is displayed on both the upper and lower displays.

[0117] refer to Figure 6 In (a), before a search is performed, each of the media, contacts, and destinations on the upper display is sorted and displayed in the order in which they were most recently selected, contacted, or set, and the media, contacts, and destinations that are searched most frequently are displayed at the top of the lower display.

[0118] refer to Figure 6 (b) When “a” is entered into the search window, media, contacts and destinations including “a” are arranged alphabetically and displayed as search results.

[0119] At this point, you could consider using both frequency and alphabetical order to display media, contacts, and destinations separately.

[0120] Figure 6 The configuration of the integrated search screen shown is defined in Table 3 below.

[0121] [Table 3]

[0122]

[0123]

[0124] Figure 7 (a) and (b) respectively illustrate the pre-search screen and search results screen of the lower display during navigation operations on the upper display according to an embodiment of the present invention.

[0125] Navigation information is displayed on the upper display, while the integrated search screen is displayed on the lower display.

[0126] Figure 7 (a) shows the case where media, contacts, and destinations are arranged in the most recent order before the search is performed, while Figure 7 (b) shows the media, contacts, and destinations arranged alphabetically based on the search results.

[0127] refer to Figure 7 (a) Before a search is performed, in each of the media, contacts and destinations on the lower display, the media, contacts and destinations are arranged and displayed in the order in which they were most recently selected, contacted or set.

[0128] refer to Figure 7 (b) When “a” is entered into the search window, media, contacts and destinations including “a” are arranged alphabetically and displayed as search results.

[0129] At this point, you could consider using both frequency and alphabetical order to display media, contacts, and destinations separately.

[0130] Figure 7 The configuration of the integrated search screen shown is defined in Table 4 below.

[0131] [Table 4]

[0132]

[0133] Figures 8 to 10 An integrated search process according to an embodiment of the present invention is illustrated.

[0134] refer to Figure 8 When you move to the integrated search, an integrated search pop-up window is provided, displaying a list of recently used items for each of media, contacts, and destinations, and displaying search results for each of media, contacts, and destinations as search terms are entered into the search window.

[0135] As described above, when using an integrated search screen with an upper and lower display, media, contacts, and destinations are displayed on the upper display in the order of most recently used or alphabetical order, and frequently searched search term buttons can be displayed on the lower display.

[0136] When a specific media is selected from the integrated search screen, the corresponding media is displayed in the media widget. When a contact is selected, the option to call or message the contact is available. When a specific destination is selected, the navigation widget displays a preview screen showing the route to that destination.

[0137] When the application is executed on the search results screen of the lower display, it is executed after the search and the corresponding search results. When the application is not executed on the search results screen, navigation is executed on the far left. When navigation is not on the screen, media or phone widgets are executed on the far left, and when navigation is on the screen, media or phone widgets are executed in the position above and to the right of navigation.

[0138] refer to Figure 9When a search icon is selected from the media widget on the upper or lower display, or from the navigation widget on the upper or lower display, or from the phone widget on the lower display, the integrated search window is invoked. Here, as described above, in the integrated search window, media, contacts, and destinations are displayed in the order of most recent use before a search is performed, and when a search term is entered, relevant media, contacts, and destinations are displayed in alphabetical or Korean alphabetical order.

[0139] refer to Figure 10 Navigation information is displayed on the upper display, and continues to be displayed on the upper display when the "Search" icon is selected from the navigation widget on the lower display. The integrated search window is only invoked on the lower display (i.e., the upper display is not used to display the integrated search window).

[0140] When you select a destination by entering a search term in the search window on the lower display, summary information about the route to the selected destination is displayed in the navigation widget on the lower display.

[0141] When a starting selection is entered in the route summary information displayed on the navigation widget of the lower display, navigation information is displayed on the upper display, and route guidance information is also displayed on the navigation widget of the lower display.

[0142] Figure 11 and Figure 12 An integrated search in passenger mode is shown according to an embodiment of the present invention.

[0143] refer to Figure 11 In passenger mode, media and destinations are arranged and displayed in the order of most recent use before a search is performed, and after a search is performed, the search results for the media and destinations are arranged and displayed in alphabetical order.

[0144] Figure 11 The configuration for integrated search in passenger mode, as shown in Table 5, is defined below.

[0145] [Table 5]

[0146]

[0147]

[0148] refer to Figure 12In passenger mode, when an icon related to "search" is selected from the navigation widget on the upper display, when the icon related to "search" is selected from the navigation widget while multitasking with other applications, or when the icon related to "search" is selected from the media widget, an integrated search window is displayed on the upper display, showing the selection history of media and destinations in recent order, and the search results of media and destinations in alphabetical order based on the entered search terms.

[0149] In the search results of the integrated search window on the upper display, when a media selection is entered, the corresponding media is reproduced, and when a specific destination selection is entered, a preview screen of the navigation path to the corresponding destination is displayed.

[0150] According to relevant technologies, images from a forward-facing camera are used to provide information, and surround-view monitoring devices are used to provide information about targets around the vehicle. However, there is a problem: images captured in a limited area are displayed on a monitor with a fixed area.

[0151] To address the aforementioned problems, the present invention aims to provide an apparatus that enables a driver to intuitively identify images obtained based on the image capture angle using multiple displays, and to provide a surround-view monitoring screen that adapts to display settings and driving conditions.

[0152] Figure 13 An apparatus for controlling and monitoring a forward-facing camera image and a panoramic view using multiple displays, according to an embodiment of the present invention, is shown.

[0153] According to an embodiment of the present invention, an apparatus for controlling the use of multiple displays to monitor a forward-facing camera image and a surround-view image includes: an input unit 210 configured to receive a vehicle surrounding confirmation request signal; a memory 220 storing a program for controlling the display of a forward-facing image and a surround-view monitoring image upon receiving the vehicle surrounding confirmation request signal; and a processor 230 configured to execute the program. The processor 230 controls the display of images using at least one of a lower display disposed in the cockpit and an upper display disposed above the lower display, and controls the adjustment of the image capture angle or image display area according to the operation of a knob.

[0154] When displaying a forward-facing image, the processor 230 adjusts the image capture angle facing the camera or the display area of ​​the acquired forward-facing image in response to a knob angle operation within a preset range.

[0155] The processor 230 controls the display of images acquired by the under-vehicle recognition camera in response to knob angle operations that deviate from a preset range.

[0156] The processor 230 controls the overlay and display of at least one of the guide lines for the total width of the vehicle and the wheel positions of the vehicle on the image.

[0157] In driver mode, the processor 230 controls the display of the surround-view surveillance image on the upper display, while in passenger mode, it controls the display of the surround-view surveillance image on the lower display.

[0158] The processor 230 controls the display of the surround-view surveillance image on the upper or lower display based on a request to display the surround-view surveillance image or by shifting into reverse "R".

[0159] When the knob is moved to the predetermined area, the processor 230 controls the pop-up screen to be invoked and waits for a request to display an image.

[0160] Figure 14 The upper and lower display screens in default mode are shown according to an embodiment of the present invention.

[0161] Images captured by facing the camera forward are displayed on the upper monitor, while gadgets are displayed on the lower monitor.

[0162] On the upper display, a warning phrase, the angle information of the camera that acquired the current image, the target detection information, and the total width information of the vehicle are displayed in an overlay format.

[0163] Figure 14 The configuration of the upper display facing the camera screen is shown in Table 6 below.

[0164] [Table 6]

[0165]

[0166]

[0167] Figure 15 An image display considering image size is shown according to an embodiment of the present invention.

[0168] According to an embodiment of the present invention, the upper display area is 1920×720 and the lower display area is 1920×1080.

[0169] When the image acquired by the camera has a resolution of 1620×1250, the image is enlarged according to the horizontal width of the monitor, which is 1920.

[0170] like Figure 15 As shown, some images can be displayed on one of the upper and lower displays.

[0171] like Figure 15As shown, an image corresponding to the entire image size can be displayed across the areas of the upper and lower displays.

[0172] Based on the knob operation information, you can adjust which part of the image captured by the camera is displayed.

[0173] For example, such as Figure 15 As shown, when the knob is adjusted clockwise to a predetermined angle, the lower part of the corresponding acquired image can be adjusted to be displayed on the upper display.

[0174] Figure 16 and Figure 17 A forward-facing image display is shown according to an embodiment of the present invention.

[0175] refer to Figure 16 Select the "Face Forward" icon from the lower display to request that the image facing forward be displayed on the upper display.

[0176] When facing the camera to display the image, adjust the knob angle to adjust the capture angle of facing the camera in real time according to the angle.

[0177] When the knob is rotated beyond a predetermined range, the camera used to identify the underside of the vehicle is activated and an image of the underside of the vehicle is displayed on the upper display.

[0178] In this way, the driver can visually identify obstacles under the vehicle.

[0179] When displaying images obtained by a forward-facing camera and a recognition camera below the vehicle, at least one of guide lines of total width and wheel positions is superimposed on the image, and the superimposed image is displayed, thereby enabling the driver to easily avoid obstacles.

[0180] When the knob is pressed while the image of the camera is being displayed on the upper display, the image display of the camera is terminated.

[0181] refer to Figure 17 When the knob is moved, a pop-up screen is brought up.

[0182] At this point, if there is no operation within a predetermined time (e.g., 10 seconds) after the pop-up screen is invoked, the pop-up screen disappears.

[0183] When the camera is focused on the pop-up screen and the knob is pressed, the image captured by the camera is displayed on the upper display.

[0184] As described above, when adjusting the knob angle, the selected display area is the image area for the camera to capture, or the capture angle for the camera to capture is adjusted.

[0185] When the operating knob exceeds a predetermined range, an image acquired by a recognition camera under the vehicle is displayed on the upper display, and at least one of the guidance information about the total width of the vehicle and the guidance information about the wheel position is superimposed on the image, and the superimposed image is displayed.

[0186] Figure 18 and Figure 19 A surround-view monitoring screen display according to an embodiment of the present invention is shown.

[0187] In driver mode, gadget information is displayed on the lower display, while the SVM screen is displayed on the upper display.

[0188] In passenger mode, the upper display is linked to the passenger's mobile phone, and the SVM screen is displayed on the lower display.

[0189] When in reverse (R), the SVM screen will automatically display on either the upper or lower monitor.

[0190] The warning phrases and vehicle top-view images shown in Table 7 are displayed on the SVM screen.

[0191] [Table 7]

[0192]

[0193] refer to Figure 18 When a request to display the SVM screen is received, or when reverse gear R is engaged, the SVM screen is executed in a global pop-up manner.

[0194] When reverse gear R is released, the upper display returns to the previous screen.

[0195] In passenger mode, the SVM screen is displayed on the lower display when a request to display the SVM screen is received or when reverse gear R is engaged.

[0196] When reverse gear R is released, or when a signal is received by touching any point in the entire area of ​​the lower display that displays the SVM screen, the lower display returns to the previous screen.

[0197] According to relevant technologies, there is a limitation that navigation information can only be displayed in a limited audio-visual navigation (AVN) display area.

[0198] To address the aforementioned problems, the present invention aims to provide a navigation control device that enables drivers to easily check route guidance information by controlling navigation according to the situation using multiple displays.

[0199] Figure 20 A navigation control device using multiple displays according to an embodiment of the present invention is shown.

[0200] A navigation control device using multiple displays according to an embodiment of the present invention includes: an input unit 310 configured to receive a navigation request signal; a memory 320 storing a program for controlling the display of navigation information using multiple displays upon receiving the navigation request signal; and a processor 330 configured to execute the program. The processor 330 controls the display of navigation information using at least one of a lower display disposed in the cockpit and an upper display disposed above the lower display.

[0201] When the processor 330 controls the display of navigation information on the lower and upper displays, the processor 330 controls the search and menu buttons not to be exposed in other areas when the navigation application is operated in either the lower or upper display.

[0202] The processor 330 uses the response information from the pop-up window call to display a 3D driving image or an augmented reality (AR) driving image on the upper display.

[0203] When navigation information is displayed on both the lower and upper displays, and the mode is changed to passenger mode, the processor 330 changes the upper display to an area for passenger content viewing and maintains the display of navigation information on the lower display.

[0204] When the navigation widget is operating in an area outside the lower display screen, the processor 330 moves the navigation widget to an area within the lower display screen when the mode changes to passenger mode.

[0205] When setting a destination, the processor 330 provides 3D image information and a destination confirmation function using rotation to support route guidance settings for specific points at the destination.

[0206] In passenger mode, when a navigation application is executed via the upper display and a request is received to send route guidance information to the driver who has set destination information, the processor 330 controls the provision of route guidance information via the lower display.

[0207] Figure 21 A navigation display before driving is shown according to an embodiment of the present invention.

[0208] Before driving, a map image is displayed on the upper display, and when a touch event occurs, a search button, zoom in / out button, and menu button are displayed.

[0209] The map image is displayed on the lower monitor, and the search, zoom in / out, and menu buttons are always visible.

[0210] Figure 21The configuration and definitions of the navigation display before driving are shown in Table 8 below.

[0211] [Table 8]

[0212]

[0213] Figure 22 A navigation display during driving is shown according to an embodiment of the present invention.

[0214] During driving, a 3D driving image is displayed on the upper display, and when a touch event occurs, a search button, a zoom-in / zoom-out button, and an end button are displayed.

[0215] During driving, a 2D driving image is displayed on the lower display, with search, zoom in / out, and end buttons always visible.

[0216] Figure 22 The configuration and definition of the navigation display during driving are shown in Table 9 below.

[0217] [Table 9]

[0218]

[0219] Figure 23 The operation of the display using an embodiment of the present invention is illustrated, while Figure 24 The operation of using the upper display according to an embodiment of the present invention is illustrated.

[0220] refer to Figure 23 When the navigation application is operated on the lower display, the search / menu button is not displayed on the upper display.

[0221] refer to Figure 24 When the navigation application is operated on the upper display, the search / menu button is not displayed on the lower display.

[0222] Figure 23 and Figure 24 The configuration and definition of the navigation widget screen shown are shown in Table 10 below.

[0223] [Table 10]

[0224]

[0225] Figure 25 Navigation settings using an upper display according to an embodiment of the present invention are shown.

[0226] When navigation information is displayed on the upper display, a 3D map image of the destination is displayed when the destination is set.

[0227] On the upper display that shows a 3D map image, users can check 3D information by rotating the building. As a result of the check, even for the same destination, users can set the point to reach (for example, the destination can be set as the entrance to the underground parking lot at the front of the department store, the entrance to the outdoor parking lot at the back of the department store, etc.).

[0228] When route setup is complete while checking the destination, a route preview screen is provided, where the destination name, estimated time / distance required, estimated remaining battery power upon arrival at the destination, route guidance start button, and button for returning to the previous screen are displayed on the upper display.

[0229] When the route guidance start button is pressed, the upper display shows a 3D route guidance screen, and when the end button is pressed on the corresponding screen, the upper display returns to the destination setting screen.

[0230] Figure 25 The configuration and definition of the navigation widget screen shown are shown in Table 11 below.

[0231] [Table 11]

[0232]

[0233]

[0234] Figure 26 and Figure 27 This illustrates a scenario where navigation is being performed while the passenger mode changes, according to an embodiment of the present invention.

[0235] refer to Figure 26 When navigation is being performed on both the upper and lower displays, navigation guidance on the upper display ends when the mode is changed to passenger mode and the upper display is used for content intended for passenger use, and route guidance information is provided via the lower display.

[0236] When the mode returns from passenger mode to driver mode, the upper display resumes the previously executed navigation screen display.

[0237] refer to Figure 27 While navigation is being performed on the upper display, the navigation widget is being performed in an area outside the screen of the lower display.

[0238] At this point, when the mode is changed to passenger mode, the navigation guidance on the upper display ends, and the navigation widget on the lower display moves to the far left.

[0239] When the mode returns from passenger mode to driver mode, the upper display resumes the previously executed navigation screen display.

[0240] Figure 28 This illustration shows a scenario where, according to an embodiment of the invention, navigation is not performed based on a change in passenger mode.

[0241] When navigation is not performed on the upper and lower displays, the widget arrangement on the lower display is maintained, even when the mode is changed to passenger mode.

[0242] When the mode returns from passenger mode to driver mode, the upper display resumes the previously executed wallpaper display.

[0243] Figure 29 The following describes a destination search process using a lower display according to an embodiment of the present invention.

[0244] When the default screen is displayed on the upper display, and navigation is dragged from the lower display to the upper display, navigation information is displayed on the upper display.

[0245] At this time, when a touch is input to the area of ​​the upper display, information about search, zoom in / out and menus is displayed, and when "1" is selected after making a menu selection, 3D map image information about the destination is provided according to the execution of the predetermined function.

[0246] Vertical linkage based on pinching in, pinching out, and translation is not allowed.

[0247] Because 3D map image information of the destination is provided on the upper display before driving begins, the driver can use the 3D map image information to register the desired point as the destination.

[0248] Based on the destination registration, the upper display shows route information, and when the route guidance start button is pressed, the upper display shows 3D navigation information, while the lower display shows 2D navigation information.

[0249] Figure 30 A destination search process using an upper display is illustrated according to an embodiment of the present invention.

[0250] When the default screen is displayed on the upper display, and navigation is dragged from the lower display to the upper display, navigation information is displayed on the upper display.

[0251] At this time, when a touch is input to the area of ​​the upper display, information about search, zoom in / out and menus is displayed, and when "1" is selected after making a menu selection, 3D map image information about the destination is provided according to the execution of the predetermined function.

[0252] Because 3D map image information of the destination is provided on the upper display before driving begins, the driver can use the 3D map image information to register the desired point as the destination.

[0253] Vertical linkage based on pinching in, pinching out, and translation is not allowed.

[0254] When a destination is registered, the upper display shows route information, and when the route guidance start button is pressed, the upper display shows 3D navigation information, while the lower display shows 2D navigation information.

[0255] Figure 31 The 3D navigation and AR navigation settings according to an embodiment of the present invention are shown.

[0256] When the menu button and settings button of the upper or lower display are pressed in sequence, the navigation settings window is displayed to receive the selection of 3D navigation or AR navigation.

[0257] Figure 31 The navigation display during driving is shown based on 3D navigation and AR navigation.

[0258] Figure 32 Navigation control in passenger mode is shown according to an embodiment of the present invention.

[0259] When a navigation application is selected to be executed on a user mobile device linked to the upper display, 2D image information and 3D image information about the destination are displayed on the upper display screen.

[0260] Passengers rotate the display on the screen above to request route guidance to their final destination.

[0261] In passenger mode, the upper display shows directions to the destination and a "Drop" button.

[0262] When the user presses the "Deliver" button, a pop-up window appears asking whether to send route guidance information to the driver.

[0263] When the user presses the "Yes" button, route guidance information is displayed in a 2D driving image on the navigation application on the lower display.

[0264] According to relevant technologies, there are limitations to simply using AVN content in vehicles.

[0265] To address the aforementioned issues, this invention aims to provide a media audio multi-connection control device using multiple displays. This device is capable of simultaneously reproducing different media and selecting audio source output devices via AVN and mobile devices, simultaneously reproducing media via multiple displays, and generating sound bubbles to prevent sound source interference between vehicle speakers and headrest speakers.

[0266] Figure 33A media audio multi-connection control device using multiple displays is shown according to an embodiment of the present invention.

[0267] A media and audio multi-connection control device using multiple displays according to an embodiment of the present invention includes: an input unit 310 configured to receive a content execution request signal; a memory 320 storing a program for controlling media and audio multi-connection using multiple displays upon receiving the content execution request signal; and a processor 330 configured to execute the program. The processor 330 controls the display of a player screen using at least one of a lower display disposed in the cockpit and an upper display disposed above the lower display, in accordance with the content execution request signal.

[0268] The processor 430 controls the output of the sound source to the vehicle speakers and headrest speakers using the sound path setting signal.

[0269] When a request is entered to move a media application from the lower display to the upper display, the processor 430 controls the display of the player screen on the upper display and the list screen on the lower display.

[0270] When the drop-down button is touched on the upper display, the processor 430 moves the player screen to the lower display and controls it to maintain sound source reproduction during the movement.

[0271] When the full-screen image playback button is touched on the lower display, the processor 430 controls the upper display to turn off and controls the lower display to reproduce the entire screen.

[0272] When a command is entered to move the entire screen from the lower display to the upper display, the processor 430 controls the reproduction of the entire screen on the upper display, and controls the display of the existing gadgets on the lower display and the display close button on the lower display.

[0273] When the touch signal is applied to the display off button on the lower display, the processor 430 turns off the lower display. When a touch signal is applied to the area of ​​the lower display, the processor 430 controls the upper display to turn off and displays the entire screen on the lower display.

[0274] When the knob located at the bottom of the lower display is operated, or when a touch input is made to the lower display, the processor 430 controls the display of the function bar on the lower display.

[0275] When different content is reproduced on the lower and upper displays in passenger mode, the processor 430 controls the output of the sound source of the content to be reproduced on the upper display through the headrest speaker, and the processor 430 controls the output of the sound source of the content to be reproduced on the lower display through the vehicle speaker.

[0276] Figure 34 Media multi-connectivity in driver mode and passenger mode is illustrated according to an embodiment of the present invention.

[0277] According to an embodiment of the present invention, Universal Serial Bus (USB) music and video can be reproduced simultaneously on the upper and lower displays, but the same files cannot be reproduced.

[0278] On the upper or lower display, while reproducing USB music and USB video, Bluetooth audio and video from a smartphone can be reproduced simultaneously.

[0279] The Bluetooth audio and video of a smartphone can be reproduced simultaneously on the upper and lower displays, but the same file cannot be reproduced.

[0280] The user receives input via the sound path setting button on the lower display to set the sound source output to the vehicle speakers or headrest speakers.

[0281] The user receives input via the sound path setting button on the smartphone to set the sound source to be output to the vehicle speakers or headrest speakers.

[0282] Figure 35 Bluetooth audio reproduction according to an embodiment of the present invention is shown.

[0283] Figure 35 The images show the scenarios where Bluetooth audio is reproduced on the lower display and the upper display.

[0284] Figure 35 The configurations and functions in the Bluetooth audio screen shown are summarized in Table 12 below.

[0285] [Table 12]

[0286]

[0287]

[0288] Figure 36 A list of Bluetooth audio files according to an embodiment of the present invention is shown.

[0289] The Bluetooth audio list includes a button to return to the previous screen, an area to display information about the Bluetooth-connected device, and a song information area.

[0290] The sound path setting button is used to set the sound path to the car speakers or headrest speakers.

[0291] Figure 36 The configurations and functions shown in the Bluetooth audio list screen are summarized in Table 13 below.

[0292] [Table 13]

[0293]

[0294] Figure 37 The Bluetooth audio reproduction process according to an embodiment of the present invention is illustrated.

[0295] The default screen is displayed on the upper monitor, while navigation and media information are displayed on the lower monitor.

[0296] When you press the sound path setting button on the lower display, you can change the settings to switch the path to the car speakers or headrest speakers.

[0297] When an input action moves a media application from the lower display to the upper display, the media player application is displayed on the upper display, and the screen changes to a list screen on the lower display.

[0298] On the lower display, when the connected device changes to another Bluetooth-connected device, a list of other Bluetooth-connected devices is displayed.

[0299] When you press the sound path setting button on the upper display, you can change the settings to change the path to the car speakers or headrest speakers.

[0300] When the drop-down button on the upper display is pressed, the media player application is moved to the lower display, and the sound source is reproduced while the player moves.

[0301] Figure 38 A USB music playback method according to an embodiment of the present invention is shown.

[0302] Figure 38 This demonstrates two scenarios: displaying the default screen on the upper display and reproducing USB music on the lower display, and reproducing USB music on the upper display and displaying a USB music list on the lower display.

[0303] Figure 38 The configuration and functions of the USB music playback screen shown are illustrated in Table 14 below.

[0304] [Table 14]

[0305]

[0306]

[0307] Figure 39 A USB music playback list is shown according to an embodiment of the present invention.

[0308] Figure 39 The configurations and functions in the USB music playback list screen shown are summarized in Table 15 below.

[0309] [Table 15]

[0310]

[0311] Figure 40 The USB music playback process according to an embodiment of the present invention is illustrated.

[0312] The default screen is displayed on the upper monitor, while navigation and media information are displayed on the lower monitor.

[0313] When you press the sound path setting button on the lower display, you can change the settings to switch the path to the car speakers or headrest speakers.

[0314] When an input action moves a media application from the lower display to the upper display, the media player application is displayed on the upper display, and the screen changes to a list screen on the lower display.

[0315] When the drop-down button on the upper display is pressed, the media player application is moved to the lower display, and the sound source is reproduced while the player moves.

[0316] When you press the sound path setting button on the upper display, you can change the settings to change the path to the car speakers or headrest speakers.

[0317] Figure 41 A USB video playback according to an embodiment of the present invention is shown.

[0318] Figure 41 This illustrates scenarios where the default screen is displayed on the upper monitor and the video is reproduced on the lower monitor, as well as scenarios where the video is reproduced on the upper monitor and video-related information is displayed on the lower monitor.

[0319] Figure 41 The configuration and functions of the video playback screen shown are summarized in Table 16 below.

[0320] [Table 16]

[0321]

[0322]

[0323] Figure 42 A list of USB video playbacks according to an embodiment of the present invention is shown.

[0324] Figure 42The configurations and features in the USB video playback list shown are summarized in Table 17 below.

[0325] [Table 17]

[0326]

[0327] Figure 43 The illustration shows video movement between displays according to an embodiment of the present invention.

[0328] Imagine a scenario where the default screen is displayed on the upper monitor, while navigation and media information are displayed on the lower monitor.

[0329] When the full-screen mode button on the lower monitor is touched, the upper monitor turns off, and the entire screen is displayed on the lower monitor.

[0330] When the button used to move the image from the lower display to the upper display is touched, the entire screen is reproduced on the upper display, while navigation and media information are displayed on the lower display.

[0331] When the display off button is touched in the media information display area of ​​the lower display, the lower display is turned off. When an area of ​​the already turned-off lower display is touched, the upper display turns off and the entire screen is displayed on the lower display.

[0332] Figure 44 The knob operation process according to an embodiment of the present invention is illustrated.

[0333] When the default screen is displayed on the upper display and navigation and media information is displayed on the lower display, touching the full-screen mode button on the lower display turns off the upper display and the entire screen is reproduced on the lower display.

[0334] When the knob is operated or the screen is touched on the lower display, a function bar appears and is displayed on the lower display.

[0335] Figure 45 The simultaneous video playback process according to an embodiment of the present invention is illustrated.

[0336] In passenger mode, the upper display is linked to the passenger's mobile device, and navigation and media information are displayed on the lower display.

[0337] When the full-screen mode button on the lower display is touched, the entire screen is displayed on the lower display, and the upper display is linked with the passenger's mobile device to play the content selected on the passenger's mobile device.

[0338] In passenger mode, the media sound source reproduced on the lower display is output through amplifying sound (in-vehicle speakers), while the media sound source reproduced on the upper display is output through headrest speakers.

[0339] In passenger mode, each speaker forms a sound bubble to prevent the sound sources provided to the driver and passengers from interfering with each other.

[0340] According to relevant technologies, there are limitations in providing only the function of adjusting the color of ambient light in the vehicle, without providing a function that allows vehicle passengers to meditate.

[0341] To address the aforementioned problems, the present invention aims to provide a device that can provide a function that allows vehicle passengers to meditate by controlling multiple displays, speakers, and ambient lighting in a vehicle.

[0342] Figure 46 An apparatus for providing meditation functionality using multiple displays according to an embodiment of the present invention is shown.

[0343] An apparatus for providing meditation functionality using multiple displays according to an embodiment of the present invention includes: an input unit 510 configured to receive a meditation function request signal; a memory 520 storing a program for providing meditation functionality using multiple displays upon receiving the meditation function request signal; and a processor 530 configured to execute the program. The processor 530 provides meditation functionality using at least one of a lower display disposed in the cockpit and an upper display disposed above the lower display, in conjunction with a speaker and ambient lighting.

[0344] In driver mode, the processor 530 controls the lower display to turn off and displays the meditation application execution screen on the upper display.

[0345] When the knob is operated, the processor 530 controls the execution of event operations on the meditation application execution screen according to the direction of the knob operation.

[0346] In passenger mode, the processor 530 controls the display of the meditation application execution screen on the upper display.

[0347] The processor 530 controls the display of the meditation multitasking screen on the upper display, and controls the display of the media thumbnail screen when the application drawer button is touched and a meditation application is selected.

[0348] The processor 530 controls the sound source linked to the meditation application displayed on the upper display to be output through the passenger headrest speakers, and controls the sound source of the media to be output through the vehicle speakers, the execution screen of which is displayed on the lower display.

[0349] The processor 530 controls the headrest speakers and car speakers to form sound bubbles.

[0350] Figure 47 A meditation execution screen in driver mode is shown according to an embodiment of the present invention.

[0351] According to an embodiment of the present invention, the meditation application execution screen is displayed on the upper display, the lower display is turned off, and the knob display is also turned off.

[0352] The ambient lighting and loudspeakers are linked to the meditation app, allowing you to set their color and sound.

[0353] Figure 47 The configurations and definitions in the execution screen shown are described in Table 18 below.

[0354] [Table 18]

[0355]

[0356] Figure 48 A meditation execution screen in passenger mode is shown according to an embodiment of the present invention.

[0357] In passenger mode, the meditation application is executed on the upper display, while the lower display shows navigation information, media information, etc., as part of the regular screen operation, and regular operations are performed on the knob display.

[0358] The loudspeakers reproduce the sound source of the content displayed on the lower monitor, while the headrest speakers in the passenger seats reproduce the sound of the meditation application.

[0359] Figure 48 The configurations and definitions in the execution screen shown are described in Table 19 below.

[0360] [Table 19]

[0361]

[0362] Figure 49 The process of providing the meditation function according to an embodiment of the present invention is illustrated.

[0363] Assume the default screen is displayed on the upper monitor, and the meditation function is selected on the lower monitor.

[0364] When the meditation application is run, the sound and ambient light of each image are synchronized with the application, and when the knob is operated, touch / drag events are performed in random areas according to the direction of the knob operation.

[0365] When a touch signal is applied to the lower display, the end button is displayed. When the end button is touched, the default screen is displayed on the upper display, and the previously executed widget is displayed on the lower display.

[0366] Figure 50 A meditation screen in passenger mode is shown according to an embodiment of the present invention.

[0367] In passenger mode, the meditation screen displayed on the upper display shows a button for returning to the previous screen when touched, an area displaying the title, an app drawer button, a meditation thumbnail button, and a meditation icon.

[0368] Figure 50 The configurations and definitions in the screen shown are described in Table 20 below.

[0369] [Table 20]

[0370]

[0371] Figure 51 The process of providing the meditation function in passenger mode according to an embodiment of the present invention is illustrated.

[0372] In passenger mode, the upper display shows information about the linked passenger mobile devices.

[0373] When running the meditation application, it displays Figure 50 The meditation screen shown.

[0374] When the meditation app is used, the headrest speakers and ambient lights in the passenger seat are synchronized with the app's execution, thus setting their sound sources and ambient lighting.

[0375] Figure 52 A meditation multitasking screen in passenger mode is shown according to an embodiment of the present invention.

[0376] In the Meditation multitasking screen of Passenger Mode, the app drawer button provides the app drawer open / close button.

[0377] When you touch the meditation thumbnail button, the entire meditation screen is executed.

[0378] Figure 53 The process of providing the meditation function in passenger mode according to an embodiment of the present invention is illustrated.

[0379] When an application is running on the upper display in passenger mode, applications such as navigation applications, media applications, and meditation applications are displayed based on the touch of the application drawer button. Meditation functions are provided based on the execution of the meditation application and the touch of the meditation thumbnail button. In conjunction with the meditation application, the sound source is reproduced by the passenger seat headrest speaker and the ambient lighting is set.

[0380] Figure 54 The illustration shows bubble formation when providing a meditation function in passenger mode according to an embodiment of the present invention.

[0381] According to an embodiment of the present invention, when the display is linked with the passenger's mobile device in passenger mode to display the screen for executing a meditation application, the headrest speaker of the passenger seat outputs a sound source according to the execution of the meditation application.

[0382] The media audio source reproduced on the lower display is output through a loudspeaker.

[0383] In passenger mode, the headrest speakers and loudspeakers form sound bubbles to prevent the sound sources provided to the driver and passengers from interfering with each other.

[0384] Based on relevant technologies, techniques have been proposed to connect in-vehicle systems to mobile devices, enabling the viewing of media reproduced on mobile devices via the in-vehicle system. When in-vehicle displays are provided in various aspects, technologies related to media delivery between the in-vehicle infotainment (IVI) system and mobile devices are required.

[0385] To achieve the above objectives, the present invention aims to provide a control device for media delivery between a mobile device and an IVI system, the device being capable of: continuously reproducing media running on the mobile device; selecting a vehicle speaker or a headrest speaker as the output device in the IVI system; and preventing sound source interference between the vehicle speaker and the passenger headrest speaker.

[0386] Figure 55 A control device for media delivery between a mobile device and an IVI system according to an embodiment of the present invention is shown.

[0387] The control device for media delivery between an IVI system and a mobile device according to the present invention includes: an input unit 610 configured to receive a delivery request from the mobile device; a memory 620 storing a program for reproducing the content of the mobile device on an in-vehicle display according to the delivery request; and a processor 630 configured to execute the program. The processor 630 controls the delivery service to be provided using an upper display positioned above a lower display located in the cockpit.

[0388] When the mobile device interacts with the upper display according to the delivery request, the processor 630 controls the display to reproduce the content in response to the signal input through the controller screen displayed on the mobile device.

[0389] The processor 630 controls the sound path to be set to either the car speaker or the headrest speaker based on the input to the sound path setting button.

[0390] When the playback is successful, the processor 630 controls the display of the sound path setting button.

[0391] When the delivery is finished, the processor 630 controls the display to show the screen that was in use before the delivery on the upper display.

[0392] When the mode is passenger mode and a passenger screen application is being executed on the upper display, the processor 630 controls the direct execution of the corresponding content on the upper display based on the delivery request.

[0393] Figure 56 The home screen of a smartphone application according to an embodiment of the present invention is shown.

[0394] The smartphone application screen according to an embodiment of the present invention includes a title area, a menu button, a video tab, a music tab, thumbnails, a motion sickness relief button, a launch icon, and a mini player.

[0395] When the motion sickness relief button is touched, visual, auditory, and tactile feedback is provided to alleviate motion sickness.

[0396] The configuration and definition of the smartphone application screen are shown in Table 21 below.

[0397] [Table 21]

[0398]

[0399]

[0400] Figure 57 A screen for a video player in a smartphone application according to an embodiment of the present invention is shown.

[0401] The video player screen of the smartphone application includes a button for returning to the previous screen when touched, a video title, a play button, control buttons, and a progress bar, and adjusts the volume of the content being played when the mobile device's hardware volume buttons are operated.

[0402] When the cast button is touched, the content is displayed on the upper display via the cast function.

[0403] The configuration and definition of the video player screen for the smartphone application are shown in Table 22 below.

[0404] [Table 22]

[0405]

[0406] Figure 58 A screen for a music player in a smartphone application according to an embodiment of the present invention is shown.

[0407] The music player screen in the smartphone application includes a minimize button for minimizing to a mini player when touched, a sound source thumbnail, content information, a progress bar, control buttons, a volume control bar, a sound path setting button, and a cast button.

[0408] When the cast button is touched, the content is displayed on the upper display via the cast function.

[0409] The sound path setting button displays the currently set sound path (car speakers or headrest speakers), and when the button is touched, a sound path change pop-up window appears. When casting, the default sound path is the headrest speakers.

[0410] The configuration and definition of the music player screen in the smartphone application are shown in Table 23 below.

[0411] [Table 23]

[0412]

[0413]

[0414] Figure 59 A controller screen during smartphone application deployment is shown according to an embodiment of the present invention.

[0415] During smartphone application deployment, the controller screen includes a button to minimize to a mini player when touched, audio / video or sound source thumbnails, content information, a progress bar, control buttons, a volume control bar, a sound path setting button, and a deployment button.

[0416] When the cast button is touched, the content is displayed on the upper display via the cast function.

[0417] The sound path setting button displays the currently set sound path (car speakers or headrest speakers), and when the button is touched, a sound path change pop-up window appears. When casting, the default sound path is the headrest speakers.

[0418] During the deployment of the smartphone application, the configuration and definition of the controller screen are shown in Table 24 below.

[0419] [Table 24]

[0420]

[0421]

[0422] Figure 60 The changes in content position according to an embodiment of the present invention are illustrated.

[0423] When you touch the top-left button, the location used for searching content can be changed to mobile devices and vehicles.

[0424] Figure 61 The video casting process according to an embodiment of the present invention is illustrated.

[0425] The selected video on the mobile device is displayed on the upper display according to the casting request, and during casting, a minimize button, thumbnail, content information, progress bar, control buttons, volume control bar, sound path setting button and casting button are displayed on the controller screen.

[0426] When the minimize button is touched, a media thumbnail file is displayed on the video tab, and information about the currently playing video is displayed on the minimize screen.

[0427] When the delivery button is touched again, the linkage ends, and the screen that was displayed before receiving the delivery request is displayed on the upper display again.

[0428] At the end of the broadcast, the video is seamlessly reproduced on the mobile device. In passenger mode, when the broadcast ends, the screen on the upper display returns to the widget screen on the passenger's mobile device.

[0429] Local videos can be reproduced on the lower display, and when the cast button is touched, the video will be reproduced immediately, even when the passenger screen application is running in passenger mode.

[0430] Figure 62 The music delivery process according to an embodiment of the present invention is illustrated.

[0431] When you move to the Music tab on the mobile device's application home screen, the corresponding music will be played when you select a media thumbnail.

[0432] When the cast button is touched, the casting will be executed on the upper display. When the casting is successful, the sound path setting button will be displayed.

[0433] When the minimize button is touched, the currently playing player screen changes to the minimized screen and is displayed, and the application's main screen is shown.

[0434] According to relevant technologies, driver status is monitored to provide warnings, but there are limitations on the criteria used to provide warnings, and there are issues with warnings being restricted to being provided only to the driver.

[0435] To address the aforementioned problems, the present invention aims to provide an apparatus for providing progressive warnings based on the driver's state. This apparatus is capable of determining whether to provide a warning by subdividing the criteria for providing a warning based on the driver's state, and notifying passengers of the driver's inattentive state.

[0436] Figure 63 An apparatus for providing progressive warnings based on driver status, according to an embodiment of the present invention, is shown.

[0437] An apparatus for providing progressive warnings based on driver status according to an embodiment of the present invention includes: an input unit 710 configured to receive driver status detection results; a memory 720 storing a program for classifying warning steps using the driver status detection results and providing predetermined warnings based on the driver status detection results; and a processor 730 configured to execute the program. The processor 730 provides warnings using warning message output, warning sound output, light illumination, etc.

[0438] The processor 730 uses the upper display located above the lower display in the cockpit to output warning messages, and controls the output of warning messages when the predetermined warning step moves to the upper display on the passenger side according to the passenger mode.

[0439] The processor 730 controls the output of a warning sound to the passenger headrest speaker according to a predetermined warning procedure.

[0440] The processor 730 provides warnings for a predetermined time period based on warning steps categorized using driver status detection results.

[0441] As a first step, the processor 730 outputs a warning message to the instrument panel and turns on the instrument panel ambient light.

[0442] As a second step, the processor 730 controls the internal ambient lighting to be turned on additionally and outputs a warning sound.

[0443] As a third step, the processor 730 controls the changing and displaying of the colors of the dashboard ambient lighting and interior ambient lighting, the mechanical controls output a beeping sound at a faster reproduction speed, and the emergency lights are turned on.

[0444] Figure 64 A DSW implementation in driver mode according to an embodiment of the present invention is shown.

[0445] When driver mode detects neglecting to keep eyes forward, fatigue, or drowsiness, a warning message is sent to the dashboard via a warning pop-up window.

[0446] The dashboard ambient lights flash in different colors depending on the warning procedure.

[0447] The ambient light flashes in different colors depending on the warning sequence.

[0448] The loudspeaker sound is reproduced with different warning sounds according to the warning steps.

[0449] Determine if the emergency lights are flashing based on the warning procedure.

[0450] In driver mode, the configuration and definitions in the DSW implementation are summarized in Table 25 below.

[0451] [Table 25]

[0452]

[0453]

[0454] Figure 65 A DSW implementation in passenger mode according to an embodiment of the present invention is shown.

[0455] When passenger mode detects that the driver has neglected to keep their eyes forward, is fatigued, or is drowsy, a warning message is sent to the dashboard via a warning pop-up.

[0456] The dashboard ambient lights flash in different colors depending on the warning procedure.

[0457] The ambient light flashes in different colors depending on the warning sequence.

[0458] The loudspeaker sound is reproduced with different warning sounds according to the warning steps.

[0459] Determine if the emergency lights are flashing based on the warning procedure.

[0460] The upper display is in a state of moving towards the passenger side according to the passenger mode, and the driver status classification result and the warning message based on the driver status classification result are provided to the upper display.

[0461] The passenger headrest speakers output different warning sounds depending on the driver's status as a result of the warning procedure.

[0462] In passenger mode, the configuration and definitions in the DSW implementation are summarized in Table 26 below.

[0463] [Table 26]

[0464]

[0465] Figure 66 A step-by-step warning delivery process in driver mode according to an embodiment of the present invention is illustrated.

[0466] When a driver is detected neglecting to keep their eyes forward, or experiencing fatigue or drowsiness while driving, as a first warning, a pop-up phrase appears on the instrument panel and the instrument panel ambient lights turn on.

[0467] As a second warning, a phrase pops up on the dashboard, turns on the dashboard ambient lights, turns on the interior ambient lights, and the warning sound is reproduced using the vehicle's speakers.

[0468] As a third warning step, a phrase pops up on the dashboard, turns on the dashboard ambient lighting in a different color than the second step (e.g., red), turns on the interior ambient lighting in a different color than the second step (e.g., red), reproduces a warning sound different from the second step using the vehicle's speakers (e.g., a warning sound reproduced at a faster cycle), and turns on the hazard lights.

[0469] Figure 67 A step-by-step warning delivery process in passenger mode according to an embodiment of the present invention is illustrated.

[0470] When a driver is detected neglecting to keep their eyes forward, or experiencing fatigue or drowsiness while driving, as a first warning, a pop-up phrase appears on the instrument panel and the instrument panel ambient lights turn on.

[0471] As a second warning step, a phrase pops up on the instrument panel, the instrument panel ambient lights turn on, the interior ambient lighting turns on, and a warning sound is reproduced using the vehicle's speakers. Additionally, the warning message is delivered using the upper display, which moves to the passenger side depending on the passenger mode.

[0472] As a third warning step, a phrase pops up on the dashboard, the dashboard ambient lighting is activated in a different color (e.g., red) than in the second step, the interior ambient lighting is activated in a different color (e.g., red) than in the second step, and a warning sound is reproduced using the vehicle's speakers in a different manner than in the second step (e.g., a warning sound reproduced at a faster cycle), and the hazard lights are activated. Additionally, a warning message is displayed on the upward-facing display, and a warning sound is output through the passenger headrest speakers.

[0473] Furthermore, the method according to embodiments of the present invention can be implemented in a computer system or recorded in a recording medium. The computer system may include at least one processor, memory, user input device, data communication bus, user output device, and storage. Each of the above components performs data communication via the data communication bus.

[0474] The computer system may also include a network interface coupled to the network. The processor may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in memory and / or storage.

[0475] Memory and storage can include various types of volatile or non-volatile storage media. For example, memory can include read-only memory (ROM) and random access memory (RAM).

[0476] Therefore, the method according to embodiments of the present invention can be implemented as a computer-executable method. When the method according to embodiments of the present invention is executed in a computer device, computer-readable instructions can execute the method according to the present invention.

[0477] According to the present invention, by using an upper / lower display arranged in an integrated cockpit to perform an integrated search, the search convenience for both the driver and passengers is improved.

[0478] It should be noted that the purpose of this invention is not limited to the above-described purpose, and other purposes of this invention will be apparent to those skilled in the art from the following description.

Claims

1. A system for performing a search using multiple displays, the multiple displays including a lower display located in the cockpit and an upper display located above the lower display, the system comprising: processor; as well as A computer-readable medium, in communication with the processor, stores instructions that, when executed by the processor, cause the processor to control the system to perform the following steps: The first user input, received from the user through the user interface, initiates the integrated search session; and In response to receiving the first user input, perform the following operations: The upper display shows multiple first lists corresponding to multiple menu categories, each first list including multiple selectable items of the corresponding menu category arranged in the order of most recent use; as well as The lower display shows: (1) an integrated search window configured to display search terms provided by the user, and (2) multiple buttons corresponding to the multiple menu categories, each button being associated with the most frequently used selectable item of the corresponding menu category; Receive second user input from the user via the user interface, the second user input including a search term containing one or more letters; and In response to receiving the second user input, perform the following operations: The received search term is displayed in the integrated search window as one or more letters; as well as The plurality of first lists displayed on the upper display are replaced with a plurality of second lists, each second list including a plurality of selectable items in a corresponding menu category related to one or more letters of the received search term.

2. The system as claimed in claim 1, wherein, When executed by the processor, the instructions further cause the processor to control the system to perform the following steps: User input identifying a user profile is received from the user via the user interface; as well as In response to receiving the user input, search results are displayed for each of the plurality of menu categories related to the identified user profile and search terms.

3. The system as described in claim 1, wherein, In order to display the integrated search window on the lower display, the instructions, when executed by the processor, further cause the processor to control the system to perform the following steps: Detects that an application function is being executed on the upper display; as well as In response to the detection that the application function is being executed on the upper display, the integrated search window is displayed on the lower display.

4. The system as claimed in claim 1, wherein, When executed by the processor, the instructions further cause the processor to control the system to perform the following steps: The system is currently in passenger mode; and The integrated search window is displayed on the upper display.

Images