System and method for controlling touch drag movement for integrated automotive display

Abstract

Disclosed is a system for controlling a touch drag movement for an integrated automotive display device. The present invention comprises: a touch display located behind a steering wheel and including a first display area for displaying driver information and a second display area for displaying infotainment information; and a processor for processing an input of the touch display, wherein, when a touch input is detected in the second display area and then a drag input is detected in the direction to the first display area, the processor identifies a UI element corresponding to the position of the touch input, moves the identified UI element to the first display area, and automatically returns the moved UI element to the original second display area after a predetermined time.

Description

Touch drag movement control system and method for an integrated display device for vehicles

[0001] The present invention relates to an integrated display device for a vehicle, and more specifically, to a touch drag movement control system and method that allows a driver to control the movement of a specific area of ​​a display by a touch drag method while driving.

[0002] In-vehicle display devices are gradually becoming larger and evolving into a form that integrates the instrument cluster and AVN (Audiovisual Navigation) screens. These integrated displays offer the advantage of significantly enhancing user convenience by providing various information at a glance. For example, driving information, entertainment data, navigation, and rear-view camera footage can all be displayed on a single screen, allowing passengers to easily access necessary information. However, there is a possibility that serious safety issues may arise if these large display devices are operated while driving.

[0003] Specifically, to operate the AVN area while driving, drivers must shift their gaze from the road to look at the screen; as larger screens display more information, it can take a long time to find the desired information or perform operations. Consequently, drivers frequently have to turn their heads to look at the display or use their hands for touch operations, and this distraction poses a high risk of accidents while driving. In particular, frequently shifting one's gaze to the screen while driving can significantly impair a driver's response ability in situations requiring quick reactions.

[0004] To address these issues, various technologies are being developed to enable drivers to drive safely while using integrated displays. For example, research is underway to minimize manual driver intervention by automatically moving on-screen information or adding voice control functions. However, since these methods are often limited in practicality or effective only under specific conditions, there is a demand for more intuitive and safer display operation methods.

[0005] Korean Patent Registration No. 2179289, "Vehicle Display System and Method for Controlling the Same," is a technology developed to solve these problems. It discloses a technology that displays various information on a display within a vehicle and allows the driver to easily control it. This patent enables more efficient display operation while driving by introducing a method of switching screens or displaying specific information through touch-based gestures. The driver can switch information and easily bring up the desired screen using gestures such as scrolling, swiping, tapping, and pinching. As a result, the driver has the advantage of being able to quickly access necessary information even while driving.

[0006] However, several problems still exist with the aforementioned conventional technology. First, gesture-based touch operation has not fundamentally solved the problem of having to take one's eyes off the road while driving. The driver must still look at the display while the touch operation is being performed, which leads to distraction while driving.

[0007] Furthermore, if drivers are unfamiliar with touch gestures or unable to accurately locate necessary information, they may spend a long time operating the screen, potentially jeopardizing safety while driving. Therefore, there is a need for more advanced technology regarding the method of operating displays while driving to ensure driving safety.

[0008] The present invention was developed to improve safety issues that may occur while driving in conventional vehicle integrated display systems, and the objective of the present invention is to provide a touch drag movement control system and method that allows a driver to intuitively access necessary information without taking their eyes off the road while driving.

[0009] In particular, the purpose is to provide a touch drag movement control method for an integrated display device for a vehicle that allows the driver to touch and drag the AVN area toward the cluster, thereby moving the AVN area toward the cluster so that necessary information can be easily checked at a glance, and automatically returning to its original position when the operation is finished, so that it can be used intuitively and conveniently without additional operation while driving.

[0010] To achieve the above objectives, the touch drag movement control system of an integrated display device for a vehicle according to the present invention comprises: a touch display including a first display area located behind the steering wheel for displaying driver information and a second display area for displaying infotainment information; and

[0011] It is equipped with a processor that processes the input of the above touch display.

[0012] At this time, when the processor detects a touch input in the second display area and a drag input in the direction of the first display area, it identifies a UI element corresponding to the touch input location, moves the identified UI element to the first display area, and automatically returns the moved UI element to the original second display area after a certain period of time.

[0013] At this time, the processor modifies the shape and position of the existing UI elements in the first display area so that the UI elements of the second display area moved to the first display area do not obscure the existing UI elements displayed in the first display area.

[0014] At this time, the processor functionally classifies UI elements displayed in the second display area, and analyzes the driver's operation history or vehicle status during the previous period to determine the UI elements to move to the first display area according to the context.

[0015] However, the decision is made based on context only when the coordinates of the touched second display area overlap with two or more distinct UI elements or are within a threshold distance from the boundary of two or more UI elements; otherwise, the touched UI element is dragged because the driver's intention can be clearly understood from the touched coordinates.

[0016] At this time, the processor adjusts the sensitivity of the touch input according to the movement or vibration of the vehicle to prevent touch malfunctions caused by vibration while driving.

[0017] The driver can arbitrarily set the time for a UI element dragged to the first display area to automatically return to the second display area. In this case, if the driver's hand is close to the UI element dragged to the first display area, the return time is automatically extended. Nevertheless, if there is no touch input from the driver, the element returns to its original position.

[0018] UI elements dragged to the first display area are displayed in a modified form to occupy the minimum amount of the first display area.

[0019] The present invention allows a driver to drag specific UI elements in the infotainment area to an area where driver information is displayed, thereby enabling the driver to intuitively check necessary information without significantly diverting their gaze from the road while driving. This minimizes visual distraction while driving, thereby improving driving safety.

[0020] In addition, moved UI elements automatically return to their original positions once the operation is completed or after a certain period of time. The return time can be set by the driver, allowing for convenient use without additional operations. The return time can also be automatically extended if the driver's hand is nearby, providing convenience tailored to the user's situation.

[0021] In addition, the size and position of UI elements moved to the first display area are automatically adjusted so that they do not overlap with existing driver information, allowing the driver to clearly perceive key information. Through this adaptive layout, visibility is maintained even while the display is in use, thereby simultaneously enhancing user experience and safety.

[0022] In addition, when the touched coordinates overlap with or are located at the boundary of two or more UI elements, the UI element corresponding to the driver's intent is automatically identified based on the driver's previous operation history or vehicle status. This enables accurate operation even at the boundary area, thereby allowing intuitive and reliable display operation even while driving.

[0023] FIG. 1 is a block diagram showing the configuration of an integrated display device according to the present invention.

[0024] Figure 2 is a flowchart illustrating the process in which a UI element of the second display area moves to the first display area by drag input and then automatically returns to its original position after a certain period of time.

[0025] Figure 3 is a screen example illustrating how a UI element in the second display area moves to the first display area by drag input.

[0026] Figure 4 illustrates a modified form of a UI element moved to a first display area so that it does not overlap with an existing UI element.

[0027] Explanation of the symbols

[0028] 100 : Integrated display device

[0029] 110 : Processor

[0030] 120 : Touch display

[0031] 121 : First display area

[0032] 122 : Second display area

[0033] 11: UI elements of the first display area

[0034] 12: UI elements of the second display area

[0035] Hereinafter, the present invention will be described in detail with reference to preferred embodiments of the invention and the accompanying drawings, under the premise that identical reference numerals in the drawings refer to identical components.

[0036] When a component is described as "comprising" another component in the detailed description of the invention or the claims, this shall not be interpreted as being limited to being composed solely of said component unless specifically stated otherwise, but shall be understood as potentially including additional components.

[0037] Additionally, components named as "~means," "~part," "~module," or "~block" in the detailed description of the invention or the claims refer to units that process at least one function or operation, and each of these may be implemented by software or hardware, or a combination thereof.

[0038] A touch drag movement control system for an integrated display device for a vehicle according to the present invention will be described below with reference to the attached drawings.

[0039] FIG. 1 is a block diagram showing the configuration of an integrated display device according to the present invention. FIG. 1 explains the location and relationship of an integrated display device (100), a processor (110) inside the device, a touch display (120), a first display area (121), and a second display area (122).

[0040] The integrated display device (100) is located in the driver's seat of a vehicle and consists of a processor (110), a touch display (120), and a first display area (121) and a second display area (122) located below it. These components are interconnected so that a user can intuitively check and control various information while driving.

[0041] First, the processor (110) is a core control unit of the integrated display device (100) that processes inputs occurring in the touch display (120) in real time and executes control commands. When processing inputs from the touch display (120), the processor (110) detects touch and drag inputs occurring in the second display area (122), identifies UI elements at the corresponding input location, and performs the function of moving them to the first display area (121). For example, if a user touches a specific UI element (12) in the second display area (122) and drags it toward the first display area (121), the processor (110) detects this location and drag path and moves the UI element to the first display area (121).

[0042] At this time, when a UI element is located across a boundary or multiple UI elements are located in an overlapping position, the processor (110) predicts the element indicated by the touched coordinates based on the driver's operation history and the vehicle's state, thereby accurately identifying the UI element intended by the driver. Through this context recognition function, intuitive operation becomes possible even when the driver's drag location is unclear. Additionally, the processor (110) adjusts the sensitivity of the touch input by reflecting the vehicle's movement or vibration state, thereby preventing malfunctions caused by vibration while driving and providing stable input processing.

[0043] The touch display (120) is a primary interface between the driver and the integrated display device (100), and is a display device designed to allow the driver to check various information and operate it directly. The touch display (120) is positioned in front of the driver's seat so that the driver can easily check various information without significantly deviating from their field of vision. The touch display (120) is divided into a first display area (121) and a second display area (122) to display different information, and the driver can interact through touch input.

[0044] The first display area (121) is located behind the steering wheel and is a space that provides essential information necessary for the driver while driving. This area displays essential information for the driver, such as driving information including speed, RPM, fuel status, and warning lights. The first display area (121) is positioned so that the driver's field of vision does not deviate significantly from the road, and static information is primarily displayed to enhance driving safety. When a UI element (12) is moved to this area, the processor (110) automatically adjusts the size and position of the UI element so that it does not overlap with the existing UI element (11). Through this, the driver can intuitively check various functions without obscuring key driving information.

[0045] The second display area (122) is located in the center of the vehicle and provides infotainment and convenience functions, displaying various information such as navigation maps, audio controls, and air conditioning operations. The second display area (122) is designed to allow the driver to interact through touch and drag inputs and is a primary area that processes inputs to move UI elements to the first display area (121). For example, if a temperature control UI element (12) displayed in the second display area (122) is touched and dragged, the UI element moves to the first display area (121), allowing the driver to easily control the interior temperature or audio settings. As such, the second display area (122) provides various functions for the driver's convenience and is configured to enable intuitive and fast operation.

[0046] UI elements (11, 12) are individual information elements displayed in each display area of ​​the integrated display device (100), and each is responsible for information necessary during driving and infotainment information. The UI elements (11) that are primarily displayed in the first display area (121) consist of information directly related to driving safety, such as speed, fuel level, and warning lights. These are displayed in a fixed form so that they can be quickly recognized while driving, and the dragged UI elements (12) are temporarily moved to the first display area (121), and their shape and position are automatically adjusted so that the UI elements (11) in the first display area (121) are not obscured.

[0047] UI elements (12) related mainly to infotainment and convenience functions are displayed in the second display area (122), and can be touched and dragged by the user. For example, a navigation map, a temperature control slider, audio settings, etc., can be displayed as UI elements (12), and the user can move these elements to the first display area (121) to temporarily check the necessary information.

[0048] When a UI element (12) is moved to the first display area, the element is temporarily displayed with its size and position adjusted, and after a certain period of time, it automatically returns to its original position in the second display area (122). At this time, the return time of the UI element can be set by the driver, and if the driver's hand position is maintained close to the first display area, the return time is automatically extended as described below to maximize user convenience.

[0049] Hereinafter, we will examine the touch drag movement control method of an integrated display device for a vehicle according to the present invention with reference to FIGS. 2 to 4.

[0050] Figure 2 is a flowchart illustrating the process in which a UI element of the second display area moves to the first display area by drag input and then automatically returns to its original position after a certain period of time.

[0051] FIG. 3 is an example screen diagram illustrating a UI element in a second display area being moved to a first display area by drag input. FIG. 3(a) illustrates a user selecting a UI element in the second display area, and FIG. 3(b) illustrates the selected UI element being moved to the first display area.

[0052] FIG. 4 illustrates a modified form of a UI element moved to a first display area so as not to overlap with an existing UI element. FIG. 4 (a) illustrates a modified form of a UI element (11) of the existing first display area that is moved sideways so as not to be obscured, and FIG. 4 (b) illustrates a modified form of a UI element (12) of a selected second display area that is dragged to the first display area and is displayed in a pre-set form different from the UI appearance that was displayed in the second display area.

[0053] In the recognition step (S11) of touch and drag input to a UI element in the second display area, the processor (110) of the integrated display device (100) detects touch and drag inputs occurring in the second display area (122) in real time. The processor analyzes the touch coordinates and drag path transmitted through the touch display (120) to determine the location where the input started and checks whether the location corresponds to a specific UI element (12) in the second display area (122). At this time, the touch display (120) immediately recognizes the start of user input and transmits it to the processor (110), and identifies whether the input location is within the specific UI element.

[0054] In the identification step (S12) of the UI element corresponding to the touch input location, the processor (110) identifies the UI element (12) corresponding to the touch coordinates within the second display area (122). The processor determines whether the touched coordinates are within the boundaries of a specific UI element (12) by referring to the position and size information of each UI element in the second display area (122) where the UI element is displayed. For example, if the touch coordinates are within the range of a specific UI element, such as a temperature control slider or an audio control, the UI element is identified as selected.

[0055] In the step (S13) of checking whether the drag input is directed toward the first display area, the processor (110) determines whether the drag direction for the touched UI element (12) is directed toward the first display area (121). The processor (110) determines the direction by analyzing the drag path based on the starting point of the drag input and the current position. If the drag direction is toward the first display area (121), the process proceeds to the next step (S14); otherwise, the input is ignored and the system returns to a waiting state.

[0056] In the step (S14) of moving a UI element to a first display area, the processor (110) moves the UI element (12) touched in the second display area (122) to the first display area (121). In this step, the processor (110) converts the graphic data of the UI element (12) and temporarily displays the corresponding UI element at an appropriate location in the first display area (121). FIG. 3 (a) illustrates a state in which a user selects a UI element (12) in the second display area and starts a drag input, and FIG. 3 (b) shows a state in which the selected UI element (12) has moved to the first display area (121) along the drag path.

[0057] In the step (S15) of adjusting the size and position so that the moved UI element does not obscure the existing UI element, the processor (110) automatically adjusts the size and position of the UI element (12') so that it does not overlap with the existing UI element (11) of the first display area (121). The processor (110) refers to the position and size information of the UI element (11) already displayed in the first display area (121) and repositions the newly moved UI element (12') to an optimal size and position so that it does not obscure the existing UI element. At this time, the UI element (12') is transformed to occupy the minimum amount of space, and FIG. 4 (a) shows the transformed appearance of the UI element (11) of the existing first display area being moved sideways so that it is not obscured, and FIG. 4 (b) shows an example where the UI element (12) of the selected second display area is moved to the first display area and displayed with a different appearance from the existing UI element.

[0058] In the step (S16) of maintaining a UI element for a certain period of time within the first display area, the processor (110) displays the moved UI element (12') within the first display area (121) for a set period of time. In this step, the UI element (12') is maintained in a position within the first display area (121) so that the driver can easily check it even while driving. The driver can use this time to intuitively check the information displayed on the UI element (12'), and the system determines whether to return the UI element by checking the automatic return time in the next step.

[0059] In the step (S17) of checking whether the set automatic return time has elapsed, the processor (110) checks whether the automatic return time of the UI element (12') has exceeded the set time. If the return time pre-specified by the user has elapsed, the process proceeds to the next step (S18); otherwise, the UI element (12') remains in the first display area. The user may set the return time arbitrarily, and if the set time has not elapsed, the process returns to step S16, and the UI element remains within the first display area (121).

[0060] In the step (S18) of checking whether the driver's hand position is near a UI element, the processor (110) detects whether the driver's hand position is near a UI element (12') located in the first display area and determines whether to extend the automatic return time. If the driver's hand position is near the UI element (12'), the processor (110) automatically extends the return time and returns to step S17 to reconfirm the return time. Conversely, if the driver's hand position is not near the UI element, the process proceeds to the next step (S19).

[0061] In the step (S19) of automatically returning the UI element to the original second display area, the processor (110) returns the UI element (12') displayed in the first display area to its original location, the second display area (122). The processor (110) provides visual continuity by applying an animation effect so that the UI element (12') moves naturally to the second display area. As a result, the driver can intuitively confirm the process of the UI element (12'), for which information verification is complete, returning to the original second display area (122), and the process of the present invention returns to a standby state.

[0062] Meanwhile, in the above embodiment, the touch display (120) is described on the premise that it is a single display device and that the area is divided into a first display area (121) and a second display area (122), but in reality, there may be cases where the display areas are physically separated.

[0063] For example, the first display area (121) and the second display area (122) may be implemented as two display devices that are physically separated from each other. In this case, even if the user starts a touch and drag in the second display area and the drag continues to the first display area, it is necessary to process this as a single input to implement the natural movement and return of the UI element.

[0064] The process of checking the drag direction in the above S13 step and determining whether to process it as a single input, followed by the step of moving a UI element to the first display area according to the drag direction recognized as a continuous input in the S14 step, can be processed as follows.

[0065] When a touch and drag input occurs in the second display area (122), the processor (110) of the integrated display device detects the input in real time and begins to continuously record the coordinates and movement path. At this time, the processor (110), taking into account the time interval between the end of the touch in the second display area (122) and the start of the drag in the first display area (121) and the continuity of the input coordinates, recognizes and processes the drag input between the two display areas as a single continuous input when it is determined to be a single continuous input.

[0066] To this end, the processor (110) first tracks in real time when the touch and drag input coordinates of the second display area (122) reach the boundary. For example, if the user moves to the first display area (121) without lifting their hand from the boundary of the second display area, the processor (110) determines this as a continuous drag input even though the two areas are separated, and naturally moves the UI element (12) to the first display area (121). To this end, the additional input coordinate interval near the boundary is reduced, and the time delay is automatically adjusted to correct the drag input so that it continues smoothly.

[0067] Additionally, when a drag in the second display area (122) is terminated and a new touch is started in the first display area (121), the processor (110) applies two criteria to regard this as a single command with the existing drag input. First, it determines the positional continuity between the point where the touch ends in the second display area (122) and the point where the new touch starts in the first display area (121), and recognizes it as a single input if the coordinates are within a certain distance. Second, if the time interval between the end and start of the touch between the two areas is within a preset threshold, it recognizes it as the same drag input, thereby enabling continuous movement of the UI element.

[0068] The processor (110) collects and processes input coordinate data in real time to accommodate such continuous drag inputs, and recognizes the two inputs as a single command and integrates them into a single control flow even when the touch coordinates of each area appear separated. Through this, when a user starts dragging in the second display area and then moves their finger to the first display area, the UI element (12) leads to a single drag operation despite the physical separation between the two areas.

[0069] Meanwhile, it has been explained that in the above S18 step, whether to extend the return is determined based on whether the driver's hand position is located near the first display area.

[0070] To implement this, a camera and proximity sensor that detect the position of the user's hand inside the vehicle are used together to accurately determine whether the user intends to further manipulate the corresponding UI element.

[0071] Furthermore, as described above, the automatic return time of the UI element may be extended when the driver's hand position is located near the first display area, but it can be set so that the automatic return time is not extended and the element returns immediately when it is determined that the driver is holding the steering wheel and has no intention to operate.

[0072] Specifically, the processor (110) continuously detects the driver's hand position through a camera (not shown) installed inside the vehicle and a proximity sensor (not shown) located in front of the first display area, thereby controlling the return time of the UI element (12').

[0073] After the UI element (12') dragged to the first display area is maintained for a set period of time, the processor (110) detects the driver's hand position at the time when the automatic return time arrives and adjusts the return time.

[0074] The processor (110) tracks the position of the hand through an in-vehicle camera and checks whether the hand is near the first display area through a proximity sensor in front of the first display area. If the driver's hand is located near a UI element (12') or is approaching the first display area, the processor (110) determines that the driver has an intention to perform additional operations on the UI element and extends the automatic return time for a certain period. This extension of the return time reflects the user's intention to operate the UI element and provides a convenience feature that prevents the UI element from suddenly returning to its original position during operation.

[0075] Additionally, the processor (110) sets additional conditions related to hand position so that even if the driver places their hand near the first display area, the return process is performed immediately without extending the return time while the driver is holding the steering wheel. To this end, the processor (110) analyzes the position and shape of the hand through a camera, and if the hand is in contact with the steering wheel, it saves the state and determines that the driver has no intention of operating the UI element (12').

[0076] This is because, due to an error in the proximity sensor, the hand position is judged to be close even when the driver's hand is holding the steering wheel, and the return of the UI element (12') may be extended contrary to the user's intention.

[0077] Meanwhile, the function of identifying the UI element corresponding to the touch input location in the above S12 step can be implemented as follows.

[0078] When a driver's finger touches the touchscreen, the touch area is not limited to a single pixel but can occupy a considerably large area.

[0079] Accordingly, when a touch input by a driver's finger is made at a boundary within the second display area (122) or at a location where multiple UI elements overlap, the processor (110) first compares the touch coordinates with the boundary values ​​of the UI elements at that location to primarily identify the UI element closest to the touched coordinates.

[0080] In other words, if a touch is made at a location more than a threshold away from the boundaries of UI elements, there is no problem because the driver's intent is clear. However, if it is within the threshold, the driver's intent is not clear, so it is inferred which UI element was selected.

[0081] To this end, the processor (110) analyzes the driver's operation history and the vehicle's current state together to determine the UI element to be finally selected.

[0082] The driver's operation history is a history of UI element usage within a recent period of time (e.g., the last 5 minutes) recorded by the processor (110), and includes UI elements that the driver has repeatedly accessed or functions that are frequently used. For example, if the driver has a history of operating a temperature control UI element multiple times recently, the processor (110) can determine that the temperature control UI element is an element that matches the driver's intention, even if the corresponding touch coordinates are located at the boundary between the temperature control UI element and another UI element. At this time, the processor (110) selects an element that matches the driver's intention by assigning weights to the operation history and giving higher priority to UI elements that are used more frequently.

[0083] Additionally, the system includes a function to select a touched UI element using information related to the vehicle's status. For example, information such as changes in the vehicle's internal and external temperatures, engine status, and current driving mode is collected as vehicle status, which may influence the priority selection of a specific UI element. If the current internal temperature of the vehicle exceeds a specific threshold and requires indoor temperature control, the processor (110) may prioritize selecting the temperature control UI element when the coordinates of the boundary area touched by the driver overlap with the temperature control UI element and another UI element.

[0084]

[0085] The technical concept of the present invention has been examined through the above examples.

[0086] It is obvious that a person skilled in the art to which the present invention pertains can make various modifications or changes to the embodiments described above from the description of the present invention. Furthermore, it is obvious that a person skilled in the art to which the present invention pertains can make various modifications including the technical concept according to the present invention from the description of the present invention, even if not explicitly illustrated or described, and such modifications still fall within the scope of the rights of the present invention. The embodiments described above with reference to the accompanying drawings are described for the purpose of explaining the present invention, and the scope of the rights of the present invention is not limited to these embodiments.

Claims

1. A touch display including a first display area for displaying driver information and a second display area for displaying infotainment information, located behind the steering wheel; and A processor for processing input of the above-mentioned touch display is provided, A touch drag movement control system characterized by the processor identifying a UI element corresponding to the touch input location when a drag input is detected in the direction of the first display area after a touch input is detected in the second display area, moving the identified UI element to the first display area, and automatically returning the moved UI element to the original second display area after a certain period of time.

2. In Claim 1, A touch drag movement control system characterized by the processor modifying the shape and position of existing UI elements in the first display area so that UI elements in the second display area moved to the first display area do not obscure existing UI elements displayed in the first display area.

3. In claim 1 or claim 2, The above processor functionally classifies UI elements displayed in the second display area, A touch drag movement control system characterized by analyzing the driver's operation history or vehicle status during the previous period and determining the UI element to move to the first display area according to the context.

Images