Interface display method and device, vehicle-mounted terminal and storage medium
By filtering and displaying controls with insufficient completeness in the edge area of the interface, and displaying their prompt information on the interface, the problem of inaccurate voice control in the on-board display interface is solved, and the user experience is improved.
Patent Information
- Application Number
- CN202411258999.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2025-01-10
AI Technical Summary
In the on-board display interface, controls that are in the edge area and lack of display integrity are difficult to control through voice, resulting in low accuracy of voice control and reducing user experience.
By filtering elements of the display interface, controls that support voice control and display completeness are less than the preset threshold are obtained, and their corresponding prompt information is displayed in the target area of the display interface, allowing users to control voice through prompt information.
Improve the accuracy of the edge area control of the voice control display interface and improve the user experience.
Smart Images
Figure CN119292719A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of interface display technology, and in particular to an interface display method, device, vehicle-mounted terminal and storage medium. Background Art
[0002] In the related art, for controls on a display interface, users can directly control the controls through voice controls using the prompt words corresponding to the controls. However, for controls located in the edge area of the display interface and when the display integrity of the controls on the display interface is insufficient, users cannot accurately control the controls through voice, resulting in low accuracy of voice control, thereby reducing the user experience. Summary of the invention
[0003] In view of this, the embodiments of the present application at least provide an interface display method, device, vehicle-mounted terminal and storage medium.
[0004] The technical solution of the embodiment of the present application is implemented as follows:
[0005] In a first aspect, an embodiment of the present application provides an interface display method, comprising screening elements in a display interface based on a first preset condition and a second preset condition to obtain a target control; the target control is an element in the display interface that satisfies the first preset condition and the second preset condition; the prompt information corresponding to the target control is displayed in a target area in the display interface; the prompt information is used to prompt the user to control the target control using a target voice; wherein the first preset condition indicates that the target control supports voice control, and the second preset condition indicates that the display completeness of the target control is less than a preset threshold.
[0006] In a second aspect, an embodiment of the present application provides an interface display device, comprising: a screening module, for screening elements in a display interface based on a first preset condition and a second preset condition to obtain a target control; the target control is an element in the display interface that satisfies the first preset condition and the second preset condition; a display module, for displaying prompt information corresponding to the target control in a target area in the display interface; the prompt information is used to prompt a user to control the target control using a target voice; wherein the first preset condition indicates that the target control supports voice control, and the second preset condition indicates that the display completeness of the target control is less than a preset threshold.
[0007] In a third aspect, an embodiment of the present application provides a vehicle-mounted terminal, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement some or all of the steps in the above method when executing the instructions.
[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements some or all of the steps in the above method when executed by a processor.
[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement some or all of the steps in the above method.
[0010] Technical effects:
[0011] In an embodiment of the present application, the elements of the display interface are screened based on the first preset condition and the second preset condition to obtain controls in the edge area of the display interface that support voice control, and the prompt information corresponding to the control is completely displayed in the target area of the display interface so that the user can control the control by voice through the prompt information, thereby improving the accuracy of voice-controlled controls in the edge area of the display interface and improving the user experience.
[0012] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.
[0014] Figure 1 A schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application;
[0015] Figure 2 A schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application;
[0016] Figure 3 A schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application;
[0017] Figure 4 A schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application;
[0018] Figure 5 A schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application;
[0019] Figure 6 A schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application;
[0020] Figure 7 A schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application;
[0021] Figure 8 A schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application;
[0022] Fig. 9 A schematic diagram of a control in a display interface provided in an embodiment of the present application;
[0023] Fig. 10A A schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application;
[0024] Fig. 10B A schematic diagram of an implementation of generating control integrity provided in an embodiment of the present application;
[0025] Fig.11 A schematic diagram of the structure of an interface display device provided in an embodiment of the present application;
[0026] Fig.12 A schematic diagram of a hardware entity of a vehicle-mounted terminal provided in an embodiment of the present application. DETAILED DESCRIPTION
[0027] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.
[0028] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. The terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific order for the objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application.
[0030] Currently, for the controls in the vehicle display interface, after the user gets in the vehicle, any control in the display interface can be controlled by voice; however, for the controls located in the edge area of the display interface and the display integrity of the control is insufficient on the display interface, the user cannot well control the controls located in the edge area of the display interface and the display integrity is insufficient by voice, which reduces the accuracy of voice control and thus reduces the user experience.
[0031] Based on the above technical problems, the present application filters the elements of the display interface based on preset conditions to obtain controls in the edge area of the display interface that support voice control, and displays the prompt information corresponding to the control in its entirety on the display interface so that users can control the control by voice through the prompt information, thereby improving the accuracy of voice-controlled controls in the edge area of the display interface and improving the user experience.
[0032] The embodiment of the present application provides an interface display method, which can be executed by a processor of a vehicle-mounted terminal, or by a server, a laptop, a tablet computer, a desktop computer, a central control device, or other device with data processing capabilities.
[0033] Figure 1 The following is a schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application, which can be executed by a processor of a vehicle terminal. Figure 1 As shown, the method includes the following steps S101 and S102, combining Figure 1 The steps shown are explained.
[0034] Step S101: based on a first preset condition and a second preset condition, filter the elements in the display interface to obtain a target control; the target control is an element in the display interface that meets the first preset condition and the second preset condition.
[0035] In some embodiments, the elements in the display interface may be icons, text, etc. displayed in the display interface.
[0036] In some embodiments, it is first necessary to obtain all elements in the display interface, that is, to obtain all icons, texts, etc. in the display interface; wherein, obtaining the elements in the display interface includes: obtaining multiple views of the display interface; traversing each view to obtain at least one element in the display interface; wherein, traversing each view to obtain at least one element and the attributes of each element; the attributes of each element may include the coordinate information, function, display label, event generated when the element is triggered, text information, annotation information, etc. of each element.
[0037] In some embodiments, it is necessary to obtain the elements in the edge area of the display interface among all the elements of the display interface, that is, to obtain icons, texts, etc. in the edge area of the display interface, including: based on the coordinate information of each element in the display interface, obtaining the elements in the edge area of the display interface.
[0038] In some embodiments, the controls in the display interface are used to display data, perform operations, interact with users, etc. In the embodiment of the present application, the target control is an element of at least one element of the display interface that satisfies a first preset condition and a second preset control, wherein the first preset condition indicates that the target control supports voice control, and the second preset condition indicates that the display completeness of the target control in the display interface is less than the second preset condition.
[0039] In some embodiments, based on a first preset condition and a second preset condition, elements in an edge area of a display interface are screened to obtain a target control; it is understandable that at least one element in an edge area of the display interface is obtained, and at least one element is screened based on the first preset condition to obtain a first element; the first element is screened based on the second preset condition to obtain a second element, and the second element is determined as a target control; wherein, at least one element is screened by the first preset condition; it is understandable that, by the attributes of each element in at least one element, it is determined whether each element supports voice control, and the element supporting voice control is determined as the first element; the first element is screened by the second preset condition; it is understandable that the display completeness of each first element in the display interface is obtained, and the first element whose display completeness is less than a preset threshold is determined as the target control.
[0040] In some embodiments, at least one element may be first screened based on a second preset condition to obtain a first element, and then the first element may be screened based on the first preset condition to obtain a second element, and the second element may be determined as a target control. Wherein, at least one element is screened based on the second preset condition; it can be understood that the display completeness of each element in at least one element is obtained, and the element in at least one element whose display completeness is less than a preset threshold is determined as a first element; then at least one element is screened based on the first preset condition; it can be understood that the first elements are screened based on the attributes of each first element, that is, the first element in the first element that supports voice control is determined as a target control.
[0041] Step S102: displaying prompt information corresponding to the target control in a target area of the display interface; the prompt information is used to prompt the user to control the target control using a target voice.
[0042] In some embodiments, the prompt information of the target control may be text information, pattern information, etc.; the prompt information of the target control is used to prompt the user to control the target control using a target voice; wherein the target voice is issued by the user based on the prompt information; exemplarily, the prompt information corresponding to the target control is a music player, then the user can send an instruction to turn on or off the music player to the display interface based on the music player to achieve control of the music player.
[0043] In some embodiments, the target area in the display interface refers to a blank area in the display interface, that is, an area in the display interface where no elements exist; wherein the target area may also be an area in at least one blank area in the display interface that is closest to the target control; or an area in at least one blank area in the display interface that is easiest for a user to capture; for example, the display interface includes multiple blank areas, and a user can most easily find the content in the center area of the display interface in the display interface, and the blank area close to the center of the display interface is determined as the target area.
[0044] In some embodiments, there is an association relationship between the target control and the prompt information of the target control; exemplarily, the association relationship is reflected as follows: the target control and the prompt information of the target control can be connected by an indicating arrow, and the display states of the target control and the prompt information of the target control are the same (for example, the target control and the prompt information are both flashing, displayed in the same color and highlighted, etc.).
[0045] In an embodiment of the present application, elements in the edge area of the display interface are screened by the first preset condition or the second preset condition to obtain controls in the edge area of the display interface whose display completeness is less than a preset threshold and which support voice control, and the prompt information corresponding to the control is displayed in the target area of the display interface so that the user can control the control by voice through the prompt information, thereby improving the accuracy of voice control of controls in the edge area of the display interface and improving the user experience.
[0046] Figure 2 The following is a schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application, which can be executed by a processor of a vehicle-mounted terminal. Figure 1 In step S101, when filtering the elements in the display interface based on the first preset condition, the steps S201 and S202 may be included. Figure 2 The steps shown are explained.
[0047] Step S201: Acquire each to-be-screened element in the display interface and description information of each to-be-screened element.
[0048] In the embodiment of the present application, at least one element to be filtered in the display interface is filtered according to a first preset condition, that is, at least one element to be filtered is filtered based on the description information of each element to be filtered.
[0049] In some embodiments, each to-be-filtered element in the display interface represents an element in the display interface that has not been filtered by the first preset condition or the second preset condition.
[0050] In some embodiments, the description information of each element to be screened includes the functions that the element to be screened can realize, display labels, events generated when the element to be screened is triggered, text information of the element to be screened, annotation information of the element to be screened, etc.
[0051] In some embodiments, by traversing at least one window displayed in the current display interface, a view element list of each window is obtained, and based on the view element list of each window, at least one element to be filtered in the display interface and description information of each element to be filtered are obtained.
[0052] In some embodiments, based on the position information of each element to be filtered in the description information of each element to be filtered, at least one element to be filtered located in the edge area of the display interface is obtained.
[0053] Step S202: obtaining elements to be screened whose description information contains target features; wherein the presence of the target features in the description information of the elements indicates that the elements support voice control.
[0054] In an embodiment of the present application, at least one element to be filtered located in the edge area of the display interface is filtered according to a first preset condition; that is, at least one element to be filtered is filtered based on the description information of each element to be filtered located in the edge area of the display interface, and if the description information of the element to be filtered contains a target feature that characterizes that the element to be filtered supports voice control, the element to be filtered is obtained.
[0055] Exemplarily, if at least one of the function of at least one element to be filtered located in the edge area of the display interface, the display label, the event generated when the element to be filtered is triggered, the text information of the element to be filtered, the annotation information of the element to be filtered, etc. can indicate that the element to be filtered supports voice control, the element to be filtered is obtained.
[0056] In an embodiment of the present application, at least one element to be filtered located in an edge area of the display interface is filtered according to a first preset condition to obtain an element to be filtered that supports voice control.
[0057] Figure 3The following is a schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application, which can be executed by a processor of a vehicle-mounted terminal. Figure 2 , the target feature includes at least one of the following: plain text information, annotation information; Figure 2 Step S202 in the above example can be updated to step S301 or step S302. Figure 3 The steps shown are explained.
[0058] In an embodiment of the present application, at least one element to be screened located in the edge area of the display interface is screened based on the target features in the description information of the element to be screened. If the target features exist in the description information of the element to be screened, the element to be screened is characterized as a control; it can be understood that if there is at least one of plain text information or annotation information in the description information of the element to be screened, the element to be screened is characterized as a control.
[0059] Step S301: Obtain elements to be screened that have plain text information in the description information; the plain text information is visible to the user.
[0060] In the embodiment of the present application, the target feature may be plain text information.
[0061] In some embodiments, the plain text information of the element to be screened is the text or data in the element that is directly displayed to the user in a visible form, and the user can intuitively see it in the display interface. The user can issue corresponding voice commands to control the element through the plain text information seen. For example Fig. 9 For the plain text information such as "network", "display", and "voice" in the app, users can say related voice commands such as "turn on network", "turn on display", and "turn on voice" to achieve the desired voice control function.
[0062] In some embodiments, the plain text information of the element to be screened may include information representing the function of the element to be screened, information representing the status of the element to be screened, and the like.
[0063] In some embodiments, if there is plain text information in the description information of the element to be screened, the element to be screened is characterized as a control. In some embodiments, the element to be screened with plain text information in the description information is obtained from at least one element to be screened in the edge area of the display interface.
[0064] Step S302: obtaining elements to be screened that have annotation information in the description information; the annotation information is invisible to the user.
[0065] In the embodiment of the present application, the target feature may be annotation information.
[0066] In some embodiments, the annotation information of the elements to be screened is a special "marker" applied to methods, parameters, variables, constructors, etc., which can be read during compilation, loading or runtime, and corresponding processing can be performed; the annotation information is metadata facing the system during the programming or development stage; exemplarily, during the compilation check process, based on the annotation information marked in the code, the encoder is able to implement the compilation check function; during the document writing process, based on the annotation information marked in the code, the document is generated; it can be understood that the annotation information is visible to the system but not to the user.
[0067] In some embodiments, an element to be filtered having annotation information in its description information is obtained from at least one element to be filtered located in an edge area of the display interface.
[0068] According to the above embodiment, if there is annotation information in the description information of the element to be filtered, it can be determined that the element to be filtered is a control.
[0069] In an embodiment of the present application, whether the element to be filtered is a control is determined based on whether there is plain text information or annotation information in the description information of the element to be filtered. It can be understood that based on the plain text information or annotation information in the description information of the element to be filtered, at least one element to be filtered located in the edge area of the display interface is filtered to obtain the control located in the edge area of the display interface.
[0070] Figure 4 1 is a schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application, and the method can be executed by a processor of a vehicle terminal. Figure 1 In step S101, when filtering the elements in the display interface based on the second preset condition, the steps S401 and S402 may be included. Figure 4 The steps shown are explained.
[0071] Step S401: Obtain each to-be-screened element in the display interface and the display completeness of each to-be-screened element.
[0072] In an embodiment of the present application, at least one element to be filtered located in the edge area of the display interface is filtered based on a second preset condition; that is, at least one element to be filtered located in the edge area of the display interface is filtered based on the display completeness of the element to be filtered; it can be understood that if there is an element to be filtered whose display completeness is less than a preset threshold among at least one element to be filtered located in the edge area of the display interface, the element to be filtered is obtained.
[0073] In some embodiments, by traversing at least one window displayed in the current display interface, a view element list of each window is obtained, and at least one element to be filtered in the display interface is obtained from the view element list of each window.
[0074] In some embodiments, obtaining the display completeness of each of the elements to be screened, that is, obtaining the display completeness of each of the elements to be screened in the edge area of the display interface, includes: obtaining the display range of each of the elements to be screened in the edge area of the display interface in the display interface; determining the display completeness of each of the elements to be screened based on the display range of each of the elements to be screened in the display interface and the complete display range of each of the elements to be screened; it can be understood that the display completeness of each of the elements to be screened is obtained based on the display range of each of the elements to be screened in the display interface divided by the complete display range of each of the elements to be screened; exemplarily, the display range of any element to be screened in the display interface is 2 square centimeters; the complete display range of the element to be screened is 4 square centimeters, and the display completeness of the element to be screened is 50%.
[0075] Step S402: Obtain elements to be screened whose display completeness is less than a preset threshold.
[0076] In some embodiments, it can be understood that, among at least one element to be filtered located in the edge area of the display interface, an element to be filtered whose display completeness is less than a preset threshold is obtained; wherein, the element to be filtered whose display completeness is less than the preset threshold is represented in the display interface, and the user cannot accurately describe the element to be filtered through voice based on the text, pattern and other information corresponding to the element to be filtered.
[0077] In some embodiments, among at least one element to be screened located in the edge area of the display interface, an element to be screened whose display completeness is less than a preset threshold is obtained; exemplarily, if there are 5 elements to be screened on the display interface, their display completenesses are 100%, 90%, 85%, 75%, and 70%, respectively; the preset threshold is 80%, then the elements to be screened whose display completenesses are 75% and 70% among the 5 elements to be screened are obtained.
[0078] In an embodiment of the present application, at least one to-be-screened element of the display interface is screened according to a second preset condition to obtain to-be-screened elements whose display completeness is less than a preset threshold.
[0079] Figure 5 1 is a schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application, and the method can be executed by a processor of a vehicle terminal. Figure 4 , Figure 4 Step S401 in can be updated to step S501 and step S502, combining Figure 5The steps shown are explained.
[0080] Step S501: Acquire the key point position and size of each element to be screened in the display interface.
[0081] In some embodiments, it is understandable that the key point positions of each to-be-screened element located in the edge area of the display interface and the size of each to-be-screened element are obtained.
[0082] In some embodiments, the key point position of each to-be-screened element may be the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates, lower right corner coordinates, center point coordinates, etc. of the to-be-screened element.
[0083] In some embodiments, the size of a control indicates the physical control size occupied by the control, which can be expressed by width*length. The size of a control defines the display range of the control content. For example, if the size of a control is 1920*1080, the content of the control can only be displayed within the range of 1920*1080.
[0084] In some embodiments, the attribute information of each of the elements to be screened that are in the edge area of the display interface is obtained; wherein the attribute information of each of the elements to be screened includes the coordinate information and size information of the element to be screened; the coordinate information of the elements to be screened includes the key point information of the elements to be screened, that is, the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates, lower right corner coordinates, center point coordinates, etc. of the element to be screened.
[0085] Step S502: When the key point position and the size indicate that the element to be screened is located in an edge area of the display interface, obtain the display completeness of the element to be screened.
[0086] In some embodiments, based on the key point position of each element to be screened and the size of the element to be screened, the display range of the element to be screened in the display interface is determined. When the display range of the element to be screened coincides with the boundary coordinates of the display interface, it is determined that the element to be screened is in the edge area of the display interface; then the display completeness of the element to be screened is obtained to screen at least one element to be screened in the display interface.
[0087] In some embodiments, it is also possible to determine whether the element to be screened is located in the edge area of the display interface based on the key point position of the element to be screened; it is understandable that when the distance between the coordinates of the key point of the element to be screened and the boundary of the display interface is less than a preset distance, the element to be screened is determined to be in the edge area of the display interface; exemplarily, the key point position of the element to be screened is the coordinates of the lower left point of the element to be screened, and the preset distance is any one of the length and width of the element to be screened. If the distance between the coordinates of the lower left point and the boundary of the display interface is less than any one of the length and width of the element to be screened, the element to be screened is determined to be in the edge area of the display interface.
[0088] In an embodiment of the present application, based on the key point position and size of the element to be screened, it is determined whether the element to be screened is located in the edge area of the display interface. If the element to be screened is located in the edge area of the display interface, the display completeness of the element to be screened is obtained to screen at least one element to be screened in the display interface.
[0089] Figure 6 1 is a schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application, and the method can be executed by a processor of a vehicle terminal. Figure 4 , the display completeness of each element to be screened in step S401 can be updated to step S601 to step S604, combining Figure 6 The steps shown are explained.
[0090] Step S601: Acquire the key point position and size of each element to be screened in the display interface.
[0091] In some embodiments, the above method is used to obtain the attribute information of each of the elements to be screened in the edge area of the display interface; wherein the attribute information of each of the elements to be screened includes the coordinate information and size information of the element to be screened; the coordinate information of the element to be screened includes the key point information of the element to be screened, that is, the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates, lower right corner coordinates, center point coordinates, etc. of the element to be screened.
[0092] Step S602: Determine the display range of the element to be screened in the screen coordinate system based on the key point position and the size.
[0093] In some embodiments, based on the key point position and size of the element to be filtered in the edge area of the display interface, the display range of the element to be filtered in the edge area of the display interface in the screen coordinate system is determined.
[0094] In some embodiments, it can be understood that based on the coordinates of the key points of the elements to be screened in the display interface coordinate system and the size of the elements to be screened, the coordinates of the display range of the elements to be screened in the screen coordinate system are determined; exemplarily, the screen coordinate system is established with the lower left corner of the display interface as the origin; the coordinates of the key points of the elements to be screened are the coordinates (1,2) of the upper left corner of the elements to be screened, and the size of the elements to be screened is 4*4, then the display range of the elements to be screened in the display interface is a rectangular range formed by the coordinates (1,2) and the coordinates (5,-2).
[0095] Step S603: Determine the display area size of the element to be screened based on the display range of the screen and the display range of the element to be screened.
[0096] In some embodiments, the display area size of the element to be screened represents the size of the display range of the element to be screened in the display interface.
[0097] In some embodiments, the display range of the screen and the display range of the element to be screened are intersected to obtain the display area size of the element to be screened; exemplarily, if the display range of the screen is a matrix range consisting of coordinates (0,0) and coordinates (10,10), and the display range of the element to be screened on the display interface is a rectangular range consisting of coordinates (1,2) and coordinates (5,-2), then the intersection of the display range of the screen and the display range of the element to be screened is a matrix range consisting of coordinates (1,2) and coordinates (4,0), and the display area size of the element to be screened is 8.
[0098] Step S604: Determine the display completeness of the element to be screened based on the complete area size of the element to be screened and the display area size.
[0099] In some embodiments, it can be understood that the display completeness of the element to be screened is obtained by dividing the display area size of the element to be screened by the complete area size of the element to be screened.
[0100] Exemplarily, according to the above embodiment, the display range of the element to be screened in the display interface is a rectangular range formed by coordinates (1, 2) and coordinates (5, -2), then the complete display range size of the element to be screened is 16, the display area size of the element to be screened is 8, and the display completeness of the element to be screened is 50%.
[0101] In an embodiment of the present application, the display range of the elements to be screened is obtained by the key point positions and sizes of the elements to be screened, and the display area size of the elements to be screened is obtained based on the display range of the elements to be screened and the display range of the display interface; the display completeness of the elements to be screened is obtained by dividing the display area size of the elements to be screened by the complete area size of the elements to be screened.
[0102] Figure 7 1 is a schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application, and the method can be executed by a processor of a vehicle terminal. Figure 1 , the step S102 can be updated to step S701 and step S702, combining Figure 7 The steps shown are explained.
[0103] Step S701: Acquire at least one to-be-selected area in the display interface; wherein no element exists in each of the to-be-selected areas.
[0104] In some embodiments, the to-be-selected area in the display interface represents an area in the display interface where no elements exist, that is, an area in the display interface that is not directly filled with text, patterns, controls, and the like.
[0105] In some embodiments, according to the above embodiments, at least one element to be filtered in the display interface and description information of at least one element to be filtered can be obtained; wherein, the description information also includes coordinate information of at least one element to be filtered; thereby, obtaining at least one selected area in the display interface includes: obtaining coordinate information of at least one element to be filtered in the display interface; determining at least one selected area in the display interface based on the coordinate information of the display interface and the coordinate information of at least one element to be filtered; it can be understood that the complement of the intersection of the coordinate information of the display interface and the coordinate information of at least one element to be filtered is taken to obtain other areas outside the area where the elements to be filtered exist in the coordinate information of the display interface.
[0106] Step S702: display the prompt information in a target area of the at least one to-be-selected area; wherein the distance between the target area and the target control is smaller than the distance between the target control and other to-be-selected areas.
[0107] In some embodiments, the target area represents an area in at least one candidate area that is easy for the user to capture; it is understandable that the target area may include at least one candidate area that is closest to the target area.
[0108] In some embodiments, the target area may further include at least one candidate area that is close to the center of the display interface.
[0109] In an embodiment of the present application, prompt information of the target control is displayed in an area that is easy for the user to capture in at least one to-be-selected area, so as to improve the accuracy of the user controlling the target control based on the prompt information, thereby improving the user experience.
[0110] In some embodiments, the display prompt information in the above embodiment can also be implemented by the following implementation methods:
[0111] Indication information is displayed in the display interface; the indication information is used to indicate the association relationship between the target control and the prompt information.
[0112] In some embodiments, the indication information may be an indication arrow between the target control and the prompt information, used to indicate to the user the position of the prompt word corresponding to the target control in the display interface, so that the user can control the operating status of the target control through the prompt information corresponding to the target control.
[0113] In some embodiments, the indication information may also be the display status of the target control and the prompt information. For example, the display brightness of the target control and the prompt word corresponding to the target control is set to highlight and flash to indicate to the user the position of the prompt word corresponding to the target control in the display interface.
[0114] In an embodiment of the present application, the indication information indicates the position of the prompt word corresponding to the user's target control in the display interface, so that the user can control the operating status of the target control through the prompt information corresponding to the target control, thereby improving the user experience.
[0115] Figure 8 The following is a schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application. The method can be executed by a processor of a vehicle terminal. The method includes steps S801 and S802. Figure 8 The steps shown are explained.
[0116] Step S801: Receive the target voice.
[0117] In some embodiments, the target voice refers to when the user interacts with the vehicle terminal by voice, the user inputs a voice control instruction for the target control to the vehicle terminal through voice input based on the prompt word corresponding to the control. For example, the prompt word of the control is a video player, and the user can input a control instruction of "open the video player" to the display interface through voice according to the prompt word, and the vehicle terminal receives the control instruction of "open the video player" input by the user.
[0118] Step S802: In response to the target voice, execute the control instruction corresponding to the target control.
[0119] In some embodiments, after the user inputs a voice control instruction for the target control to the vehicle-mounted terminal through voice input, the target control is controlled based on the voice control instruction; it can be understood that the operating state of the target control is controlled based on the voice control instruction; exemplarily, the target voice is "open the video player", then based on the control instruction of "open the video player", the control in the display interface is opened as the interface of the video player, and the interface of the video player after opening is displayed on the display interface.
[0120] In an embodiment of the present application, the user can send accurate control instructions for the target control to the vehicle terminal through voice based on the prompt word of the target control whose display completeness is less than a preset threshold, and accurately control the operating status of the target control based on the control instructions for the target control, thereby improving the user voice control display interface for controls whose display completeness is less than a preset threshold, thereby improving the user experience.
[0121] An exemplary application of an interface display method provided in an embodiment of the present application in an actual scenario is described below.
[0122] Currently, for the controls in the vehicle display interface, after the user gets in the vehicle, any control in the display interface can be controlled by voice, such as Fig. 9 As shown, Fig. 9 A schematic diagram of controls in a display interface provided in an embodiment of the present application, when the user says "switch to network interface" through voice based on the display content of network 901, the display interface displays the interface corresponding to network 901 based on the user's instructions. If AP hotspot 902 is seen, the "modify hotspot" command is said based on the display content of AP hotspot 902, and the display interface displays the hotspot modification interface.
[0123] In the related art, for controls located at the edge of the display, the display may be incomplete, and the user cannot clearly see the displayed content on the control, making it difficult to control the control by voice. Please continue to refer to Fig. 9 The AP hotspot 902 is a control located at the edge of the display interface. Only a partial area of the control AP hotspot 902 is displayed on the display interface. The user cannot accurately control the AP hotspot 902 through voice based on the content displayed by the control.
[0124] In an embodiment of the present application, when a user performs voice interaction with a display interface, prompt words corresponding to controls in the edge area of the display interface are displayed through a customized UI interface to prompt the user to perform voice control, thereby improving the accuracy of voice control.
[0125] Fig. 10A The following is a schematic diagram of an implementation flow of an interface display method provided in an embodiment of the present application. The method can be executed by a processor of a vehicle-mounted terminal. The method includes steps S1001 to S1007. Fig. 10A The steps shown are explained.
[0126] Step S1001: In response to the user waking up the voice interaction function, the boundary controls of the display interface are scanned by the system service.
[0127] In some embodiments, the user can wake up the voice interaction function of the in-vehicle display interface based on the wake-up word; for example, words such as "Hello, XX", "XX", etc., activate the voice interaction function of the in-vehicle display screen.
[0128] In some embodiments, the Application Programming Interface (API) of all elements in the current display interface can be obtained to determine whether the attributes of all elements are controllable elements, so as to determine the controls of the current display interface and the location information of the controls.
[0129] In some embodiments, it is also possible to traverse the handles of all elements in the current display interface and send messages to the elements. If the parameter returned by the element is a valid control handle, the control in the display interface and the position information of the control can be determined.
[0130] In some embodiments, the boundary controls of the display interface are scanned; it is understandable that whether the control is in the edge area of the display interface is determined based on the position of the control; for example, a coordinate system is established with the border of the display interface, and when the coordinates of the control are close to the coordinates corresponding to the border of the display interface, it is determined that the control is in the edge area of the display interface.
[0131] Step S1002: determine whether there are controls whose completeness of critical area controls is less than 80%.
[0132] In some embodiments, the display completeness of the control in the display interface may be calculated by a completeness calculation algorithm; it is understandable that the display range of the control displayed in the current display interface is determined by the coordinate information of the control and the size of the control, the size of the display area of the control displayed in the current display interface is determined according to the display range of the current display interface and the display range of the control, and the current display completeness of the control is obtained by dividing the size of the display area of the control displayed by the size of the complete display area of the control; Fig. 10B A schematic diagram of an implementation of generating control integrity provided in an embodiment of the present application, Fig. 10B In the figure, 1010 represents the display interface, 1020 represents the display area of the control in the display interface 1010, and 1030 represents the complete display area of the control. The display completeness of the control that is not fully displayed in the display interface 1010 is obtained by dividing the size of the display area 1020 of the control in the display interface by the size of the complete display area 1030 of the control.
[0133] Step S1003: If there is a control whose completeness is less than 80%, determine whether the control supports voice control.
[0134] In some embodiments, when there is a control in the display interface whose display completeness is less than 80%, it is determined whether the control has plain text and / or annotated text. If at least one of the plain text or annotated text exists, it indicates that the control supports voice control; wherein the plain text is visible to the user, and the annotated text is visible to the system but invisible to the user.
[0135] Step S1004: If there is no control with a completeness less than 80%, then end this operation.
[0136] Step S1005: If the control supports voice control, the prompt word corresponding to the control is displayed in the visible area of the display interface.
[0137] In some embodiments, when a control supports voice control, the prompt word corresponding to the control is displayed in a blank display area closest to the control in the display interface; wherein the control is connected to the nearest blank display area through indication information; please continue to refer to Fig. 10B , 1040 represents other controls with a display completeness of 100% in the display interface, and 1050 represents prompt words corresponding to controls with a display completeness of less than 80%.
[0138] Step S1006: receiving a user's voice control instruction for the control.
[0139] In some embodiments, the user can input voice control instructions for the control to the display interface through voice according to the prompt word corresponding to the control. For example, the prompt word of the control is video player, then the user can input control instructions for opening the video player to the display interface through voice according to the prompt word.
[0140] Step S1007: Control the control based on the voice control instruction.
[0141] In some embodiments, the display interface controls the operating state of the control based on the voice control command input by the user for the control; illustratively, the video player is opened by an instruction to open the video player.
[0142] In an embodiment of the present application, in response to the voice interaction function being awakened, a control located in the edge area of the display interface is obtained, and when the display completeness of the control located in the edge area of the display interface is less than 80%, a prompt word corresponding to the control is displayed in the displayable area of the display interface. Thus, the user can perform voice control on the control at the edge of the display based on the prompt word, thereby improving the accuracy of voice control and improving the user experience.
[0143] Based on the foregoing embodiments, an embodiment of the present application provides an interface display device, which includes the units included and the modules included in the units, and can be implemented by a processor in a vehicle-mounted terminal; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.
[0144] Fig.11 A schematic diagram of the structure of an interface display device provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, the interface display device 1100 includes: a screening module 1101 and a display module 1102, wherein: the screening module 1101 is used to screen the elements in the display interface based on a first preset condition and a second preset condition to obtain a target control; the target control is an element in the display interface that satisfies the first preset condition and the second preset condition; the display module 1102 is used to display the prompt information corresponding to the target control in the target area of the display interface; the prompt information is used to prompt the user to control the target control using a target voice; wherein, the first preset condition indicates that the target control supports voice control, and the second preset condition indicates that the display completeness of the target control is less than a preset threshold.
[0145] In some embodiments, when filtering elements in a display interface based on a first preset condition; the filtering module 1101 is also used to obtain each element to be filtered in the display interface and description information of each element to be filtered; obtain the elements to be filtered that have target features in the description information; wherein the presence of the target features in the description information of the element indicates that the element supports voice control.
[0146] In some embodiments, the target feature includes at least one of the following: plain text information, annotation information; the filtering module 1101 is also used to obtain the elements to be filtered that have plain text information in the description information; the plain text information is visible to the user; and / or, obtain the elements to be filtered that have annotation information in the description information; the annotation information is not visible to the user.
[0147] In some embodiments, when filtering elements in the display interface based on a second preset condition, the filtering module 1101 is also used to obtain each element to be filtered in the display interface and the display completeness of each element to be filtered; and obtain the elements to be filtered whose display completeness is less than a preset threshold.
[0148] In some embodiments, the filtering module 1101 is also used to obtain the key point position of each element to be filtered and the size of each element to be filtered in the display interface; when the key point position and the size indicate that the element to be filtered is located in the edge area of the display interface, the display completeness of the element to be filtered is obtained.
[0149] In some embodiments, the screening module 1101 is also used to obtain the key point position of each element to be screened in the display interface and the size of each element to be screened; based on the key point position and the size, determine the display range of the element to be screened in the screen coordinate system; based on the display range of the screen and the display range of the element to be screened, determine the display area size of the element to be screened; based on the complete area size of the element to be screened and the display area size, determine the display completeness of the element to be screened.
[0150] In some embodiments, the display module 1102 is also used to obtain at least one area to be selected in the display interface; wherein no elements exist in each of the areas to be selected; and the prompt information is displayed in a target area of the at least one area to be selected; wherein the distance between the target area and the target control is smaller than the distance between the target control and other areas to be selected.
[0151] In some embodiments, the display module 1102 is further used to display indication information in the display interface; the indication information is used to indicate the association relationship between the target control and the prompt information.
[0152] In some embodiments, the interface display device 1100 further includes an execution module (not shown in the figure) for receiving the target voice; and executing a control instruction corresponding to the target control in response to the target voice.
[0153] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiment of the present application can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.
[0154] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The software product is stored in a storage medium, including several instructions to enable a vehicle-mounted terminal (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.
[0155] An embodiment of the present application provides a vehicle-mounted terminal, including a memory and a processor, wherein the memory stores instructions that can be executed on the processor, and when the processor executes the instructions, some or all of the steps in the above method are implemented.
[0156] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.
[0157] An embodiment of the present application provides a computer instruction, including a computer-readable code. When the computer-readable code is executed in a vehicle-mounted terminal, a processor in the vehicle-mounted terminal executes some or all of the steps in the above method.
[0158] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.
[0159] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.
[0160] Fig.12 A schematic diagram of a hardware entity of a vehicle-mounted terminal provided in an embodiment of the present application is shown in FIG. Fig.12 As shown, the hardware entity of the vehicle terminal 1200 includes: a processor 1201 and a memory 1202, wherein the memory 1202 stores instructions that can be executed on the processor 1201, and the processor 1201 implements the steps in the method of any of the above embodiments when executing the instructions.
[0161] The memory 1202 stores instructions that can be executed on the processor. The memory 1202 is configured to store instructions and applications executable by the processor 1201. It can also cache data to be processed or processed by the processor 1201 and various modules in the vehicle terminal 1200 (for example, image data, audio data, voice communication data, and video communication data). This can be achieved through flash memory (FLASH) or random access memory (Random Access Memory, RAM).
[0162] When the processor 1201 executes the instructions, the steps of any of the above methods are implemented. The processor 1201 generally controls the overall operation of the vehicle terminal 1200.
[0163] An embodiment of the present application provides a computer storage medium, which stores one or more instructions. The one or more instructions can be executed by one or more processors to implement the steps of the method of any of the above embodiments.
[0164] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.
[0165] The processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It is understandable that the electronic device that implements the functions of the processor may also be other, and the embodiments of the present application are not specifically limited.
[0166] The above-mentioned computer storage medium / memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and the like; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.
[0167] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.
[0168] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0169] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0170] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0171] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art can understand that all or part of the steps of the above method embodiments can be completed by hardware related to program instructions, and the above program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above method embodiments are executed; and the above storage medium includes: mobile storage devices, read-only memory (ROM), disks or optical disks, etc. Various media that can store program codes.
[0172] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a vehicle-mounted terminal (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.
[0173] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.
Claims
1. An interface display method, characterized in that: The method comprises: Based on the first preset condition and the second preset condition, the elements in the display interface are screened to obtain a target control; the target control is an element in the display interface that meets the first preset condition and the second preset condition; Displaying prompt information corresponding to the target control in the target area of the display interface; the prompt information is used to prompt the user to control the target control using the target voice; Among them, the first preset condition indicates that the target control supports voice control, and the second preset condition indicates that the display completeness of the target control is less than a preset threshold.
2. The method according to claim 1, characterized in that When the elements in the display interface are screened based on the first preset condition, the following steps are included: Acquire each to-be-screened element in the display interface and description information of each to-be-screened element; An element to be screened having a target feature in its description information is obtained; wherein the presence of the target feature in the description information of the element indicates that the element supports voice control.
3. The method according to claim 2, characterized in that The target feature includes at least one of the following: plain text information and annotation information; the elements to be screened that have the target feature in the acquired description information include: Obtaining elements to be screened that contain plain text information in the description information; the plain text information is visible to the user; and / or, Obtain elements to be screened that have annotation information in the description information; the annotation information is invisible to the user.
4. The method according to claim 1, characterized in that When the elements in the display interface are screened based on the second preset condition, it includes: Obtaining each to-be-screened element in the display interface and the display completeness of each to-be-screened element; Get the elements to be filtered whose display completeness is less than the preset threshold.
5. The method according to claim 4, characterized in that The obtaining of each to-be-screened element in the display interface and the display completeness of each to-be-screened element includes: Obtaining the key point position and size of each element to be screened in the display interface; When the key point position and the size indicate that the element to be screened is located in an edge region of the display interface, the display completeness of the element to be screened is obtained.
6. The method according to claim 4, characterized in that The obtaining of the display completeness of each of the elements to be screened includes: Acquire the key point position of each to-be-screened element in the display interface and the size of each to-be-screened element; determine the display range of the to-be-screened element in the screen coordinate system based on the key point position and the size; Determining the display area size of the element to be screened based on the display range of the screen and the display range of the element to be screened; The display completeness of the element to be screened is determined based on the complete area size of the element to be screened and the display area size.
7. The method according to claim 1, characterized in that The step of displaying the prompt information corresponding to the target control in the target area of the display interface includes: Acquire at least one area to be selected in the display interface; wherein no element exists in each of the areas to be selected; The prompt information is displayed in a target area of the at least one to-be-selected area; wherein the distance between the target area and the target control is smaller than the distance between the target control and other to-be-selected areas.
8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Indication information is displayed in the display interface; the indication information is used to indicate the association relationship between the target control and the prompt information.
9. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: receiving the target speech; In response to the target voice, a control instruction corresponding to the target control is executed.
10. An interface display device, characterized in that: The device comprises: A screening module, used to screen the elements in the display interface based on a first preset condition and a second preset condition to obtain a target control; the target control is an element in the display interface that meets the first preset condition and the second preset condition; A display module is used to display the prompt information corresponding to the target control in the target area of the display interface; the prompt information is used to prompt the user to control the target control using a target voice; wherein the first preset condition indicates that the target control supports voice control, and the second preset condition indicates that the display completeness of the target control is less than a preset threshold.
11. A vehicle-mounted terminal, characterized in that: The vehicle-mounted terminal includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps in the method described in any one of claims 1 to 9.
12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 9 are implemented.