Vehicle-mounted central control interaction method and device and vehicle

Abstract

The invention discloses a vehicle-mounted central control interaction method and device and a vehicle, and belongs to the technical field of vehicle-mounted central control interaction.The vehicle-mounted central control interaction method comprises the steps that after a vehicle machine system startup signal is received, clickable elements of a current display page are obtained, page element information is obtained, and the page element information is registered into a voice hot word engine container; after a voice instruction issued by a user is obtained, according to the voice instruction issued by the user, page element information is matched through the hot word engine container, and a click event of the specific position coordinates of the control is executed through barrier-free service. According to the vehicle-mounted central control interaction method and device and the vehicle provided by the invention, retrieval and event clicking of page visible elements are carried out by combining voice with barrier-free service, and a user can operate a related interface of a multimedia application under the condition of driving, so that the driving safety is ensured, the convenience of the user is reflected, and the user experience is improved. And the driving experience of the vehicle is improved.

Description

Technical Field

[0001] The invention discloses an in-vehicle central control interaction method, a device and a vehicle, and belongs to the technical field of in-vehicle central control interaction. Background Art

[0002] With the rapid development of the Internet of Vehicles, in-vehicle multimedia has become an indispensable and important function. Listening to music, radio, and radio are all high-frequency applications. It is very unsafe and not recommended for users to manually operate these functions while driving. Currently, all vehicles have voice functions, but the voice function currently controls some common functions, such as turning on some car control functions, playing the next song, pausing the multimedia, etc. However, if you click on some song lists, song details, a song fixed on the play page, or a channel on a fixed radio station on the multimedia application, you can only click manually. Summary of the invention

[0003] In view of the defects of the prior art, the present invention proposes an interactive method, device and vehicle for a vehicle-mounted central control, which retrieves and identifies elements on the screen and simulates click events through barrier-free services.

[0004] The technical solution of the present invention is as follows:

[0005] According to a first aspect of an embodiment of the present invention, there is provided a vehicle-mounted central control interaction method, comprising:

[0006] When a vehicle system power-on signal is received, the clickable elements of the current display page are obtained to obtain page element information, and the page element information is registered in the speech hot word engine container;

[0007] After the voice command issued by the user is obtained, the page element information is matched through the hot word engine container according to the voice command issued by the user, and the accessibility service is used to execute the click event of the specific position coordinates of the control.

[0008] Preferably, the step of acquiring clickable elements of the currently displayed page to obtain page element information and registering the page element information into a speech hot word engine container comprises:

[0009] Initialize the accessibility service scanning parameters and obtain the clickable elements of the current display page;

[0010] Identify and scan the characteristics of element nodes to obtain page element information;

[0011] A corresponding sub-node set is obtained according to the page element information, and the sub-node sets corresponding to all pages are registered in a hot word engine container.

[0012] Preferably, the page element information includes: text attribute information, category attribute information and id attribute information of the window element node and the control element node.

[0013] Preferably, the step of matching page element information through a hotword engine container according to the voice instruction issued by the user and using an accessibility service to execute a click event of a specific position coordinate of a control includes:

[0014] Recognize the voice command issued by the user to obtain voice text information;

[0015] According to the voice text information, the hot word engine container uses a string similarity algorithm to obtain the control page element information and position coordinates to be clicked;

[0016] According to the page element information and position coordinates of the control to be clicked, an accessibility service is used to execute a click event of the specific position coordinates of the control.

[0017] According to a second aspect of an embodiment of the present invention, there is provided an interactive device for a vehicle-mounted central control, including:

[0018] A registration module, which is used to obtain clickable elements of the current display page to obtain page element information after receiving a startup signal of the vehicle system, and register the page element information into the hot word engine container of the voice;

[0019] The execution module is used to, after obtaining the voice command issued by the user, match the page element information through the hot word engine container according to the voice command issued by the user and use the accessibility service to execute the click event of the specific position coordinates of the control.

[0020] Preferably, the registration module is used to:

[0021] Initialize the accessibility service scanning parameters and obtain the clickable elements of the current display page;

[0022] Identify and scan the characteristics of element nodes to obtain page element information;

[0023] A corresponding sub-node set is obtained according to the page element information, and the sub-node sets corresponding to all pages are registered in a hot word engine container.

[0024] Preferably, the execution module is used to:

[0025] Recognize the voice command issued by the user to obtain voice text information;

[0026] According to the voice text information, the hot word engine container uses a string similarity algorithm to obtain the control page element information and position coordinates to be clicked;

[0027] According to the page element information and position coordinates of the control to be clicked, an accessibility service is used to execute a click event of the specific position coordinates of the control.

[0028] According to a third aspect of an embodiment of the present invention, there is provided a vehicle, comprising:

[0029] one or more processors;

[0030] a memory for storing the one or more processor-executable instructions;

[0031] Wherein, the one or more processors are configured to:

[0032] Execute the method described in the first aspect of the embodiment of the present invention.

[0033] According to a fourth aspect of an embodiment of the present invention, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the method described in the first aspect of the embodiment of the present invention.

[0034] According to a fifth aspect of the embodiments of the present invention, an application product is provided. When the application product is running on a terminal, the terminal executes the method described in the first aspect of the embodiments of the present invention.

[0035] The beneficial effects of the present invention are:

[0036] The present invention provides an interactive method, device and vehicle for an in-vehicle central control, which retrieves visible elements of a page and clicks on events through voice combined with barrier-free services. A user can operate relevant interfaces of multimedia applications while driving, which not only ensures driving safety, but also reflects user convenience and improves the driving experience of the vehicle.

[0037] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 The present invention is a flow chart of an in-vehicle central control interaction method according to an exemplary embodiment.

[0039] Figure 2 The present invention is a schematic block diagram of the structure of an interactive device for a vehicle-mounted central control according to an exemplary embodiment.

[0040] Figure 3 The present invention is a schematic block diagram of a vehicle structure according to an exemplary embodiment. DETAILED DESCRIPTION

[0041] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they cannot be understood as limitations on the present invention.

[0043] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0044] An embodiment of the present invention provides an in-vehicle central control interaction method implemented by a terminal, and the terminal at least includes a CPU and the like.

[0045] Embodiment 1: Figure 1 A vehicle-mounted central control interaction method is shown according to an exemplary embodiment, including:

[0046] Step S101, after receiving the vehicle system power-on signal, obtain the clickable elements of the current display page to obtain page element information, and register the page element information into the speech hot word engine container. The specific steps are as follows:

[0047] Before receiving the power-on signal of the vehicle system, each configured attribute information also requires each application to do some corresponding support. Each control that needs to be clicked needs to be configured with the ContentDescription attribute, that is, the specific text attribute information. For example, for the play button, configure the play and continue playing attributes. The list is the specific text content to be displayed returned by the configuration interface. Initialize the accessibility service scanning parameters, and continuously obtain the elements on the vehicle window page. When a page is opened and displayed, after obtaining the interface window changes, the node of each clickable control will be obtained through the accessibility service, and the configuration attributes and other information attributes of the clickable control can be identified;

[0048] That is, the characteristics of the scanned element nodes are identified to obtain the page element information, which includes text attribute information, category attribute information and id attribute information of the window element nodes and the control element nodes.

[0049] The window node and the control node are obtained according to the page element information, the corresponding child node set is found through the text information displayed on the page, and the clickable element information scanned from all pages is registered in a hot word engine container, so that after receiving the voice command, the issued command information can be accurately matched from the hot word engine.

[0050] Step S102, after obtaining the voice command issued by the user, according to the voice command issued by the user, the accessibility service is used to match the page element information through the hot word engine container to execute the click event of the specific position coordinates of the control, and the specific steps are as follows:

[0051] The user sends a voice command through the microphone, and the Android system audio acquisition is used to convert it into an audio stream, and then the audio is processed by noise reduction, compression, encoding, etc. The voice recognition engine then decodes and recognizes the audio stream and maps it into text information. When converted into text information, the voice will also be generalized into text. For example, the generalization process is that the word "play" may be generalized into two phrases, "continue playing" and "play". Through generalization, the user's specific intention can be more accurately identified, so that the execution efficiency is higher when matching. The voice text information is obtained by recognizing the voice command sent by the user.

[0052] According to the voice text information, the hot word engine container uses a string similarity algorithm to obtain the page element information and position coordinates of the control to be clicked. The string similarity algorithm is used to calculate the minimum number of edit operations between two strings to convert one string into another. The smaller the edit distance, the more similar the two strings are. According to the page element information and position coordinates of the control to be clicked, an accessibility service is used to execute a click event of the specific position coordinates of the control.

[0053] Embodiment 2: Figure 2 An interactive device for a vehicle-mounted central control according to an exemplary embodiment includes:

[0054] The registration module 210 is used to obtain the clickable elements of the current display page to obtain page element information after receiving the vehicle system startup signal, and register the page element information into the speech hot word engine container;

[0055] The execution module 220 is used to, after obtaining the voice command issued by the user, match the page element information through the hot word engine container according to the voice command issued by the user and use the accessibility service to execute the click event of the specific position coordinates of the control.

[0056] Preferably, the registration module 210 is used to:

[0057] Initialize the accessibility service scanning parameters and obtain the clickable elements of the current display page;

[0058] Identify and scan the characteristics of element nodes to obtain page element information;

[0059] A corresponding sub-node set is obtained according to the page element information, and the sub-node sets corresponding to all pages are registered in a hot word engine container.

[0060] Preferably, the execution module 220 is used to:

[0061] Recognize the voice command issued by the user to obtain voice text information;

[0062] According to the voice text information, the hot word engine container uses a string similarity algorithm to obtain the control page element information and position coordinates to be clicked;

[0063] According to the page element information and position coordinates of the control to be clicked, an accessibility service is used to execute a click event of the specific position coordinates of the control.

[0064] This application uses voice combined with accessibility services to retrieve visible elements on the page and click on events. Users can operate the relevant interfaces of multimedia applications while driving, which not only ensures driving safety, but also reflects the convenience of users and improves the driving experience of the vehicle.

[0065] Embodiment three: Figure 3 300 is a block diagram of a vehicle 300 provided in an embodiment of the present application. For example, the vehicle 300 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles. The vehicle 300 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle. The vehicle 300 may also be equipped with a brake-by-wire system.

[0066] Reference Figure 3 , the vehicle 300 may include various subsystems, for example, an infotainment system 310, a perception system 320, a decision control system 330, a drive system 340, and a computing platform 350. The vehicle 300 may also include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and each component of the vehicle 300 may be interconnected by wire or wireless means.

[0067] In some embodiments, the infotainment system 310 may include a communication system, an entertainment system, and a navigation system, among others.

[0068] The perception system 320 may include several sensors for sensing information about the environment around the vehicle 300. For example, the perception system 320 may include a global positioning system (the global positioning system may be a GPS system, or a Beidou system or other positioning systems), an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar, and a camera.

[0069] The decision control system 330 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0070] The drive system 340 may include components that provide power movement for the vehicle 300. In one embodiment, the drive system 340 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of an internal combustion engine, an electric motor, and an air compression engine. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0071] Some or all functions of the vehicle 300 are controlled by a computing platform 350. The computing platform 350 may include at least one processor 351 and a memory 352, and the processor 351 may execute instructions 353 stored in the memory 352.

[0072] The processor 351 may be any conventional processor, such as a commercially available CPU. The processor may also include a graphics processor (Graphic Process Unit, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), a system on chip (System on Chip, SOC), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or a combination thereof.

[0073] The memory 352 may be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0074] In addition to instructions 353 , memory 352 may also store data, such as road maps, route information, vehicle location, direction, speed, etc. The data stored in memory 352 may be used by computing platform 350 .

[0075] In the embodiment of the present disclosure, the processor 351 can execute the instruction 353 to complete the above-mentioned vehicle-mounted central control interactive method.

[0076] All or part of the steps.

[0077] Embodiment 4: In an exemplary embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the program is executed by a processor, an interactive method of a vehicle-mounted central control provided in all the inventive embodiments of the present application is implemented.

[0078] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0079] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0080] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0081] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0082] Embodiment 5: In an exemplary embodiment, an application product is also provided, including one or more instructions, which can be executed by the processor 351 of the above-mentioned device to complete the above-mentioned vehicle-mounted central control interaction method.

[0083] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.

Claims

1. An interactive method for a vehicle-mounted central control, characterized in that: include: When a vehicle system power-on signal is received, the clickable elements of the current display page are obtained to obtain page element information, and the page element information is registered in the speech hot word engine container; After the voice command issued by the user is obtained, the page element information is matched through the hot word engine container according to the voice command issued by the user, and the accessibility service is used to execute the click event of the specific position coordinates of the control.

2. The vehicle-mounted central control interactive method according to claim 1, characterized in that: The step of obtaining clickable elements of the currently displayed page to obtain page element information and registering the page element information into the speech hot word engine container comprises: Initialize the accessibility service scanning parameters and obtain the clickable elements of the current display page; Identify and scan the characteristics of element nodes to obtain page element information; A corresponding sub-node set is obtained according to the page element information, and the sub-node sets corresponding to all pages are registered in a hot word engine container.

3. The in-vehicle central control interactive method according to claim 2, characterized in that: The page element information includes: text attribute information, category attribute information and id attribute information of the window element node and the control element node.

4. The in-vehicle central control interactive method according to claim 3, characterized in that: The method of matching page element information through a hotword engine container according to the voice command issued by the user and using an accessibility service to execute a click event of a specific position coordinate of a control includes: Recognize the voice command issued by the user to obtain voice text information; According to the voice text information, the hot word engine container uses a string similarity algorithm to obtain the control page element information and position coordinates to be clicked; According to the page element information and position coordinates of the control to be clicked, an accessibility service is used to execute a click event of the specific position coordinates of the control.

5. An interactive device for a vehicle-mounted central control, characterized in that: include: A registration module, which is used to obtain clickable elements of the current display page to obtain page element information after receiving a startup signal of the vehicle system, and register the page element information into the hot word engine container of the voice; The execution module is used to, after obtaining the voice command issued by the user, match the page element information through the hot word engine container according to the voice command issued by the user and use the accessibility service to execute the click event of the specific position coordinates of the control.

6. The interactive device for a vehicle-mounted central control according to claim 5, characterized in that: The registration module is used to: Initialize the accessibility service scanning parameters and obtain the clickable elements of the current display page; Identify and scan the characteristics of element nodes to obtain page element information; A corresponding sub-node set is obtained according to the page element information, and the sub-node sets corresponding to all pages are registered in a hot word engine container.

7. The interactive device for a vehicle-mounted central control according to claim 5, characterized in that: The execution module is used to: Recognize the voice command issued by the user to obtain voice text information; According to the voice text information, the hot word engine container uses a string similarity algorithm to obtain the control page element information and position coordinates to be clicked; According to the page element information and position coordinates of the control to be clicked, an accessibility service is used to execute a click event of the specific position coordinates of the control.

8. A vehicle, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the steps of the interaction method of a new energy vehicle-mounted central control as described in any one of claims 1 to 4.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of an interactive method for a vehicle-mounted central control as described in any one of claims 1 to 4 are implemented.

Images