Visible-to-talk control method and system adaptive to vehicle-mounted infotainment system, and electronic equipment
By monitoring the application page lifecycle and control hierarchy, and combining this with voice command parsing, we have achieved fast and accurate voice control operation, solving the problems of high development costs and poor compatibility in existing technologies, and improving response speed and information accuracy.
Patent Information
- Application Number
- CN202511779807.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-28
- Publication Date
- 2026-02-27
AI Technical Summary
Existing solutions require each application to integrate an SDK, resulting in high development costs, poor compatibility, insufficient page control information, and slow response speed.
By listening to the lifecycle of the application page, obtaining page hierarchy information, scanning the hierarchical tree structure of controls, and combining voice command parsing, the system accurately matches and executes the target control operation. The core logic is concentrated at the system layer, and no adaptation code is required at the application layer.
It enables fast-response voice control operation, reduces development costs, improves compatibility and information accuracy, and simplifies the application development process.
Smart Images
Figure CN121583245A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of vehicle infotainment system control, and in particular to a visible-to-talk control method adapted to vehicle infotainment systems, a visible-to-talk control system adapted to vehicle infotainment systems, electronic devices, and storage media. Background Technology
[0002] In the current automotive technology field, common "see-and-say" solutions typically require application integration SDKs, and each application needs to be customized to work with the SDK for various scenarios. This leads to the following drawbacks:
[0003] 1. Each application needs to be adapted, resulting in high development costs.
[0004] 2. Using the SDK, the scanning information for page controls is not detailed or accurate enough.
[0005] 3. The response speed is slow, from collecting page information to executing voice commands.
[0006] 4. Poor compatibility with third-party applications
[0007] Typical Case:
[0008] Developing a "visible and speakable" SDK typically requires each application to integrate the SDK and adapt its interfaces, resulting in a larger application package size. In addition to developing its own business logic, the application also needs to accommodate the "visible and speakable" SDK, increasing development costs. Summary of the Invention
[0009] In view of this, the purpose of this invention is to provide a visible-to-talk control method adapted to vehicle infotainment systems, a visible-to-talk control system adapted to vehicle infotainment systems, electronic devices and storage media, aiming to solve the technical problems of high development cost, poor compatibility and insufficient detail of page control information in visible-to-talk services.
[0010] This invention provides the following solution:
[0011] According to one aspect of this application, a visible-and-talk control method adapted to a vehicle infotainment system is provided, comprising the following steps:
[0012] Monitor the lifecycle of application pages to obtain page hierarchy information;
[0013] Based on the page hierarchy information, obtain the core information of the page;
[0014] The core information of the page includes: page unique identifier (id), page type, screen unique identifier, user unique identifier, page level, application package name, and page mode;
[0015] Scan the hierarchical tree structure of the application page controls, listen for changes in control elements, obtain the core information of the controls through the hierarchical tree structure and report it to the cloud server. The core information of the controls includes: control unique identifier id, control type, control enabled status, list item badge, combo control status, control text, control position, control size and the bound parent control.
[0016] The system acquires the voice commands of the target user, performs semantic parsing on the voice commands, extracts the command intent and key parameters, and reports them to the cloud server.
[0017] The cloud server, based on the core information of the page and the core information of the control, combined with the instruction intent and key parameters, matches the target page and the target control, generates a voice execution instruction including the target page ID, the target control ID, the instruction intent and key parameters, and sends it to the vehicle system.
[0018] The vehicle system locates the target page based on the target page ID, matches the target control based on the target control ID, and calls the corresponding execution method to complete the instruction execution according to the instruction intent.
[0019] Furthermore, including:
[0020] The target page hierarchy information is obtained through the framework layer. Specifically, the voice command controller manages the collection, updating, and reporting of information for all pages within the target application process.
[0021] Furthermore, including:
[0022] Scan the hierarchical tree structure of application page controls through the framework layer;
[0023] The voice command controller collects, updates, and reports the core information of controls on all pages.
[0024] Furthermore, including:
[0025] The hierarchical tree structure of the application page controls includes:
[0026] Maintain a control pool for global visualization components, recursively traverse the view tree to extract core control information; update the corresponding information of a control when its text changes, progress changes, or its on / off state changes.
[0027] Furthermore, including:
[0028] The execution method invoked according to the instruction intent includes:
[0029] If the instruction is intended to be a click, then a touch event is simulated to trigger a click operation on the target control;
[0030] If the instruction is to slide, the standard control calls the system's sliding interface, and the custom control simulates the sliding trajectory.
[0031] If the instruction is intended to adjust the progress, the standard progress control calls the control interface, and the custom progress control simulates sliding to the target progress value;
[0032] If the instruction is intended to input text, then the target control's control interface is called to pass in the text content;
[0033] If the instruction is intended to select a list item, then the list item is located based on its index, and a touch event is simulated to complete the selection.
[0034] Furthermore, including:
[0035] Based on the voice command service, the system receives page and control information reported by each application process through the voice command controller, integrates it, and reports it to the cloud in real time.
[0036] Furthermore, including:
[0037] The vehicle system filters the set of pages belonging to the currently logged-in user based on the user's unique identifier, excluding private pages belonging to other users;
[0038] Locate the target page based on the target page ID.
[0039] Furthermore, including:
[0040] After the vehicle's infotainment system executes the command using the corresponding execution method,
[0041] If the command is a click or list selection, then verify whether the target control triggers the preset response;
[0042] If the task involves adjusting the progress or inputting text commands, then verify that the progress value or text content of the target control matches the key parameters.
[0043] Once the verification is complete, the vehicle system will broadcast the verification result to the target user via the voice module, and simultaneously report the execution result to the cloud server for log storage.
[0044] Furthermore, including:
[0045] The voice command controller uses a singleton pattern to maintain a list of ViewRootImpl objects within the process. When a ViewRootImpl object is added, the addVoicePageInfo() method is called to collect page information; when it is removed, the removeVoicePageInfo() method is called to delete the page information.
[0046] And through tryConnectToServiceLocked()
[0047] The method is bound to the VoiceCommandService to implement information reporting.
[0048] According to two aspects of this application, a visible-and-say control system adapted to an in-vehicle infotainment system is provided, comprising:
[0049] The module includes a page information acquisition module, a control information acquisition module, a voice command acquisition module, a cloud matching and distribution module, and a terminal execution module.
[0050] The page information acquisition module is used to monitor the lifecycle of application pages and obtain page hierarchy information;
[0051] Used to obtain core page information based on page hierarchy information;
[0052] The core information of the page includes: page unique identifier (id), page type, screen unique identifier, user unique identifier, page level, application package name, and page mode;
[0053] The control information acquisition module is used to scan the hierarchical tree structure of the application page controls and listen for changes in control elements;
[0054] Used to transmit core information through a hierarchical tree structure control to the cloud server;
[0055] The core information of the control includes: the control's unique identifier (id), control type, control enabled status, list item badge, combo control status, control text, control position, control size, and the bound parent control;
[0056] The voice command acquisition module is used to acquire the voice commands of the target user, perform semantic parsing on the voice commands, extract the command intent and key parameters and report them to the cloud server.
[0057] The cloud matching and delivery module is used by the cloud server to match the target page and target control based on the core information of the page and the core information of the control, combined with the instruction intent and key parameters, to generate a voice execution instruction including the target page ID, the target control ID, the instruction intent and key parameters, and then deliver it to the vehicle system.
[0058] The terminal execution module is used to locate the target page based on the target page ID, match the target control based on the target control ID, and call the corresponding execution method to complete the instruction execution according to the instruction intent.
[0059] According to three aspects of the present invention, an electronic device is provided, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;
[0060] The memory stores a computer program, which, when executed by the processor, causes the processor to perform steps of a visible-and-say control method adapted to the vehicle system.
[0061] According to four aspects of the present invention, a computer-readable storage medium is provided that stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a visible-and-say control method adapted to a vehicle infotainment system.
[0062] Compared with the prior art, the present invention has the following advantages:
[0063] This application significantly speeds up the entire response time from user-initiated voice commands to control execution within a preset threshold time, resulting in a substantial speedup compared to traditional SDK solutions; page / control information updates are delayed without waiting for a full scan.
[0064] This application further improves efficiency by integrating the development of multiple applications into a single system module.
[0065] This application extends the functionality from only applications that have an SDK-compatible application to all Android applications having the "visible and speak" feature.
[0066] This application makes the instruction control process more precise by accurately capturing various information of page controls, including but not limited to: screen ID, user ID, page level, page type, control type, control size, and control text.
[0067] This application focuses its core logic at the system layer, with no adaptation code at the application layer, enabling one-time development and full application reuse.
[0068] Communicating through a standard interface ensures strong interface compatibility, and subsequent iterations will not affect the information reporting and command execution of existing applications.
[0069] This application uses a system module to scan all application page information and uploads the scanned information to the cloud. The cloud then parses the user's voice commands, matches them with the vehicle's infotainment page controls, and accurately issues the corresponding voice commands to complete the user's voice control of the vehicle's infotainment system. The system module is primarily responsible for scanning all visible page information and executing voice commands issued from the cloud, without requiring any involvement from individual applications. Attached Figure Description
[0070] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0071] Figure 1 This is a flowchart of a visible-to-talk control method adapted to a vehicle infotainment system provided by one or more embodiments of the present invention.
[0072] Figure 2 This is a structural diagram of a visible-to-talk control system adapted to a vehicle infotainment system provided by one or more embodiments of the present invention.
[0073] Figure 3 This is an architecture diagram of a visible-to-talk control system adapted to an in-vehicle infotainment system, which is a specific embodiment of the present invention.
[0074] Figure 4 This is a schematic diagram illustrating the collection of page information according to a specific embodiment of the present invention.
[0075] Figure 5 This is a schematic diagram illustrating the collection of page information according to a specific embodiment of the present invention.
[0076] Figure 6 This is a schematic diagram of voice command execution according to a specific embodiment of the present invention.
[0077] Figure 7 This is a block diagram of an electronic device structure for a visible-to-talk control method adapted to a vehicle infotainment system, provided by one or more embodiments of the present invention. Detailed Implementation
[0078] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0079] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.
[0080] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.
[0081] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.
[0082] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”
[0083] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.
[0084] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.
[0085] Figure 1 This is a flowchart of a visible-to-talk control method adapted to a vehicle infotainment system provided by one or more embodiments of the present invention.
[0086] like Figure 1 As shown, it includes the following steps:
[0087] Step S1: Listen to the lifecycle of the application page and obtain page hierarchy information;
[0088] Based on the page hierarchy information, obtain the core information of the page;
[0089] The core information of the page includes: page unique identifier (id), page type, screen unique identifier, user unique identifier, page level, application package name, and page mode;
[0090] Specifically, the target page hierarchy information is obtained through the framework layer. In particular, the voice command controller manages the collection, updating, and reporting of information for all pages within the target application process.
[0091] By using the page's unique ID, the currently displayed target page can be quickly located, avoiding incorrect command pointing and further improving the accuracy of page positioning.
[0092] Step S2: Scan the hierarchical tree structure of the application page controls, listen for changes in control elements, obtain the core information of the controls through the hierarchical tree structure and report it to the cloud server. The core information of the controls includes: control unique identifier id, control type, control enabled status, list item badge, combined control status, control text, control position, control size and bound parent control.
[0093] Specifically, the hierarchical tree structure of application page controls is scanned through the framework layer;
[0094] The voice command controller collects, updates, and reports the core information of controls on all pages.
[0095] The hierarchical tree structure of the application page controls includes:
[0096] Maintain a control pool for global visualization components, recursively traverse the view tree to extract core control information; update the corresponding information of a control when its text changes, progress changes, or its on / off state changes.
[0097] Step S3: Obtain the voice command of the target user, perform semantic parsing on the voice command, extract the command intent and key parameters and report them to the cloud server;
[0098] Step S4: The cloud server matches the target page and target control based on the core information of the page and the core information of the control, combined with the instruction intent and key parameters, generates a voice execution instruction including the target page ID, the target control ID, the instruction intent and key parameters, and sends it to the vehicle system.
[0099] In step S5, the vehicle system locates the target page based on the target page ID, matches the target control based on the target control ID, and calls the corresponding execution method to complete the instruction execution according to the instruction intent.
[0100] Furthermore, including:
[0101] The execution method invoked according to the instruction intent includes:
[0102] If the instruction is intended to be a click, then a touch event is simulated to trigger a click operation on the target control;
[0103] If the instruction is to slide, the standard control calls the system's sliding interface, and the custom control simulates the sliding trajectory.
[0104] If the instruction is intended to adjust the progress, the standard progress control calls the control interface, and the custom progress control simulates sliding to the target progress value;
[0105] If the instruction is intended to input text, then the target control's control interface is called to pass in the text content;
[0106] If the instruction is intended to select a list item, then the list item is located based on its index, and a touch event is simulated to complete the selection.
[0107] Furthermore, including:
[0108] Based on the voice command service, the system receives page and control information reported by each application process through the voice command controller, integrates it, and reports it to the cloud in real time.
[0109] Furthermore, including:
[0110] The vehicle system filters the set of pages belonging to the currently logged-in user based on the user's unique identifier, excluding private pages belonging to other users;
[0111] Locate the target page based on the target page ID.
[0112] Furthermore, including:
[0113] After the vehicle's infotainment system executes the command using the corresponding execution method,
[0114] If the command is a click or list selection, then verify whether the target control triggers the preset response;
[0115] If the task involves adjusting the progress or inputting text commands, then verify that the progress value or text content of the target control matches the key parameters.
[0116] Once the verification is complete, the vehicle system will broadcast the verification result to the target user via the voice module, and simultaneously report the execution result to the cloud server for log storage.
[0117] Furthermore, including:
[0118] The voice command controller uses a singleton pattern to maintain a list of ViewRootImpl objects within the process. When a ViewRootImpl object is added, the addVoicePageInfo() method is called to collect page information; when it is removed, the removeVoicePageInfo() method is called to delete the page information.
[0119] And through tryConnectToServiceLocked()
[0120] The method is bound to the VoiceCommandService to implement information reporting.
[0121] Specifically, ensure that there is only one instance of VoiceCommandController in the application process by maintaining the root node of the ViewRootImpl view hierarchy, which is responsible for communicating with...
[0122] The WindowManagerService handles communication and core logic such as view drawing and touch event dispatching.
[0123] By tracking all active windows in the current application process, we ensure that the collected page information is comprehensive and real-time.
[0124] When windows are added or removed, the page information is quickly updated via a list to avoid missing or retaining invalid data.
[0125] List management logic:
[0126] When added: addVoicePageInfo() is triggered when ViewRootImpl is added to the window list by WindowManager.
[0127] When removed: removeVoicePageInfo() is triggered when ViewRootImpl is destroyed (e.g., when the Activity finishes).
[0128] Figure 2 This is a structural diagram of a visible-to-talk control system adapted to a vehicle infotainment system provided by one or more embodiments of the present invention.
[0129] like Figure 2 As shown, it includes:
[0130] The module includes a page information acquisition module, a control information acquisition module, a voice command acquisition module, a cloud matching and distribution module, and a terminal execution module.
[0131] The page information acquisition module is used to monitor the lifecycle of application pages and obtain page hierarchy information;
[0132] Used to obtain core page information based on page hierarchy information;
[0133] The core information of the page includes: page unique identifier (id), page type, screen unique identifier, user unique identifier, page level, application package name, and page mode;
[0134] The control information acquisition module is used to scan the hierarchical tree structure of the application page controls and listen for changes in control elements;
[0135] Used to transmit core information through a hierarchical tree structure control to the cloud server;
[0136] The core information of the control includes: the control's unique identifier (id), control type, control enabled status, list item badge, combo control status, control text, control position, control size, and the bound parent control;
[0137] The voice command acquisition module is used to acquire the voice commands of the target user, perform semantic parsing on the voice commands, extract the command intent and key parameters and report them to the cloud server.
[0138] The cloud matching and delivery module is used by the cloud server to match the target page and target control based on the core information of the page and the core information of the control, combined with the instruction intent and key parameters, to generate a voice execution instruction including the target page ID, the target control ID, the instruction intent and key parameters, and then deliver it to the vehicle system.
[0139] The terminal execution module is used to locate the target page based on the target page ID, match the target control based on the target control ID, and call the corresponding execution method to complete the instruction execution according to the instruction intent.
[0140] It is worth noting that although only some basic functional modules are disclosed in this embodiment, it does not mean that the composition of this system is limited to the above-mentioned basic functional modules. On the contrary, what this embodiment intends to express is that, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with existing technology to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. The fact that this embodiment only discloses a few basic functional modules does not mean that the scope of protection of the claims of this invention is limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above device is described separately according to its functions as various units and modules. Of course, in implementing this invention, the functions of each unit and module can be implemented in one or more software and / or hardware.
[0141] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0142] Figure 3 This is an architecture diagram of a visible-to-talk control system adapted to an in-vehicle infotainment system, which is a specific embodiment of the present invention.
[0143] like Figure 3 As shown, it can be said that Android system views are typically organized in units of windows, with one window corresponding to one view. The Android system, at the framework layer, listens to the page lifecycle and collects page information, while WMS (WindowManagerService) listens to the page hierarchy.
[0144] The collected page information (including but not limited to the following information) is as follows:
[0145] Page ID A unique identifier for each page, used to match pages. Page Type activity / system_window / application_sub_window Screen ID The displayId can be used to indicate whether the device is visible on multiple screens. User ID userId, adaptable to multiple users Page hierarchy window_order, multi-application, multi-page hierarchical processing Application package name packageName Page mode windowMode, the page is in split-screen / small window / full-screen mode.
[0146] Among them, see Figure 4 This is a diagram illustrating the process of collecting page information.
[0147] As can be seen, the control information collection includes: the Android system maintains all Android view controls in framework.jar, can scan the page's ViewTree, extract all control information, and at the same time, the system can accurately monitor changes in individual control elements without scanning all controls every time the page changes. The collected controls are reported to the cloud in a tree structure.
[0148] The collected control information (including but not limited to the following information) is as follows:
[0149] control id A unique identifier for each control, used to match controls. type Control types: buttons, sliders, progress bars, switches, etc. isEnabled Is the control available? index Subscripts: 1, 2…8, indicating the order of items in the list. isViewGroup Is it a composite control? text Used to match user voice location Position of the control on the screen size Control size parent The bound parent control
[0150] Among them, see Figure 5 This is a diagram illustrating the process of collecting page information.
[0151] The Android module executes voice commands. After the cloud parses the user's semantics, it matches the corresponding Android control based on the voice and then sends out the voice command. The sent voice command carries the page ID and control ID of the matched control information. After receiving the voice command, the Android side can accurately find the corresponding control based on the page ID and control ID. After finding the control, the Android system directly executes the voice command in the corresponding control.
[0152] Supported voice commands include, but are not limited to, the following:
[0153] voice commands Intent Click Click a control slide Slide a control (including: swipe up, swipe down, swipe left, swipe right). Adjusting the schedule For controls with an input bar, voice adjustment of the progress is supported, such as adjusting the brightness to 30 or the playback progress to 20. Input text Voice input can be used to input text. Select which one For lists, display list icons and respond to user voice requests for "which one".
[0154] Among them, see Figure 6 This is a diagram illustrating the execution of voice commands.
[0155] In one of the software-defined system-level visible-and-say embodiments, the following steps are included:
[0156] 1. The system framework module collects information about individual page controls: By scanning the ViewTree, it traverses and listens to all controls on the page to collect information about all controls, listens for changes in controls in real time, and updates the latest information of controls in a timely manner; at the same time, this module receives voice commands from the cloud and executes related actions.
[0157] 2. The framework module collects all page information for a single process: The VoiceCommandCtroller module is responsible for collecting and updating all page information in the process.
[0158] 3. System Page Management Service collects information on all processes and page controls: The VoiceCommandService service is responsible for coordinating the reporting and updating of all page information in the entire system to the cloud server.
[0159] 4. Voice module parses user semantics: listens to users and parses their semantics, then reports the parsed semantics to the cloud.
[0160] 5. Cloud server matches user commands: It is responsible for receiving information from all controls in the entire system and receiving the user's semantics. Through semantic matching, it matches the corresponding control and sends the corresponding command to the vehicle system.
[0161] For example, the following is a partial code implementation logic for steps 1-5:
[0162] All page-related information is maintained in VoicePageInfo, including page control information:
[0163] public class VoicePageInfo implements Parcelable {
[0164] / / Information on all controls on the page:
[0165] ArrayList <voiceviewinfo>mVoiceViewInfos = new ArrayList<>();
[0166] public int mId;
[0167] public int mWindowType;
[0168] public int mDisplayId;
[0169] public int mUserId;
[0170] public int mWindowMode;
[0171] public String mPackageName;
[0172] public IBinder mWindowToken;
[0173] public FawVoiceViewInfo mRootViewInfo;
[0174] public String mWinTitle = ""; .......
[0176] }
[0177] The information of the control is maintained in VoiceViewInfo:
[0178] public class VoiceViewInfo implements Parcelable {
[0179] / / Information related to the control
[0180] int mId;
[0181] int mType;
[0182] String mText;
[0183] boolean mSwitched;
[0184] int index;
[0185] View mView;
[0186] VoicePageInfo mPageInfo;
[0187] / / Update control information
[0188] void onTextChange(String text) {}
[0189] / / Update control information
[0190] void onProgressChange(int progress) {}
[0191] / / Update control information
[0192] void onSwitchChange(boolean on) {}
[0193] / / Update control information
[0194] void updateVoiceViewInfo() {}
[0195] }
[0196] 2. The framework module collects all page information for a single process.
[0197] The system collects page information in the Android Framework module, specifically when the client process shows / hides the window. See the sample code:
[0198] / / Maintain all page information in the application process
[0199] public class VoiceCommandCtroller {
[0200] HashMap<ViewRootImpl, QisiVoicePageInfo> mVoicePageInfos = newHashMap<>();
[0201] / / During init, collect page information
[0202] void initVoicePageInfos() {}
[0203] / / Add page information and collect relevant parameters for this page.
[0204] void addVoicePageInfo(ViewRootImpl viewRoot) {}
[0205] / / Delete the page control information when the page is not visible.
[0206] void removeVoicePageInfo(ViewRootImpl viewRoot) {}
[0207] / / Update page control information when page controls change.
[0208] void updateVoicePageInfos() {}
[0209] / / Execute voice commands within the application
[0210] void performVoiceAction() {}
[0211] 3. The system page management service collects information on all processes and all page controls.
[0212] The VoiceCommandService service is used to collect control information from all pages in the system. Various processes in the system report and update page control information by calling the relevant interfaces of VoiceCommandService.
[0213] Package name: com.voice.command
[0214] Class Name: VoiceCommandService
[0215] Class methods:
[0216] public class VoiceCommandService {
[0217] / / Each process uses this interface to report / update the page information within its own process.
[0218] public void updateVoicePageInfo(List <fawvoiceviewinfo>voiceInfos) {}
[0219] / / Provide an interface to the outside world, returning information about all pages in the system.
[0220] public Map <fawvoicepageinfo>getVoicePageInfos() {}
[0221] / / Report all system page information to the cloud
[0222] public void updateVoicePageInfosToClound() {}
[0223] }
[0224] 4. The voice module parses user semantics.
[0225] The voice module is responsible for collecting the user's voice, parsing the corresponding semantics, and then uploading it to the cloud server.
[0226] public class VoiceCollect {
[0227] / / Collect user voice
[0228] listenerVoice() {}
[0229] / / Parse user semantics
[0230] convertToString() {}
[0231] / / Upload to cloud server
[0232] uploadToCloud() {
[0233] }
[0234] }
[0235] 5. Cloud Service Module
[0236] It is responsible for receiving information from all controls in the entire system and receiving user semantics. Through semantic matching, it matches the corresponding controls and sends the corresponding instructions to the vehicle system.
[0237] public class CloudVoiceService {
[0238] / / Information on all page controls in the vehicle infotainment system
[0239] List <voicepageinfos>mPageInfos;
[0240] / / receive all page control information of the car system
[0241] receiveAllPageInfos() {}
[0242] / / receive user semantics
[0243] receivePersonVoiceString() {}
[0244] / / match the car system control according to the user semantics
[0245] findVoiceInfoComponent() {}
[0246] / / send voice command
[0247] sendVoiceCommandToClient() {}
[0248] } Figure 7 is a structural block diagram of an electronic device provided by the visible speak control method for adapting a car system according to one or more embodiments of the present application.
[0249] As shown in Figure 7 , the present application provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus.
[0250] The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the visible speak control method for adapting a car system.
[0251] The present application also provides a computer readable storage medium storing a computer program executable by an electronic device, and when the computer program runs on the electronic device, the electronic device executes the steps of the visible speak control method for adapting a car system.
[0252] For the method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited by the action sequence described, because according to the embodiments of the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily necessary for the embodiments of the present application.
[0253] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.
[0254] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.< / voicepageinfos> < / fawvoicepageinfo> < / fawvoiceviewinfo> < / voiceviewinfo>
Claims
1. A visible-to-talk control method adapted to vehicle infotainment systems, characterized in that, Includes the following steps: Monitor the lifecycle of application pages to obtain page hierarchy information; Based on the page hierarchy information, obtain the core information of the page; The core information of the page includes: page unique identifier (id), page type, screen unique identifier, user unique identifier, page level, application package name, and page mode; Scan the hierarchical tree structure of the application page controls, listen for changes in control elements, obtain the core information of the controls through the hierarchical tree structure, and report it to the cloud server; The core information of the control includes: the control's unique identifier (id), control type, control enabled status, list item badge, combo control status, control text, control position, control size, and the bound parent control; The system acquires the voice commands of the target user, performs semantic parsing on the voice commands, extracts the command intent and key parameters, and reports them to the cloud server. The cloud server, based on the core information of the page and the core information of the control, combined with the instruction intent and key parameters, matches the target page and the target control, generates a voice execution instruction including the target page ID, the target control ID, the instruction intent and key parameters, and sends it to the vehicle system. The vehicle system locates the target page based on the target page ID, matches the target control based on the target control ID, and calls the corresponding execution method to complete the instruction execution according to the instruction intent.
2. The visible-to-talk control method for an adapted vehicle infotainment system according to claim 1, characterized in that, include: The target page hierarchy information is obtained through the framework layer. Specifically, the information collection, updating and reporting of all pages within the target application process are managed through the voice command controller. Scan the hierarchical tree structure of application page controls through the framework layer; The voice command controller collects, updates, and reports the core information of controls on all pages.
3. The visible-to-talk control method for adapting to a vehicle infotainment system according to claim 1, characterized in that, The hierarchical tree structure of the scanning application page controls includes: Maintain a control pool for global visualization components and recursively traverse the view tree to extract core control information; When the text, progress, or on / off state of a control changes, update the corresponding information for that control.
4. The visible-to-talk control method for adapting to a vehicle infotainment system according to claim 1, characterized in that, The method of invoking the corresponding execution mode according to the instruction intent includes: If the instruction is intended to be a click, then a touch event is simulated to trigger a click operation on the target control; If the instruction is to slide, the standard control calls the system's sliding interface, and the custom control simulates the sliding trajectory. If the instruction is intended to adjust the progress, the standard progress control calls the control interface, and the custom progress control simulates sliding to the target progress value; If the instruction is intended to input text, then the target control's control interface is called to pass in the text content; If the instruction is intended to select a list item, then the list item is located based on its index, and a touch event is simulated to complete the selection.
5. The visible-to-talk control method for an adapted vehicle infotainment system according to claim 1, characterized in that, include: Based on the voice command service, the system receives page and control information reported by each application process through the voice command controller, integrates it, and reports it to the cloud in real time.
6. The visible-to-talk control method for an adapted vehicle infotainment system according to claim 1, characterized in that, include: The vehicle system filters the set of pages belonging to the currently logged-in user based on the user's unique identifier, excluding private pages of other users; Locate the target page based on the target page ID.
7. The visible-to-talk control method for an adapted vehicle infotainment system according to claim 1, characterized in that, include: After the vehicle's infotainment system executes the command using the corresponding execution method, If the command is a click or list selection, then verify whether the target control triggers the preset response; If the task involves adjusting the progress or inputting text commands, then verify that the progress value or text content of the target control matches the key parameters. Once the verification is complete, the vehicle system will broadcast the verification result to the target user via the voice module, and simultaneously report the execution result to the cloud server for log storage.
8. A visible-to-talk control system adapted to vehicle infotainment systems, characterized in that, include: The module includes a page information acquisition module, a control information acquisition module, a voice command acquisition module, a cloud matching and distribution module, and a terminal execution module. The page information acquisition module is used to monitor the lifecycle of application pages and obtain page hierarchy information; Used to obtain core page information based on page hierarchy information; The core information of the page includes: page unique identifier (id), page type, screen unique identifier, user unique identifier, page level, application package name, and page mode; The control information acquisition module is used to scan the hierarchical tree structure of the application page controls and listen for changes in control elements; Used to transmit core information through a hierarchical tree structure control to the cloud server; The core information of the control includes: the control's unique identifier (id), control type, control enabled status, list item badge, combo control status, control text, control position, control size, and the bound parent control; The voice command acquisition module is used to acquire the voice commands of the target user, perform semantic parsing on the voice commands, extract the command intent and key parameters and report them to the cloud server. The cloud matching and delivery module is used by the cloud server to match the target page and target control based on the core information of the page and the core information of the control, combined with the instruction intent and key parameters, to generate a voice execution instruction including the target page ID, the target control ID, the instruction intent and key parameters, and then deliver it to the vehicle system. The terminal execution module is used to locate the target page based on the target page ID, match the target control based on the target control ID, and call the corresponding execution method to complete the instruction execution according to the instruction intent.
9. An electronic device, characterized in that, include: The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus. The memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of the visible-to-talk control method for an adapted vehicle infotainment system as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, It stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a visible-to-talk control method for an adapted vehicle system as described in any one of claims 1-7.