Vehicle-mounted voice control method and device in butt joint with third-party application

Abstract

The invention relates to the technical field of intelligent cabins, and provides a vehicle-mounted voice control method and device in butt joint with a third-party application. The method comprises the steps that when it is monitored that a target screen displays an interface of a third-party application, control types of all controls are obtained; taking a control corresponding to the control type meeting a preset control type condition as a specified control, determining a control position of the specified control on the target screen, and storing the control position in a specified control list; obtaining a character command character string corresponding to each specified control and updating the character command character string to a specified control list; registering a voice command object according to a command corresponding relation between each character command character string and the control position; when the voice information is received, determining the content of the voice information; and if the target voice command object matched with the voice information content exists in the voice command objects, executing corresponding operation. According to the vehicle-mounted voice control method in butt joint with the third-party application provided by the invention, the user experience of using the vehicle-mounted voice and the driving safety are improved.

Description

Technical Field

[0001] The present application relates to the field of smart cockpit technology, and in particular to a vehicle-mounted voice control method and device for docking with third-party applications. Background Art

[0002] With the development of technology, smart cockpits are becoming more and more common. Among them, the more important smart cockpit technology is voice control technology, which can be operated when it is inconvenient for users to touch or operate buttons. More and more vehicles are equipped with voice collection modules, which can collect vehicle control commands based on voice and then send the voice control commands to the car computer through command conversion to achieve voice control.

[0003] In the existing technology, various manufacturers have developed self-developed voice assistants and some self-developed in-vehicle applications. Self-developed voice assistants can control self-developed in-vehicle applications. However, many non-self-developed third-party applications are incompatible with self-developed voice assistants, and require a lot of effort, time and cost to be controlled by self-developed voice assistants. In addition, the voice functions in third-party applications require frequent maintenance and updates, which makes it difficult and has high long-term maintenance costs. As a result, it is difficult for third-party applications to develop special in-vehicle voice versions, and users still need to manually control them. The in-vehicle voice is not convenient enough, which reduces the user's car experience. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a method and device for in-vehicle voice control that is connected to a third-party application, so as to solve the problem in the prior art that in-vehicle voice is not convenient enough and reduces the user's car experience.

[0005] According to a first aspect of an embodiment of the present application, a vehicle voice control method for connecting to a third-party application is provided, comprising:

[0006] When the target screen is detected to display the interface of a third-party application, all controls in the current interface are scanned to obtain the control types of all controls;

[0007] Taking a control corresponding to a control type that meets a preset control type condition as a designated control, determining a control position of each designated control on a target screen, and storing all designated controls and the control positions corresponding to each designated control in a designated control list, wherein the control type condition includes at least one of a button control type, a text box control type, or an image control type;

[0008] Parse the current interface, obtain the text command string corresponding to each specified control in the specified control list, and update each text command string corresponding to each specified control to the specified control list;

[0009] According to the command correspondence between each text command string and the control position in the specified control list, register the voice command object corresponding to each text command string;

[0010] When receiving voice information, performing voice analysis on the voice information to obtain the voice information content corresponding to the voice information;

[0011] If there is a target voice command object that matches the voice information content in the voice command object, based on the command correspondence in the specified control list, the corresponding operation is performed according to the text command string corresponding to the target voice command object.

[0012] A second aspect of an embodiment of the present application provides a vehicle-mounted voice control device for docking with a third-party application, including:

[0013] The acquisition module is configured to scan all controls in the current interface and acquire the control types of all controls when detecting that the target screen displays the interface of the third-party application;

[0014] A first processing module is configured to use a control corresponding to a control type that meets a preset control type condition as a designated control, determine a control position of each designated control on a target screen, and store all designated controls and the control positions corresponding to each designated control in a designated control list, wherein the control type condition includes at least one of a button control type, a text box control type, or an image control type;

[0015] The second processing module is configured to parse the current interface, obtain the text command string corresponding to each designated control in the designated control list, and update each text command string corresponding to each designated control to the designated control list;

[0016] A third processing module is configured to register a voice command object corresponding to each text command string according to the command correspondence between each text command string and the control position in the specified control list;

[0017] The receiving module is configured to perform voice analysis on the voice information when receiving the voice information to obtain the voice information content corresponding to the voice information;

[0018] The execution module is configured to execute a corresponding operation according to the text command string corresponding to the target voice command object based on the command correspondence in the specified control list if there is a target voice command object matching the voice information content in the voice command object.

[0019] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0020] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0021] Compared with the prior art, the embodiments of the present application have the following beneficial effects: by monitoring the interface of a third-party application displayed on the target screen, all controls in the current interface are scanned to obtain the control type, a designated control is selected according to a preset control type condition, and the control position corresponding to the designated control is obtained, and at the same time, the text command string in each designated control is obtained, and a one-to-one correspondence between the text command string, the designated control and the control position is determined as a command correspondence relationship, and a voice command object corresponding to each text command string is registered according to the command correspondence relationship. When a voice message is received, a voice analysis is performed on the voice message to obtain the voice information content. If a target voice command object corresponding to the voice information content exists in the voice command object, the corresponding operation of the target voice command object is executed, thereby packaging the designated control on the third-party application interface and the corresponding text command string into a voice command object that can be recognized and executed by the vehicle computer, determining the target voice command object through the voice message and then realizing the operation of voice control of the third-party application, thereby improving the controllable range of the in-vehicle voice, reducing the cost of maintenance and development, and eliminating the need for a third-party application to specially develop a version of the in-vehicle voice, thereby improving the user experience and driving safety of the in-vehicle voice. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 It is a schematic diagram of an application scenario of an embodiment of the present application;

[0024] Figure 2 It is a flowchart of a vehicle voice control method for docking a third-party application provided in an embodiment of the present application;

[0025] Figure 3 is a schematic diagram of a vehicle-mounted voice control device for docking with a third-party application provided in an embodiment of the present application;

[0026] Figure 4 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0028] A vehicle-mounted voice control method and device for docking with a third-party application according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0029] Figure 1 1 is a schematic diagram of an application scenario of an embodiment of the present application. The application scenario may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a server 104, and a network 105.

[0030] The first terminal device 101, the second terminal device 102 and the third terminal device 103 can be hardware or software. When the first terminal device 101, the second terminal device 102 and the third terminal device 103 are hardware, they can be various electronic devices with display screens and supporting communication with the server 104, including but not limited to vehicle-mounted computers, vehicle-mounted tablets, vehicle-mounted processors, smart phones, tablet computers, laptop portable computers and desktop computers, etc.; when the first terminal device 101, the second terminal device 102 and the third terminal device 103 are software, they can be installed in the electronic devices described above. The first terminal device 101, the second terminal device 102 and the third terminal device 103 can be implemented as multiple software or software modules, or as a single software or software module, which is not limited in the embodiment of the present application. Further, various applications can be installed on the first terminal device 101, the second terminal device 102 and the third terminal device 103, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0031] The server 104 may be a server that provides various services, for example, a background server that receives a request sent by a terminal device that establishes a communication connection with the server, and the background server may receive and analyze the request sent by the terminal device, and generate a processing result. The server 104 may be a single server, or a server cluster composed of several servers, or a cloud computing service center, which is not limited in the present embodiment of the application.

[0032] It should be noted that the server 104 can be hardware or software. When the server 104 is hardware, it can be various electronic devices that provide various services for the first terminal device 101, the second terminal device 102, and the third terminal device 103. When the server 104 is software, it can be multiple software or software modules that provide various services for the first terminal device 101, the second terminal device 102, and the third terminal device 103, or it can be a single software or software module that provides various services for the first terminal device 101, the second terminal device 102, and the third terminal device 103, and the embodiment of the present application does not limit this.

[0033] The network 105 can be a wired network connected by coaxial cable, twisted pair and optical fiber, or it can be a wireless network that can interconnect various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), infrared, etc., which is not limited in the embodiments of the present application.

[0034] It should be noted that the specific types, quantities and combinations of the first terminal device 101, the second terminal device 102 and the third terminal device 103, the server 104 and the network 105 can be adjusted according to the actual needs of the application scenario, and the embodiments of the present application do not limit this.

[0035] Figure 2 1 is a flow chart of a method for in-vehicle voice control for connecting to a third-party application provided in an embodiment of the present application. Figure 2 As shown, the vehicle voice control method for docking a third-party application includes the following steps:

[0036] S201, when it is detected that the target screen displays the interface of a third-party application, all controls in the current interface are scanned to obtain the control types of all controls;

[0037] S202, taking a control corresponding to a control type that meets a preset control type condition as a designated control, determining a control position of each designated control on a target screen, and storing all designated controls and control positions corresponding to each designated control in a designated control list;

[0038] S203, parsing the current interface, obtaining the text command string corresponding to each designated control in the designated control list, and updating each text command string corresponding to each designated control to the designated control list;

[0039] S204, registering a voice command object corresponding to each text command string according to the command correspondence between each text command string and the control position in the specified control list;

[0040] S205, when the voice information is received, voice analysis is performed on the voice information to obtain the voice information content corresponding to the voice information;

[0041] S206: If there is a target voice command object matching the voice information content in the voice command object, based on the command correspondence in the specified control list, a corresponding operation is performed according to the text command string corresponding to the target voice command object.

[0042] The control type condition includes at least one of a button control type, a text box control type, or an image control type.

[0043] Figure 2 The in-vehicle voice control method for docking third-party applications can be Figure 1 The speech analysis can be performed by any terminal device including a target screen in the first terminal device, the second terminal device or the third terminal device. Figure 1 The provided server implementation.

[0044] In an exemplary embodiment of the present application, when the interface of a third-party application is monitored and displayed on the target screen, all controls in the currently displayed interface are scanned to identify and obtain the control types of all controls on the interface, where the third-party application is an application that is not self-developed, and the control types include button controls, text box controls, form controls, scroll bar controls, image controls, etc.

[0045] After completing the control type scan of the controls on the interface, they are screened according to the preset control type conditions, and the controls that meet the preset control type conditions are used as designated controls, wherein the preset control type conditions include at least one of a button control type (Button), a text box control type (TextView), or an image control type (ImageView).

[0046] While filtering designated controls from all scanned control types, determine the control position of each designated control on the target screen, so that the voice content in the voice message sent by the user can be accurately identified and executed based on the control position, and store all designated controls and the control positions corresponding to the designated controls in a designated control list.

[0047] Parse the executable text in the current interface and obtain the text command string corresponding to each specified control in the specified control list. The text command string can be understood as a functional description or operation instruction of the control, which is important information when the user interacts with the control. After obtaining the text command string, update it to the corresponding position of each specified control in the specified control list to ensure that the specified control list includes the specified control, the corresponding control position and its corresponding text command string.

[0048] According to the command correspondence between each text command string in the specified control list and the control position corresponding to each specified control, the specified control list is traversed to register the voice command object (Cache) corresponding to each text command string.

[0049] When a voice message sent by a user is received, the voice message is analyzed to extract the specific voice content, obtain the voice information content, traverse the registered voice command objects to check whether there is a target voice command object that matches the voice information content. If so, according to the command correspondence in the specified control list, execute the operation represented by the text command string corresponding to the target voice command object, so that the user can interact with the interface of the third-party application displayed on the target screen through a simple voice message, thereby improving driving safety and achieving a more convenient and efficient operating experience.

[0050] According to the technical solution provided in the embodiment of the present application, the designated controls and corresponding text command strings on the third-party application interface are packaged as voice command objects that are recognizable and executable by the vehicle computer. The target voice command object is determined through voice messages to realize voice control of the third-party application, thereby increasing the controllable range of the in-vehicle voice and enhancing the user experience of using the in-vehicle voice and driving safety.

[0051] In some embodiments, determining the control position of each designated control on the target screen includes:

[0052] Taking the upper left corner of the target screen as the coordinate origin, a two-dimensional coordinate system is constructed as the preset screen coordinate system;

[0053] For each specified control, do the following:

[0054] Determine the vertex coordinates corresponding to each vertex of the specified control based on a preset screen coordinate system, where the vertex coordinates include a horizontal coordinate and a vertical coordinate;

[0055] Find the average of the horizontal coordinates of all vertex coordinates as the horizontal coordinate of the control position;

[0056] Find the average of the ordinates of all vertex coordinates and use it as the ordinate of the control's position.

[0057] In order to accurately determine the control position of each specified control on the target screen, a two-dimensional coordinate system is constructed with the upper left corner of the target screen as the origin, the right as the positive direction of the X axis, and the downward as the positive direction of the Y axis as the preset screen coordinate system.

[0058] For each specified control, do the following:

[0059] Determine the vertex coordinates of each vertex of the specified control in the preset screen coordinate system. The vertex of the control is usually a point on its edge. The shape of the specified control is generally a rectangle, that is, the vertices of the control include the upper left corner, the upper right corner, the lower right corner and the lower left corner. The vertex coordinates of each vertex of the control in the preset screen coordinate system can be obtained by screen capture or image processing basics, where the vertex coordinates include abscissa (X value) and ordinate (Y value).

[0060] The vertex coordinates of each control include two horizontal coordinates with different values ​​and two vertical coordinates with different values.

[0061] After obtaining the coordinates of all vertices of the control, calculate the average value of the vertex coordinates to represent the control position. For the horizontal coordinate, add up all the X values ​​and divide by the number of vertices to get the average position of the control in the X-axis direction as the horizontal coordinate of the control position.

[0062] Similarly, for the ordinate, add up the Y values ​​of all vertices and divide by the number of vertices to get the average position of the control in the Y-axis direction as the ordinate of the control position.

[0063] Among them, the basis of the above method is that the shape of the specified control is a regular geometric shape and the vertices of the specified control can be clearly identified. When the shape of the specified control is an irregular control or the vertices are difficult to accurately identify, more complex image processing technology or machine learning algorithms can be used to assist in determining the control position of the specified control.

[0064] According to the technical solution provided in the embodiment of the present application, a precise control position can be obtained to indicate the center position of each specified control on the target screen, thereby facilitating the user to control the interface of the third-party application displayed on the current target screen through voice messages.

[0065] In some embodiments, parsing the current interface to obtain the text command string corresponding to each specified control in the specified control list includes:

[0066] When the control type of the specified control is a button control type or a text box control type, the text included in the specified control is obtained as a text command string through the Get Text function;

[0067] When the control type of the specified control is an image control type, the image included in the specified control is obtained through the image acquisition function, and the image command in the image is identified as a text command string through the preset image interface. The image command includes one or more of icons and texts.

[0068] Parse the current interface. When the control type of the specified control is a button control type or a text box control type, obtain the text included in the control through the Get Text function corresponding to the control type and use it as a text command string.

[0069] Specifically, when the control type of the specified control is a button control type, the text in the button control type can be directly obtained through the button text obtaining function Button.getText() as the corresponding text command string. When the control type of the specified control is a text box control type, the text in the text box control type can be directly obtained through the text box text obtaining function TextView.getText() as the corresponding text command string.

[0070] When the control type is an image control type, the information included is generally an image. The image data in the image control type is obtained through the image acquisition function ImageView.getImage(), and the image data is input into the preset image interface for identification, so as to obtain the image command in the image control type, and the image command is used as a text command string, wherein the image command includes one or more icons or texts.

[0071] Among them, the preset image interface includes a series of image recognition algorithms, which can identify specific elements in the image, such as the play button, pause button, fast forward button, etc. as image type controls. Through image recognition, the content represented by the icons in the above buttons can be directly identified, thereby determining the text command string corresponding to the image type control.

[0072] When recognizing image commands, it is necessary to ensure that the preset image interface has been fully trained and optimized for various icons and texts that may appear to ensure the accuracy and efficiency of recognition. In addition, since image recognition itself may have a certain error rate, it is also necessary to manually check or correct the recognition results when necessary.

[0073] According to the technical solution provided in the embodiment of the present application, by traversing the specified control list and obtaining text command strings according to different control categories, the text command string corresponding to each specified control is determined, and the relationship between each specified control and the text command string can be clarified.

[0074] In some embodiments, when it is detected that the target screen displays the interface of a third-party application, all controls in the current interface are scanned to obtain the control types of all controls, including:

[0075] When the in-vehicle voice service is in an enabled state, the display content of the target screen is monitored based on the in-vehicle voice service;

[0076] When the interface displaying the content is a third-party application, all controls in the current interface are scanned based on a preset control scanning interface to obtain the control types of all controls.

[0077] The in-vehicle voice service can provide users with basic functions such as voice navigation and music playback. In the embodiment provided in this application, the in-vehicle voice service is endowed with the function of monitoring the display content of the target screen. When the in-vehicle voice service is turned on, the display content of the target screen is monitored according to the in-vehicle voice service, so that the interface changes displayed on the target screen can be captured more quickly and accurately.

[0078] When the in-vehicle voice service detects an interface displaying content of a third-party application, it triggers the preset scanning interface to scan all controls in the current interface. During the scanning process, the preset scanning interface will identify each control on the current interface one by one and obtain the control type of each control.

[0079] According to the technical solution provided in the embodiment of the present application, by combining the monitoring function of the in-vehicle voice service and the scanning capability of the preset control scanning interface, it is possible to achieve a comprehensive scan of the third-party application interface on the target screen and obtain the control type, thereby improving the efficiency of processing the target display screen and ensuring the stability and reliability of the in-vehicle system.

[0080] In some embodiments, if there is a target voice command object matching the voice information content in the voice command object, based on the command correspondence in the specified control list, a corresponding operation is performed according to the text command string corresponding to the target voice command object, including:

[0081] Based on the fuzzy matching method, determine whether the voice information content matches the text command string corresponding to the voice command object;

[0082] If so, it is determined that there is a target voice command object matching the voice information content in the voice command object, and the voice command object corresponding to the matching text command string is used as the target voice command object.

[0083] When the in-vehicle voice system receives a voice message sent by a user and determines the content of the voice message, it uses fuzzy matching to compare the voice message content and the text command string corresponding to the voice command object one by one to see if they match. This takes into account possible errors in voice recognition and possible errors caused by different user expression habits. Therefore, fuzzy matching is used to improve the accuracy and flexibility of matching.

[0084] During the matching process, the keywords, phrases and overall semantics in the voice information content are comprehensively considered and compared with the text command string in the voice command object.

[0085] If a voice command object that is highly similar or completely matches the voice information content is found in the voice command object, it will be determined as the target voice command object. The text command string and control position corresponding to the target voice command object can be further determined by specifying a control list. This is an important basis for the in-vehicle voice system to control the interface of third-party applications.

[0086] If no voice command object that is highly similar to or completely matches the voice information content is found in the voice command object, a prompt message is sent to inform the user that the operation has failed.

[0087] According to the technical solution provided in the embodiment of the present application, the in-vehicle voice system can accurately understand the meaning included in the voice information sent by the user, and perform corresponding operations according to the command correspondence in the specified control list, thereby improving the interaction efficiency between the user and the vehicle computer and improving the user's usage experience.

[0088] In some embodiments, based on the command correspondence in the specified control list, a corresponding operation is performed according to the text command string corresponding to the target voice command object, including:

[0089] Determine the control position corresponding to the target voice command object based on the command correspondence in the specified control list;

[0090] Through the preset event simulation interface, a click event is simulated at the position corresponding to the control position on the target screen, so that the interface performs the corresponding operation through the text command string corresponding to the target voice command object.

[0091] The specified control list contains detailed information of all specified controls in the interface displayed on the current target screen, including control type, control position, and text command string.

[0092] By traversing the specified control list, by comparing the identifier or keyword of the target voice command object, such as the corresponding text command string, the control entry of the target voice command object can be quickly located as the target control entry, and the control position corresponding to the target voice command object can be determined according to the target control entry. The control position is represented in the form of a coordinate point, which accurately describes the position of the specified control on the target screen.

[0093] Through the preset event simulation interface, a simulated click event is executed at the position corresponding to the control position on the target screen. This can be achieved by calling the underlying interface or function of the vehicle system. The simulated click event can simulate the user's click or touch operation, and the simulated click event can trigger the response of the specified control. The corresponding simulated animation can also be displayed on the target screen.

[0094] When simulating a click event, it is possible to ensure that the coordinates of the click accurately match the position of the specified control, thereby ensuring that the click operation can accurately act on the specified control.

[0095] When the designated control is triggered by a simulated click event, the current interface will perform corresponding operations according to the text command string corresponding to the target voice command object, which may include switching interfaces, adjusting settings, turning up the volume, turning down the volume, reducing the screen, etc., depending on the functions and instructions represented by the text command string.

[0096] According to the technical solution provided in the embodiment of the present application, it is possible to accurately execute operations specified by the user through voice commands based on the command correspondence in the specified control list, thereby improving the intelligence level of the vehicle-mounted system and providing users with a more convenient and efficient interactive experience.

[0097] In some embodiments, based on the command correspondence in the specified control list, after executing the corresponding operation according to the text command string corresponding to the target voice command object, the method further includes:

[0098] Send a success prompt message, which is used to prompt that the voice message is successfully executed;

[0099] When the interface switching of the third-party application displayed on the target screen is detected, the current specified control list and all voice command objects are cleared.

[0100] After executing the corresponding operation according to the text command string corresponding to the target voice command object, a success prompt message is sent to the target screen to inform the user that the voice message has been successfully executed. The success prompt message can be presented in the form of text, sound or visual prompt, such as popping up a short prompt box on the target screen, or playing a confirmation sound through the car audio.

[0101] Since interface switching often means that the current operation has been completed or the user has switched to other functions or applications, in this case, continuing to retain the current specified control list and voice command object may cause the information to be outdated or erroneous. Therefore, the interface changes of the target screen are continuously monitored to ensure the real-time and effectiveness of the specified control list.

[0102] When the interface switch of a third-party application is detected, the current specified control list and all related voice command objects are immediately cleared to ensure that the status of the specified control list and voice command objects are consistent with the current interface, avoiding confusion or errors that may occur in subsequent operations. At the same time, prepare for the next voice information recognition and execution to ensure that the system can respond to the user's new instructions at any time.

[0103] According to the technical solution provided in the embodiment of the present application, the consistency and satisfaction of the user experience are improved, and it is also ensured that the in-vehicle voice system can accurately and efficiently handle various interface changes and voice commands.

[0104] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0105] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0106] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0107] Figure 3 Schematic diagram of a vehicle voice control device for docking with a third-party application provided in an embodiment of the present application. Figure 3 As shown, the vehicle-mounted voice control device for docking third-party applications includes: an acquisition module 301, a first processing module 302, a second processing module 303, a third processing module 304, a receiving module 305, and an execution module 306, wherein:

[0108] The acquisition module 301 is configured to scan all controls in the current interface and acquire the control types of all controls when detecting that the target screen displays the interface of the third-party application;

[0109] The first processing module 302 is configured to use a control corresponding to a control type that meets a preset control type condition as a designated control, determine a control position of each designated control on the target screen, and store all designated controls and the control positions corresponding to each designated control in a designated control list, wherein the control type condition includes at least one of a button control type, a text box control type, or an image control type;

[0110] The second processing module 303 is configured to parse the current interface, obtain the text command string corresponding to each designated control in the designated control list, and update each text command string corresponding to each designated control to the designated control list;

[0111] The third processing module 304 is configured to register the voice command object corresponding to each text command string according to the command correspondence between each text command string and the control position in the specified control list;

[0112] The receiving module 305 is configured to perform voice analysis on the voice information when receiving the voice information to obtain the voice information content corresponding to the voice information;

[0113] The execution module 306 is configured to execute a corresponding operation according to the text command string corresponding to the target voice command object based on the command correspondence in the specified control list if there is a target voice command object matching the voice information content in the voice command object.

[0114] In some embodiments, the first processing module 302 is configured to:

[0115] Taking the upper left corner of the target screen as the coordinate origin, a two-dimensional coordinate system is constructed as the preset screen coordinate system;

[0116] For each specified control, do the following:

[0117] Determine the vertex coordinates corresponding to each vertex of the specified control based on a preset screen coordinate system, where the vertex coordinates include a horizontal coordinate and a vertical coordinate;

[0118] Find the average of the horizontal coordinates of all vertex coordinates as the horizontal coordinate of the control position;

[0119] Find the average of the ordinates of all vertex coordinates and use it as the ordinate of the control's position.

[0120] In some embodiments, the second processing module 303 is configured to:

[0121] When the control type of the specified control is a button control type or a text box control type, the text included in the specified control is obtained as a text command string through the Get Text function;

[0122] When the control type of the specified control is an image control type, the image included in the specified control is obtained through the image acquisition function, and the image command in the image is identified as a text command string through the preset image interface. The image command includes one or more of icons and texts.

[0123] In some embodiments, the acquisition module 301 is configured to:

[0124] When the in-vehicle voice service is in an enabled state, the display content of the target screen is monitored based on the in-vehicle voice service;

[0125] When the interface displaying the content is a third-party application, all controls in the current interface are scanned based on a preset control scanning interface to obtain the control types of all controls.

[0126] In some embodiments, the execution module 306 is configured to:

[0127] Based on the fuzzy matching method, determine whether the voice information content matches the text command string corresponding to the voice command object;

[0128] If so, it is determined that there is a target voice command object matching the voice information content in the voice command object, and the voice command object corresponding to the matching text command string is used as the target voice command object.

[0129] In some embodiments, the execution module 306 is configured to:

[0130] Determine the control position corresponding to the target voice command object based on the command correspondence in the specified control list;

[0131] Through the preset event simulation interface, a click event is simulated at the position corresponding to the control position on the target screen, so that the interface performs the corresponding operation through the text command string corresponding to the target voice command object.

[0132] In some embodiments, after executing the corresponding operation according to the text command string corresponding to the target voice command object based on the command correspondence in the specified control list, the execution module 306 is further configured to:

[0133] Send a success prompt message, which is used to prompt that the voice message is successfully executed;

[0134] When the interface switching of the third-party application displayed on the target screen is detected, the current specified control list and all voice command objects are cleared.

[0135] Figure 4 Schematic diagram of an electronic device 4 provided in an embodiment of the present application. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0136] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 The electronic device 4 is merely an example and does not limit the electronic device 4 , and may include more or less components than those shown in the figure, or different components.

[0137] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0138] The memory 402 may be an internal storage unit of the electronic device 4, for example, a hard disk or memory of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. The memory 402 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0139] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0140] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. Computer-readable storage media may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0141] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A vehicle voice control method for docking with a third-party application, characterized in that: include: When it is detected that the target screen displays the interface of the third-party application, all controls in the current interface are scanned to obtain the control types of all the controls; taking the control corresponding to the control type that meets the preset control type condition as a designated control, determining the control position of each designated control on the target screen, and storing all the designated controls and the control positions corresponding to each designated control in a designated control list, wherein the control type condition includes at least one of a button control type, a text box control type, or an image control type; Parse the current interface, obtain the text command string corresponding to each designated control in the designated control list, and update each text command string corresponding to each designated control to the designated control list; According to the command correspondence between each of the text command strings and the control position in the specified control list, registering the voice command object corresponding to each of the text command strings; When receiving voice information, performing voice analysis on the voice information to obtain voice information content corresponding to the voice information; If there is a target voice command object matching the voice information content in the voice command object, based on the command correspondence in the designated control list, a corresponding operation is performed according to the text command string corresponding to the target voice command object.

2. The method according to claim 1, characterized in that Determining the control position of each of the designated controls on the target screen includes: Taking the upper left corner of the target screen as the coordinate origin, constructing a two-dimensional coordinate system as a preset screen coordinate system; For each of the specified controls, perform the following operations: Determine the vertex coordinates corresponding to each control vertex of the specified control based on the preset screen coordinate system, wherein the vertex coordinates include a horizontal coordinate and a vertical coordinate; Calculate the average value of the horizontal coordinates of all the vertex coordinates as the horizontal coordinate of the control position; The vertical coordinates of all the vertex coordinates are averaged to obtain the vertical coordinate of the control position.

3. The method according to claim 1, characterized in that Parse the current interface and obtain the text command string corresponding to each of the specified controls in the specified control list, including: When the control type of the designated control is the button control type or the text box control type, obtaining the text included in the designated control as a text command string by using a text acquisition function; When the control type of the designated control is the image control type, the image included in the designated control is acquired through an image acquisition function, and an image command in the image is identified as the text command string through a preset image interface, wherein the image command includes one or more of icons and texts.

4. The method according to claim 1, characterized in that: When the target screen is detected to display the interface of a third-party application, all controls in the current interface are scanned to obtain the control types of all the controls, including: When the in-vehicle voice service is in an enabled state, monitoring display content of the target screen based on the in-vehicle voice service; When it is detected that the displayed content is an interface of a third-party application, all controls in the current interface are scanned based on a preset control scanning interface to obtain control types of all the controls.

5. The method according to claim 1, characterized in that: If there is a target voice command object matching the voice information content in the voice command object, based on the command correspondence in the specified control list, a corresponding operation is performed according to the text command string corresponding to the target voice command object, including: Based on the fuzzy matching method, determining whether the voice information content matches the text command string corresponding to the voice command object; If so, it is determined that there is a target voice command object matching the voice information content in the voice command object, and the voice command object corresponding to the text command string matching the text command string is used as the target voice command object.

6. The method according to claim 1, characterized in that Based on the command correspondence in the specified control list, performing a corresponding operation according to the text command string corresponding to the target voice command object includes: Determining the control position corresponding to the target voice command object based on the command correspondence in the designated control list; Through a preset event simulation interface, a click event is simulated at a position corresponding to the control position on the target screen, so that the interface performs a corresponding operation through a text command string corresponding to the target voice command object.

7. The method according to any one of claims 1 to 6, characterized in that After executing a corresponding operation according to the text command string corresponding to the target voice command object based on the command correspondence in the designated control list, the method further includes: Sending a success prompt message, wherein the success prompt message is used to prompt that the voice information is successfully executed; When the interface switching of the third-party application displayed on the target screen is detected, the current designated control list and all the voice command objects are cleared.

8. A vehicle-mounted voice control device for docking with a third-party application, characterized in that: include: The acquisition module is configured to scan all controls in the current interface and acquire control types of all controls when detecting that the target screen displays the interface of the third-party application; A first processing module is configured to take the control corresponding to the control type that meets a preset control type condition as a designated control, determine the control position of each designated control on the target screen, and store all the designated controls and the control positions corresponding to each designated control in a designated control list, wherein the control type condition includes at least one of a button control type, a text box control type, or an image control type; The second processing module is configured to parse the current interface, obtain the text command string corresponding to each designated control in the designated control list, and update each text command string corresponding to each designated control to the designated control list; A third processing module is configured to register a voice command object corresponding to each text command string according to the command correspondence between each text command string and the control position in the specified control list; The receiving module is configured to perform voice analysis on the voice information when receiving the voice information to obtain the voice information content corresponding to the voice information; The execution module is configured to execute a corresponding operation according to the text command string corresponding to the target voice command object based on the command correspondence in the specified control list if there is a target voice command object matching the voice information content in the voice command object.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images