Control method and control system applied to screen projection scene and computer storage medium

Abstract

The invention relates to the field of intelligent driving, in particular to a control method and system applied to a screen projection scene and a computer readable storage medium. One aspect of the invention provides a control method applied to a screen projection scene, and the control method is used for establishing communication connection between a mobile terminal and a vehicle-mounted terminal and projecting first screen content of the mobile terminal to a vehicle-mounted terminal screen for display. The control method comprises the following steps: entering a voice control scene in response to a start instruction received from the vehicle-mounted terminal; obtaining text information corresponding to the change of the first screen content; and in response to a received instruction indicating that user voice control from the vehicle-mounted terminal is matched with the text information, adjusting the first screen content to a second screen content based on the text information matched with the user voice control.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving, and more specifically to a control method applied to a screen projection scenario, a control system applied to a screen projection scenario, and a computer-readable storage medium. Background Art

[0002] Screen projection technology is to mirror the screen display content of one terminal to the screen of another terminal for display. For example, the screen content of a mobile terminal can be mirrored to the screen of a car computer for display. In this way, the content of a small screen can be projected to a large screen to enhance the display effect, and various applications of small-screen devices can be fully utilized, including map navigation, video playback, entertainment games, etc.

[0003] At present, after the mobile terminal establishes a connection with the vehicle terminal via Wi-Fi or Bluetooth, the user can turn on the screen projection mode on the mobile terminal to project the screen content of the mobile terminal to the vehicle screen for display, thereby triggering the mobile terminal to start real-time recording of the screen content, and encode the recorded screen content and transmit it to the vehicle terminal for decoding and display, thereby achieving the effect of projecting the screen content of the mobile terminal to the vehicle screen for display. However, in the projection scenario where the screen content of the mobile terminal is mirrored to the vehicle screen for display, many control display functions that are the same or similar to those in the non-projection scenario cannot be achieved. For example, in the non-projection scenario, the user can directly control the display of the terminal screen through the voice input function. Due to the many functional limitations in the projection scenario, the user experience is affected and may affect driving safety. Summary of the invention

[0004] In order to solve or at least alleviate one or more of the above problems, the following technical solutions are provided.

[0005] According to the first aspect of the present application, a control method applied to a screen projection scenario is provided, wherein the control method is used to establish a communication connection between a mobile terminal and a vehicle-mounted terminal and project the first screen content of the mobile terminal to the vehicle-mounted screen for display, and the control method includes the following steps: entering a voice control scenario in response to receiving a start command from the vehicle-mounted terminal; acquiring text information corresponding to changes in the first screen content; and in response to receiving an instruction indicating that the user voice control from the vehicle-mounted terminal matches the text information, adjusting the first screen content to the second screen content based on the text information matching the user voice control.

[0006] According to the control method applied to the screen projection scenario described in one embodiment of the present application, the mobile terminal establishes a communication connection with the vehicle terminal through near-field communication; and / or the control method also includes exiting the voice control scenario in response to receiving an end instruction and / or a vehicle control instruction from the vehicle terminal.

[0007] According to the control method applied to the screen projection scenario according to one embodiment of the present application or any of the above embodiments, one or more of the start command, the end command, and the vehicle control command are generated based on the user's voice input.

[0008] According to the control method applied to the screen projection scenario according to one embodiment of the present application or any one of the above embodiments, obtaining text information corresponding to the changes in the first screen content includes: detecting screen switching behavior to determine the active page with focus in the current task stack; traversing the root layout corresponding to the active page to obtain a text type control; and obtaining information on text attributes in the text type control as text information corresponding to the changes in the first screen content.

[0009] According to the control method applied to the screen projection scenario according to one embodiment of the present application or any of the above embodiments, detecting the screen switching behavior includes: using the activity management service to monitor the page user interface tree to detect the screen switching behavior.

[0010] According to the control method applied to the screen projection scenario described in one embodiment or any one of the above embodiments of the present application, in response to receiving an instruction indicating that the user voice control from the vehicle-mounted terminal matches the text information, adjusting the first screen content to the second screen content based on the text information matching the user voice control includes: determining a control corresponding to the text information in response to receiving an instruction indicating that the user voice control from the vehicle-mounted terminal matches the text information; and calling the operation behavior of the control corresponding to the text information to adjust the first screen content to the second screen content.

[0011] According to the second aspect of the present application, a control method applied to a screen projection scenario is provided, wherein the control method is used to establish a communication connection between a vehicle-mounted terminal and a mobile terminal and display a first screen content projected from the mobile terminal on the vehicle screen, the control method comprising the following steps: receiving a start instruction and sending the start instruction to the mobile terminal, so that the mobile terminal enters a voice control scenario, wherein, in the voice control scenario, the mobile terminal obtains text information corresponding to changes in the first screen content; receives user voice control and the text information; and in response to the mobile terminal adjusting the first screen content to the second screen content based on the text information matching the user voice control, displays the second screen content.

[0012] According to the control method applied to the screen projection scenario described in one embodiment of the present application, the control method further includes: generating or receiving an instruction indicating that the user voice control matches the text information, and sending the instruction indicating that the user voice control matches the text information to the mobile terminal.

[0013] According to the control method applied to the screen projection scenario according to one embodiment of the present application or any of the above embodiments, the vehicle-mounted terminal establishes a communication connection with the mobile terminal through near-field communication; and / or the control method also includes instructing the mobile terminal to exit the voice control scenario in response to receiving an end instruction and / or a vehicle control instruction.

[0014] According to the control method applied to the screen projection scenario according to one embodiment of the present application or any of the above embodiments, one or more of the start command, the end command, and the vehicle control command are generated based on the user's voice input.

[0015] According to the control method applied to the screen projection scenario according to one embodiment of the present application or any one of the above embodiments, the control method further includes: controlling the in-vehicle intelligent terminal to perform a prompt operation in response to the user voice control in response to displaying the second screen content; and maintaining a listening state to receive one or more of user voice control, end instructions and vehicle control instructions.

[0016] According to the third aspect of the present application, a control system applied to a screen projection scenario is provided, the system comprising: a memory; a processor coupled to the memory; and a computer program stored on the memory and running on the processor, the execution of the computer program results in the execution of the steps of the control method applied to the screen projection scenario according to the first aspect of the present application or the execution of the steps of the control method applied to the screen projection scenario according to the second aspect of the present application.

[0017] According to the fourth aspect of the present application, a computer storage medium is provided, which includes instructions, which, when running, execute the steps of the control method applied to the screen projection scene according to the first aspect of the present application or execute the steps of the control method applied to the screen projection scene according to the second aspect of the present application.

[0018] The screen projection solution according to one or more embodiments of the present application can control the screen content through user voice input in the screen projection scenario of the mobile terminal to the vehicle, thereby realizing the control functions and display effects of the terminal screen consistent with the non-screen projection scenario, avoiding the driving risks brought by touching the screen, and improving the user experience and driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or other aspects and advantages of the present application will become clearer and easier to understand through the following description of various aspects in conjunction with the accompanying drawings, in which the same or similar units are represented by the same reference numerals. The accompanying drawings include:

[0020] Figure 1 A flow chart of a control method applied to a screen projection scenario according to one or more embodiments of the present application is shown.

[0021] Figure 2 A flow chart of a control method applied to a screen projection scenario according to one or more embodiments of the present application is shown.

[0022] Figure 3 A flow chart of a control method applied to a screen projection scenario according to one or more embodiments of the present application is shown.

[0023] Figure 4 A block diagram of a control system applied to a screen projection scenario according to one or more embodiments of the present application is shown.

[0024] Figure 5 A block diagram of a control system applied to a screen projection scenario according to one or more embodiments of the present application is shown. DETAILED DESCRIPTION

[0025] The present application is described more fully below with reference to the accompanying drawings in which illustrative embodiments of the present application are illustrated. However, the present application may be implemented in different forms and should not be interpreted as being limited to the embodiments given herein. The above embodiments are given to make the disclosure herein comprehensive and complete, so as to more fully convey the scope of protection of the present application to those skilled in the art.

[0026] In this specification, terms such as "comprise" and "include" indicate that in addition to the units and steps directly and explicitly stated in the specification and claims, the technical solution of the present application does not exclude the situation where there are other units and steps that are not directly or explicitly stated.

[0027] Unless otherwise specified, terms such as "first" and "second" do not indicate the order of the units in terms of time, space, size, etc., but are merely used to distinguish the units.

[0028] Hereinafter, various exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings.

[0029] Figure 1 A flow chart of a control method applied to a screen projection scenario according to one or more embodiments of the present application is shown.

[0030] Figure 1The control method applied to the screen projection scenario according to one or more embodiments of the present application shown in can be used to establish a communication connection between a mobile terminal and a vehicle-mounted terminal and project the first screen content of the mobile terminal to the vehicle-mounted terminal's vehicle screen for display. An exemplary screen projection scenario may be that during the use of the car, the user searches for a destination through the mobile terminal and starts navigation, but it is sometimes inconvenient to view the navigation information through the screen of the mobile terminal, or when the user wants to watch the video or image on the mobile terminal through the vehicle screen, the screen projection operation of the mobile terminal to the vehicle screen can be enabled. Exemplarily, the vehicle screen may include a central control screen, an instrument panel screen, a steering wheel screen, a front windshield screen, a rear windshield screen, a co-pilot screen, a window screen, a sunroof screen, and the like.

[0031] Optionally, the mobile terminal can perform video encoding on the first screen content, and send the encoded display information to the vehicle-mounted terminal. The vehicle-mounted terminal decodes the received display information through a decoder, and creates a Surface to carry and display the decoded display information. Exemplarily, the mobile terminal may include but is not limited to personal computers, laptops, smart phones, tablet computers, and portable wearable devices. Optionally, the mobile terminal and the vehicle-mounted terminal can be authenticated based on the same account and establish a communication connection through near-field communication. After the connection is successful, the two parties can conduct secure and reliable near-field communication. Optionally, the mobile terminal and the vehicle-mounted terminal can also establish a communication connection through a Wi-Fi network, Bluetooth, etc.

[0032] like Figure 1 As shown in , in step S101, the voice control scene is entered in response to receiving a start command from the vehicle terminal. It should be noted that the voice control scene refers to a scene in which the mobile terminal projects the screen to the vehicle terminal, and the user can directly control the content displayed on the vehicle screen of the vehicle terminal through voice input commands without touching the screens of the mobile terminal and the vehicle terminal. The control may include, for example, starting a new screen content, closing the current screen content, switching the current screen content to another screen content, etc.

[0033] Optionally, in step S101, the in-vehicle intelligent device of the in-vehicle terminal can be awakened by the user and send the start command input by the user to the mobile terminal through near field communication, and the mobile terminal can enter the voice control scene in response to receiving the start command.

[0034] In the voice control scenario, the mobile terminal can also exit the voice control scenario in response to receiving an end instruction and / or a vehicle control instruction from the vehicle terminal. Exemplarily, the start instruction, the end instruction, and the vehicle control instruction can be generated based on the user's voice input. Exemplarily, the vehicle-mounted smart device can be installed in the approximate middle position of the dashboard of the vehicle to interact with the passengers and the driver in the cockpit, for example, information interaction, action interaction, etc. In one embodiment, when the mobile terminal receives the vehicle control instruction "open the sunroof" from the vehicle terminal, the voice control scenario can be exited, and the vehicle terminal can perform the operation of opening the sunroof.

[0035] In step S103, text information corresponding to the change of the first screen content is obtained.

[0036] Optionally, in step S103, the screen switching behavior can be detected to determine the active page with focus in the current task stack, the root layout corresponding to the active page can be traversed to obtain a text type control, and the information of the text attributes in the text type control can be obtained as text information corresponding to the change of the first screen content. Optionally, after the mobile terminal enters the voice control scene in response to receiving the start command, the mobile terminal can start a monitoring service to detect its screen switching behavior. Optionally, the mobile terminal can use the Activity Manager Service (AMS) to monitor the page user interface tree to detect screen switching behavior, thereby being compatible with more third-party applications, and can improve practicality and efficiency compared to the conventional AccessibilityService solution. It should be noted that the text information corresponding to the change of the first screen content may include one or more text information.

[0037] In one embodiment, AMS can be used to obtain the active page Activity with focus in the current task stack. After obtaining the Activity, the root layout DercoView can be obtained through the window for traversal, and all text type controls can be obtained by filtering TextView and its derived custom View, thereby obtaining the text attribute information in the text type control as the text information corresponding to the changes in the first screen content.

[0038] In one embodiment, while the mobile terminal enters the voice control scenario, the vehicle-mounted terminal can maintain a listening state. For example, the vehicle-mounted smart device of the vehicle-mounted terminal can maintain a listening state to receive user voice control, and send the received user voice control to the cloud. At the same time, text information corresponding to the changes in the content of the first screen can also be sent to the cloud via the vehicle-mounted terminal. After receiving the user voice control and the text information corresponding to the changes in the content of the first screen, the cloud can determine whether the user voice control matches the text information corresponding to the changes in the content of the first screen, and send the result indicating that the user voice control matches the text information corresponding to the changes in the content of the first screen to the vehicle-mounted terminal, and the vehicle-mounted terminal can send the result to the mobile terminal. Exemplarily, the cloud can be implemented using various existing voice recognition technologies or voice recognition technologies that will appear in the future.

[0039] In one embodiment, during the period when the mobile terminal enters the voice control scenario, the vehicle-mounted terminal can maintain a listening state, for example, the vehicle-mounted smart device of the vehicle-mounted terminal can maintain a listening state to receive user voice control. At the same time, the vehicle-mounted terminal can also receive text information corresponding to the changes in the first screen content from the mobile terminal. After receiving the user voice control and the text information corresponding to the changes in the first screen content, the vehicle-mounted terminal can determine whether the user voice control matches the text information corresponding to the changes in the first screen content, and send the result indicating that the user voice control matches the text information corresponding to the changes in the first screen content to the mobile terminal. Exemplarily, the vehicle-mounted terminal can be implemented using various currently existing voice recognition technologies or voice recognition technologies that will appear in the future.

[0040] In step S105 , in response to receiving an instruction indicating that the user voice control from the vehicle-mounted terminal matches the text information, the first screen content is adjusted to the second screen content based on the text information matching the user voice control.

[0041] Optionally, in step S105, an instruction instructing the vehicle terminal to match the user voice control with the text information may be received from the vehicle terminal, and the matching instruction may be received by the vehicle terminal from the cloud or directly generated by the vehicle terminal.

[0042] Optionally, in step S105, in response to receiving an instruction indicating that the user voice control from the vehicle terminal matches the text information, the control corresponding to the text information can be determined, and the operation behavior of the control corresponding to the text information can be called to adjust the first screen content to the second screen content, so that the first screen content displayed on the vehicle screen is correspondingly switched to the second screen content corresponding to the user voice control. Taking the music application of the mobile terminal as an example, when playing music, the controls may include a play control, a pause control, a download control, a share control, a volume adjustment control, a lyrics display control, a loop setting control, a previous song selection control, a next song selection control, etc. When the user voice control "play next song" or other similar voice control from the vehicle terminal matches the "play next song" included in the acquired text information corresponding to the change of the first screen content, the operation behavior triggered by the next song selection control being clicked can be executed, so that the first screen content displayed on the vehicle screen is switched to the second screen content corresponding to "play next song". Taking the chat application of the mobile terminal as an example, when a video call is connected, the controls may include an answer control, a hang-up control, a switch voice call control, etc. When the user voice control "hang up" or other similar voice controls from the vehicle-mounted terminal matches the "hang up" included in the acquired text information corresponding to the change of the first screen content, the operation behavior triggered after the hang up control is clicked can be executed, so that the first screen content displayed on the vehicle screen is switched to the second screen content corresponding to "hang up".

[0043] It should be noted that the term "switch" in the context of this application should be broadly understood as various operations such as turning on, off, and updating to change the first screen content displayed on the vehicle screen. The control method for screen projection scenarios proposed in one aspect of this application can be applied to various applications on a mobile terminal, where the application can be a native application, such as settings, phone, camera, etc., and can also be a downloaded third-party application, such as video playback, chat tools, etc.

[0044] According to one aspect of the present application, the control method proposed for screen projection scenarios can control the screen content through user voice input in the scenario of screen projection from a mobile terminal to a vehicle, thereby realizing the control functions and display effects of the terminal screen consistent with non-screen projection scenarios, avoiding driving risks brought by touching the screen, and improving user experience and driving safety.

[0045] Figure 2 A flow chart of a control method applied to a screen projection scenario according to one or more embodiments of the present application is shown.

[0046] Figure 2The control method for screen projection according to one or more embodiments of the present application shown in the figure can be used to establish a communication connection between the vehicle-mounted terminal and the mobile terminal and display the first screen content projected from the mobile terminal on the vehicle-mounted terminal's vehicle screen. Optionally, the vehicle-mounted terminal and the mobile terminal can be authenticated based on the same account and establish a communication connection through near-field communication, and after the connection is successful, both parties can conduct safe and reliable near-field communication.

[0047] like Figure 2 As shown in, in step S201, a start instruction is received and sent to a mobile terminal, so that the mobile terminal enters a voice control scenario, wherein in the voice control scenario, the mobile terminal obtains text information corresponding to changes in the first screen content.

[0048] Optionally, in step S201, the in-vehicle intelligent device of the in-vehicle terminal can be awakened by the user to receive a start command input by the user and send the start command to the mobile terminal via near field communication. The mobile terminal can enter the voice control scene in response to receiving the start command.

[0049] In the voice control scenario, the vehicle terminal can also receive an end instruction and / or a vehicle control instruction and send the received end instruction and / or vehicle control instruction to the mobile terminal, so that the mobile terminal exits the voice control scenario. Exemplarily, the start instruction, the end instruction, and the vehicle control instruction can be generated based on the user's voice input. Exemplarily, the vehicle-mounted smart device can be installed in the approximate middle position of the dashboard of the vehicle to interact with the passengers and the driver in the cockpit, for example, information interaction, action interaction, etc. In one embodiment, when the vehicle-mounted terminal receives the vehicle control instruction "open the sunroof", the operation of opening the sunroof can be performed, and when the mobile terminal receives the vehicle control instruction "open the sunroof" from the vehicle-mounted terminal, the voice control scenario can be exited.

[0050] In step S203, user voice control and text information are received.

[0051] Optionally, in step S203, while the mobile terminal enters the voice control scenario, the vehicle-mounted terminal may remain in a listening state, for example, the vehicle-mounted intelligent device of the vehicle-mounted terminal may remain in a listening state to receive user voice control. At the same time, the vehicle-mounted terminal may also receive text information corresponding to changes in the first screen content from the mobile terminal. After receiving the user voice control and the text information corresponding to the changes in the first screen content, the vehicle-mounted terminal may determine whether the user voice control matches the text information of the changes in the first screen content, and send the result indicating that the user voice control matches the text information of the changes in the first screen content to the mobile terminal. Exemplarily, the vehicle-mounted terminal can be implemented using various currently existing voice recognition technologies or voice recognition technologies that will appear in the future.

[0052] Optionally, in step S203, while the mobile terminal enters the voice control scenario, the vehicle-mounted terminal can maintain a listening state, for example, the vehicle-mounted intelligent device of the vehicle-mounted terminal can maintain a listening state to receive user voice control, and send the received user voice control to the cloud. At the same time, the vehicle-mounted terminal can also receive text information corresponding to the changes in the content of the first screen and send the text information to the cloud. After receiving the user voice control and the text information corresponding to the changes in the content of the first screen, the cloud can determine whether the user voice control matches the text information of the changes in the content of the first screen, and send the result indicating that the user voice control matches the text information of the changes in the content of the first screen to the vehicle-mounted terminal, and the vehicle-mounted terminal can send the result to the mobile terminal. Exemplarily, the cloud can be implemented using various existing voice recognition technologies or voice recognition technologies that will appear in the future.

[0053] In step S205, in response to the mobile terminal adjusting the first screen content to the second screen content based on the text information matching the user voice control, the second screen content is displayed.

[0054] Optionally, in step S205, the mobile terminal can determine the control corresponding to the text information in response to receiving an instruction indicating that the user voice control from the vehicle-mounted terminal matches the text information, and call the operation behavior of the control corresponding to the text information to adjust the first screen content to the second screen content, so that the first screen content displayed on the vehicle screen is correspondingly switched to the second screen content corresponding to the user voice control.

[0055] Optionally, the control method applied to the screen projection scene according to one or more embodiments of the present application may also include controlling the vehicle-mounted intelligent terminal to execute a prompt operation (e.g., expression, action, voice prompt, etc.) in response to the display of the second screen content, and maintaining monitoring of one or more of the user voice control, end command and vehicle control command. When the user voice control is monitored, it can be determined whether the user voice control matches the text information corresponding to the change of the first screen content. The above-mentioned matching judgment process can be executed in the cloud or the vehicle-mounted terminal, and the command judged to be matched is sent to the mobile terminal; when the end command and / or the vehicle control command are monitored, the end command and / or the vehicle control command can be sent to the mobile terminal, so that the mobile terminal exits the voice control scene. For example, taking the chat application of a mobile terminal as an example, during a video call, when the user voice control "switch voice call" or other similar voice control from the vehicle-mounted terminal matches the "switch voice call" included in the acquired text information corresponding to the change of the first screen content, the operation behavior triggered by clicking the switch voice call control can be executed, so that the first screen content displayed on the vehicle screen is switched to the second screen content corresponding to "switch voice call", and at the same time, the vehicle-mounted intelligent terminal can be controlled to issue a voice prompt "It has been switched to voice call for you".

[0056] Figure 3 A flow chart of a control method applied to a screen projection scenario according to one or more embodiments of the present application is shown.

[0057] See also Figure 3 In step S1, a communication connection is established between the mobile terminal and the vehicle terminal and the first screen content of the mobile terminal is projected to the vehicle screen of the vehicle terminal for display. Optionally, the mobile terminal and the vehicle terminal can be authenticated based on the same account and establish a communication connection through near field communication. After the connection is successful, both parties can conduct safe and reliable near field communication.

[0058] In step S2, the vehicle terminal receives a start instruction and sends the received start instruction to the mobile terminal, so that the mobile terminal enters the voice control scene. Optionally, in step S2, the vehicle-mounted intelligent device of the vehicle terminal can be awakened by the user to receive the start instruction input by the user and send the start instruction to the mobile terminal through near field communication, and the mobile terminal can enter the voice control scene in response to receiving the start instruction. In the voice control scene, the vehicle terminal can also receive an end instruction and / or a vehicle control instruction and send the received end instruction and / or vehicle control instruction to the mobile terminal, so that the mobile terminal exits the voice control scene. Exemplarily, the start instruction, the end instruction, and the vehicle control instruction can be generated based on the user's voice input. In one embodiment, when the vehicle terminal receives the vehicle control instruction "open the sunroof", the operation of opening the sunroof can be performed, and when the mobile terminal receives the vehicle control instruction "open the sunroof" from the vehicle terminal, the voice control scene can be exited.

[0059] In step S3, the mobile terminal enters the voice control scene in response to receiving the start command from the vehicle-mounted terminal, and obtains text information corresponding to the change of the first screen content. Optionally, in step S3, the mobile terminal can detect the screen switching behavior and determine the active page with focus in the current task stack, traverse the root layout corresponding to the active page to obtain a text type control, and obtain information on text attributes in the text type control as the text information corresponding to the change of the first screen content.

[0060] In step S4, the mobile terminal sends the acquired text information corresponding to the change of the first screen content to the cloud.

[0061] In step S5, the vehicle-mounted terminal receives the user voice control and sends the user voice control to the cloud.

[0062] In step S6, the cloud determines whether the user voice control from the vehicle-mounted terminal matches the text information corresponding to the change in the first screen content.

[0063] In step S7, the cloud sends an instruction to the vehicle terminal indicating that the user voice control from the vehicle terminal matches the text information corresponding to the change of the first screen content.

[0064] As an alternative implementation of the above steps S4-S7, after acquiring the text information corresponding to the change of the first screen content in step S3, in step S4', the mobile terminal can send the acquired text information corresponding to the change of the first screen content to the vehicle terminal. In step S5', the vehicle terminal can determine whether the received user voice control matches the text information corresponding to the change of the first screen content, and if so, proceed to step S8.

[0065] In step S8, the vehicle terminal sends an instruction to the mobile terminal instructing the user voice control to match the text information corresponding to the change of the first screen content.

[0066] In step S9, the mobile terminal adjusts the first screen content to the second screen content based on the text information matching the user voice control in response to receiving the instruction indicating that the user voice control matches the text information. Optionally, the mobile terminal can determine the control corresponding to the text information in response to receiving the instruction indicating that the user voice control matches the text information, and call the operation behavior of the control corresponding to the text information to adjust the first screen content to the second screen content.

[0067] In step S10, the vehicle-mounted terminal can display the second screen content in response to the mobile terminal adjusting the first screen content to the second screen content based on text information matching the user voice control, and at the same time control the vehicle-mounted intelligent terminal to perform prompt operations in response to the user voice control (for example, expressions, actions, voice prompts, etc.), and keep monitoring one or more of the user voice control, end instructions and vehicle control instructions.

[0068] It should be noted that the above steps S1-S10 and their execution order are only illustrative. Without departing from the spirit and scope of the present application, one or more of steps S1-S10 may be omitted or combined into one step or the execution order may be changed. For example, step S5 may be before step S4, or step S5 and step S4 may be combined into one step. It should be noted that the above steps S3-S10 are repeatedly executed in a loop until an end instruction and / or a vehicle control instruction is received from the vehicle terminal to exit the voice control scene.

[0069] Figure 4 A block diagram of a control system applied to a screen projection scenario according to one or more embodiments of the present application is shown.

[0070] like Figure 4 As shown in the figure, the control system 40 applied to the screen projection scenario includes a communication unit 410, a memory 420 (for example, a non-volatile memory such as a flash memory, a ROM, a hard disk drive, a disk, an optical disk, etc.), a processor 430, and a computer program 440 stored on the memory 420 and executable on the processor 430.

[0071] The communication unit 410 serves as a communication interface and is configured to establish a communication connection between the control system 40 applied to the screen projection scenario and an external device or network (for example, a cloud server, a vehicle-mounted terminal, etc.).

[0072] The memory 420 stores a computer program 440 executable by the processor 430. In addition, the memory 420 may also store data generated when the processor 430 executes the computer program and data or commands (eg, text information, start instructions, etc.) received from the outside via the communication unit 410.

[0073] The processor 430 is configured to execute the computer program 440 to implement a control method applied to a screen projection scenario according to one or more embodiments of the present application.

[0074] Figure 5 A block diagram of a control system applied to a screen projection scenario according to one or more embodiments of the present application is shown.

[0075] like Figure 5 As shown in the figure, the control system 50 applied to the screen projection scenario includes a communication unit 510, a memory 520 (for example, a non-volatile memory such as a flash memory, a ROM, a hard disk drive, a disk, an optical disk, etc.), a processor 530, and a computer program 540 stored on the memory 520 and executable on the processor 530.

[0076] The communication unit 510 serves as a communication interface and is configured to establish a communication connection between the control system 40 applied to the screen projection scenario and an external device or network (for example, a cloud server, a mobile terminal, etc.).

[0077] The memory 520 stores a computer program 540 executable by the processor 530. In addition, the memory 520 may also store data generated when the processor 530 executes the computer program and data or commands (eg, user voice control, start instruction, etc.) received from the outside via the communication unit 510.

[0078] The processor 530 is configured to execute a computer program 540 to implement a control method applied to a screen projection scenario according to one or more embodiments of the present application.

[0079] In the applicable situation, hardware, software or a combination of hardware and software can be used to realize the various embodiments provided by the application. Moreover, in the applicable situation, without departing from the scope of the application, the various hardware components and / or software components set forth herein can be combined into a composite component comprising software, hardware and / or both. In the applicable situation, without departing from the scope of the application, the various hardware components and / or software components set forth herein can be divided into subcomponents comprising software, hardware or both. In addition, in the applicable situation, it is contemplated that the software component can be implemented as a hardware component, and vice versa.

[0080] Software (such as program code and / or data) according to the present application can be stored on one or more computer storage media. It is also contemplated that the software identified herein can be implemented using one or more general or special-purpose computers and / or computer systems, networked and / or otherwise. Where applicable, the order of the various steps described herein can be changed, combined into composite steps, and / or divided into sub-steps to provide the features described herein.

[0081] The embodiments and examples set forth herein are provided to best illustrate embodiments according to the present application and its specific applications, and thereby enable those skilled in the art to implement and use the present application. However, those skilled in the art will appreciate that the above description and examples are provided only for ease of illustration and example. The description set forth is not intended to cover all aspects of the present application or to limit the present application to the precise form disclosed.

[0082] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.

[0083] The user personal information processed by the applicant will vary depending on the specific product / service scenario, and shall be based on the specific scenario in which the user uses the product / service, which may involve the user's account information, device information, driving information, vehicle information or other related information. The applicant will treat the user's personal information and its processing with a high degree of diligence.

[0084] The Applicant attaches great importance to the security of user personal information and has adopted reasonable and feasible security protection measures that comply with industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.

Claims

1. A control method applied to a screen projection scene, characterized in that: The control method is used to establish a communication connection between a mobile terminal and a vehicle-mounted terminal and project the first screen content of the mobile terminal onto the vehicle-mounted screen for display. The control method comprises the following steps: Entering a voice control scenario in response to receiving a start instruction from the vehicle-mounted terminal; Acquiring text information corresponding to the change of the first screen content; and In response to receiving an instruction indicating that the user voice control from the in-vehicle terminal matches the text information, the first screen content is adjusted to the second screen content based on the text information matching the user voice control.

2. The control method according to claim 1, wherein the mobile terminal establishes a communication connection with the vehicle-mounted terminal via near field communication; and / or The control method further includes exiting the voice control scenario in response to receiving an end instruction and / or a vehicle control instruction from the vehicle-mounted terminal. 3 . The control method according to claim 1 , wherein one or more of a start instruction, an end instruction, and a vehicle control instruction are generated based on a user's voice input. 4 .

4. The control method according to claim 1, wherein acquiring text information corresponding to the change of the first screen content comprises: Detect screen switching behavior and determine the active page with focus in the current task stack; Traverse the root layout corresponding to the active page to obtain a text type control; as well as The information of the text attribute in the text type control is obtained as the text information corresponding to the change of the first screen content.

5. The control method according to claim 4, wherein detecting the screen switching behavior comprises: The page user interface tree is monitored by an activity management service to detect the screen switching behavior.

6. The control method according to claim 1, wherein in response to receiving an instruction indicating that a user voice control from the vehicle-mounted terminal matches the text information, adjusting the first screen content to the second screen content based on the text information matching the user voice control comprises: determining a control corresponding to the text information in response to receiving an instruction indicating that the user voice control from the vehicle-mounted terminal matches the text information; as well as An operation behavior of a control corresponding to the text information is called to adjust the first screen content to the second screen content.

7. A control method applied to a screen projection scene, characterized in that: The control method is used to establish a communication connection between the vehicle-mounted terminal and the mobile terminal and display the first screen content projected from the mobile terminal on the vehicle screen, and the control method includes the following steps: receiving a start instruction and sending the start instruction to a mobile terminal, so that the mobile terminal enters a voice control scenario, wherein in the voice control scenario, the mobile terminal obtains text information corresponding to changes in the first screen content; receiving user voice control and the text message; and The second screen content is displayed in response to the mobile terminal adjusting the first screen content to the second screen content based on the text information matching the user voice control.

8. The control method according to claim 7, wherein the control method further comprises: An instruction indicating that the user voice control matches the text information is generated or received, and an instruction indicating that the user voice control matches the text information is sent to the mobile terminal.

9. The control method according to claim 7, wherein the vehicle-mounted terminal establishes a communication connection with the mobile terminal via near field communication; and / or The control method further includes instructing the mobile terminal to exit the voice control scenario in response to receiving an end instruction and / or a vehicle control instruction. 10 . The control method according to claim 7 , wherein one or more of a start command, an end command, and a vehicle control command are generated based on a user's voice input.

11. The control method according to claim 7 or 9, wherein the control method further comprises: In response to displaying the second screen content, controlling the in-vehicle intelligent terminal to execute a prompting operation in response to the user voice control; as well as Maintain the monitoring state to receive one or more of user voice control, end command and vehicle control command.

12. A control system applied to screen projection scenarios, characterized in that: The system comprises: Memory; a processor coupled to the memory; and A computer program stored on the memory and running on the processor, wherein the running of the computer program results in the execution of the control method applied to the screen projection scene according to any one of claims 1-6 or 7-11.

13. A computer-readable storage medium, characterized in that: The computer storage medium includes instructions, which, when running, execute the control method applied to the screen projection scene according to any one of claims 1-6 or 7-11.

Images